Navigation in virtual and real environment using brain computer interface:a progress report

Abstract

A brain-computer interface (BCI) facilitates bypassing the peripheral nervous system and directly communicating with surrounding devices. Navigation technology using BCI has developed—from exploring the prototype paradigm in the virtual environment (VE) to accurately completing the locomotion intention of the operator in the form of a powered wheelchair or mobile robot in a real environment. This paper summarizes BCI navigation applications that have been used in both real and VEs in the past 20 years. Horizontal comparisons were conducted between various paradigms applied to BCI and their unique signal-processing methods. Owing to the shift in the control mode from synchronous to asynchronous, the development trend of navigation applications in the VE was also reviewed. The contrast between high-level commands and low-level commands is introduced as the main line to review the two major applications of BCI navigation in real environments: mobile robots and unmanned aerial vehicles (UAVs). Finally, applications of BCI navigation to scenarios outside the laboratory; research challenges, including human factors in navigation application interaction design; and the feasibility of hybrid BCI for BCI navigation are discussed in detail.

Keyword

Brain-computer interface ; Virtual reality ; Human-computer interface ; Navigation ; Motor imagery ; Steady-state visual evoked potential

Cite this article

Haochen HU, Yue LIU, Kang YUE, Yongtian WANG. Navigation in virtual and real environment using brain computer interface:a progress report. Virtual Reality & Intelligent Hardware, 2022, 4(2): 89-114 DOI:10.1016/j.vrih.2021.10.002

References

1. Nicolas-Alonso L F, Gomez-Gil J. Brain computer interfaces, a review. Sensors (Basel, Switzerland), 2012, 12(2): 1211–1279 DOI:10.3390/s120201211

2. Beverina F, Palmas G, Silvoni S, Piccione F, Giove S. User adaptive BCIs: SSVEP and P300 based interfaces. PsychNology Journal, 2003, 1(4): 331–354

3. Wang Y J, Wang R P, Gao X R, Hong B, Gao S K. A practical VEP-based brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2006, 14(2): 234–240 DOI:10.1109/tnsre.2006.875576

4. Galán F, Nuttin M, Lew E, Ferrez P W, Vanacker G, Philips J, Millán J D R. A brain-actuated wheelchair: Asynchronous and non-invasive Brain-computer interfaces for continuous control of robots. Clinical Neurophysiology, 2008, 119(9): 2159–2169 DOI:10.1016/j.clinph.2008.06.001

5. Iturrate I, Antelis J M, Kubler A, Minguez J. A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation. IEEE Transactions on Robotics, 2009, 25(3): 614–627 DOI:10.1109/tro.2009.2020347

6. Tsui C S, Gan J Q, Hu H. A self-paced motor imagery based brain-computer interface for robotic wheelchair control. Clinical EEG and Neuroscience, 2011, 42(4): 225–229 DOI:10.1177/155005941104200407

7. Lopes A C, Pires G, Nunes U. RobChair: Experiments evaluating Brain-Computer Interface to steer a semi-autonomous wheelchair. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. Vilamoura-Algarve, Portugal, IEEE, 2012, 5135–5136 DOI:10.1109/iros.2012.6386276

8. Piña-Ramirez O, Valdes-Cristerna R, Yanez-Suarez O. Scenario screen: a dynamic and context dependent P300 stimulator screen aimed at wheelchair navigation control. Computational and Mathematical Methods in Medicine, 2018, 2018: 7108906 DOI:10.1155/2018/7108906

9. Escolano C, Ramos Murguialday A, Matuz T, Birbaumer N, Minguez J. A telepresence robotic system operated with a P300-based brain-computer interface: Initial tests with ALS patients. 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 2010, 4476–4480 DOI:10.1109/iembs.2010.5626045

10. Vourvopoulos A, Ferreira A, Badia S B I. NeuRow: an immersive VR environment for motor-imagery training with the use of brain-computer interfaces and vibrotactile feedback. In: Proceedings of the 3rd International Conference on Physiological Computing Systems. Lisbon, Portugal, SCITEPRESS—Science and Technology Publications, 2016, 43–53 DOI:10.5220/0005939400430053

11. Leeb R, Settgast V, Fellner D, Pfurtscheller G. Self-paced exploration of the Austrian National Library through thought. International Journal of Bioelectromagnetism, 2007, 9(4): 237–244

12. Leeb R, Keinrath C, Friedman D, Guger C, Scherer R, Neuper C, Garau M, Antley A, Steed A, Slater M, Pfurtscheller G. Walking by thinking: the brainwaves are crucial, not the muscles!. Presence: Teleoperators and Virtual Environments, 2006, 15(5): 500–514 DOI:10.1162/pres.15.5.500

13. Pfurtscheller G, Leeb R, Keinrath C, Friedman D, Neuper C, Guger C, Slater M. Walking from thought. Brain Research, 2006, 1071(1): 145–152 DOI:10.1016/j.brainres.2005.11.083

14. Leeb R, Scherer R, Keinrath C, Guger C, Pfurtscheller G. Exploring virtual environments with an EEG-based BCI through motor imagery. Biomedizinische Technik Biomedical Engineering, 2005, 50(4): 86–91 DOI:10.1515/BMT.2005.012

15. Shin B G, Kim T, Jo S. Non-invasive brain signal interface for a wheelchair navigation. In: ICCAS 2010. Gyeonggi-do, Korea(South), IEEE, 2010, 2257–2260 DOI:10.1109/iccas.2010.5669830

16. Hema C R, Paulraj M P. Control brain machine interface for a power wheelchair. 5th Kuala Lumpur International Conference on Biomedical Engineering 2011, 2011, 287–291 DOI:10.1007/978-3-642-21729-6_75

17. Chai R, Ling S H, Hunter G P, Nguyen H T. Mental non-motor imagery tasks classifications of brain computer interface for wheelchair commands using genetic algorithm-based neural network. In: The 2012 International Joint Conference on Neural Networks (IJCNN). Brisbane, QLD, Australia, IEEE, 2012, 1–7 DOI:10.1109/ijcnn.2012.6252499

18. Duan J D, Li Z J, Yang C G, Xu P. Shared control of a brain-actuated intelligent wheelchair. In: Proceeding of the 11th World Congress on Intelligent Control and Automation. Shenyang, China, IEEE, 2014, 341–346 DOI:10.1109/wcica.2014.7052737

19. Jiang J, Wang A, Ge Y, Zhou Z T. Brain-actuated humanoid robot control using one class motor imagery task. In: 2013 Chinese Automation Congress. Changsha, China, IEEE, 2013, 587–590 DOI:10.1109/cac.2013.6775803

20. Varona-Moya S, Velasco-Álvarez F, Sancha-Ros S, Fernández-Rodríguez Á, Blanca M J, Ron-Angevin R. Wheelchair navigation with an audio-cued, two-class motor imagery-based brain-computer interface system. 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), 2015, 174–177 DOI:10.1109/ner.2015.7146588

21. Ron-Angevin R, Fernández-Rodríguez Á, Velasco-Álvarez F. Brain-controlled wheelchair through discrimination of two mental tasks. Proceedings of SAI Intelligent Systems Conference (IntelliSys) 2016, 2018, 563–574 DOI:10.1007/978-3-319-56994-9_38

22. Puanhvuan D, Khemmachotikun S, Wechakarn P, Wijarn B, Wongsawat Y. Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cognitive Neurodynamics, 2017, 11(2): 117–134 DOI:10.1007/s11571-017-9424-6

23. Farmaki C, Christodoulakis G, Sakkalis V. Applicability of SSVEP-based brain-computer interfaces for robot navigation in real environments. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando, FL, USA, IEEE, 2016, 2768–2771 DOI:10.1109/embc.2016.7591304

24. Kucukyildiz G, Ocak H, Karakaya S, Sayli O. Design and implementation of a multi sensor based brain computer interface for a robotic wheelchair. Journal of Intelligent & Robotic Systems, 2017, 87(2): 247–263 DOI:10.1007/s10846-017-0477-x

25. Malete T N, Moruti K, Thapelo T S, Jamisola R S. EEG-based control of a 3D game using 14-channel emotiv epoc+. In: 2019 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM). Bangkok, Thailand, IEEE, 2019, 463–468 DOI:10.1109/cis-ram47153.2019.9095807

26. Lamti H A, Gorce P, Ben Khelifa M M, Alimi A M. When mental fatigue maybe characterized by Event Related Potential (P300) during virtual wheelchair navigation. Computer Methods in Biomechanics and Biomedical Engineering, 2016, 19(16): 1749–1759 DOI:10.1080/10255842.2016.1183198

27. Hazrati M K, Hofmann U G. Avatar navigation in Second Life using brain signals. In: 2013 IEEE 8th International Symposium on Intelligent Signal Processing. Funchal, Portugal, IEEE, 2013, 1–7 DOI:10.1109/wisp.2013.6657473

28. Congedo M, Lotte F, Lécuyer A. Classification of movement intention by spatially filtered electromagnetic inverse solutions. Physics in Medicine and Biology, 2006, 51(8): 1971–1989 DOI:10.1088/0031-9155/51/8/002

29. Lotte F, Lecuyer A, Arnaldi B. FuRIA: an inverse solution based feature extraction algorithm using fuzzy set theory for brain-computer interfaces. IEEE Transactions on Signal Processing, 2009, 57(8): 3253–3263 DOI:10.1109/tsp.2009.2020752

30. Rivet B, Souloumiac A, Attina V, Gibert G. xDAWN algorithm to enhance evoked potentials: application to brain–computer interface. IEEE Transactions on Biomedical Engineering, 2009, 56(8): 2035–2043 DOI:10.1109/tbme.2009.2012869

31. Wang H T, Bezerianos A. Brain-controlled wheelchair controlled by sustained and brief motor imagery BCIs. Electronics Letters, 2017, 53(17): 1178–1180 DOI:10.1049/el.2017.1637

32. Yu Y, Liu Y D, Jiang J, Yin E W, Zhou Z T, Hu D W. An asynchronous control paradigm based on sequential motor imagery and its application in wheelchair navigation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(12): 2367–2375 DOI:10.1109/tnsre.2018.2881215

33. Chae Y, Jo S, Jeong J. Brain-actuated humanoid robot navigation control using asynchronous Brain-Computer Interface. In: 2011 5th International IEEE/EMBS Conference on Neural Engineering. Cancun, Mexico, IEEE, 2011, 519–524 DOI:10.1109/ner.2011.5910600

34. Zhang R, Li Y Q, Yan Y Y, Zhang H, Wu S Y. An intelligent wheelchair based on automated navigation and BCI techniques. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014, 1302–1305 DOI:10.1109/embc.2014.6943837

35. Lin Y T, Kuo C H. Development of SSVEP-based intelligent wheelchair brain computer interface assisted by reactive obstacle avoidance. In: 2016 IEEE International Conference on Industrial Technology (ICIT). Taipei, Taiwan, China, IEEE, 2016, 1572–1577 DOI:10.1109/icit.2016.7474995

36. Woehrle H, Krell M M, Straube S, Kim S K, Kirchner E A, Kirchner F. An adaptive spatial filter for user-independent single trial detection of event-related potentials. IEEE Transactions on Biomedical Engineering, 2015, 62(7): 1696–1705 DOI:10.1109/tbme.2015.2402252

37. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F. A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update. Journal of Neural Engineering, 2018, 15(3): 031005 DOI:10.1088/1741-2552/aab2f2

38. Kalunga E K, Chevallier S, Barthélemy Q, Djouani K, Monacelli E, Hamam Y. Online SSVEP-based BCI using Riemannian geometry. Neurocomputing, 2016, 191: 55–68 DOI:10.1016/j.neucom.2016.01.007

39. Lawhern V J, Solon A J, Waytowich N R, Gordon S M, Hung C P, Lance B J. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 2018, 15(5): 056013 DOI:10.1088/1741-2552/aace8c

40. Kwak N S, Müller K R, Lee S W. A convolutional neural network for steady state visual evoked potential classification under ambulatory environment. PLoS One, 2017, 12(2): e0172578 DOI:10.1371/journal.pone.0172578

41. Schirrmeister R, Gemein L, Eggensperger K, Hutter F, Ball T. Deep learning with convolutional neural networks for decoding and visualization of EEG pathology. 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), 2017, 1–7 DOI:10.1109/spmb.2017.8257015

42. Sannelli C, Vidaurre C, Müller K R, Blankertz B. A large scale screening study with a SMR-based BCI: Categorization of BCI users and differences in their SMR activity. PLoS One, 2019, 14(1): e0207351 DOI:10.1371/journal.pone.0207351

43. Ahn M, Jun S C. Performance variation in motor imagery brain-computer interface: a brief review. Journal of Neuroscience Methods, 2015, 243, 103–110 DOI:10.1016/j.jneumeth.2015.01.033

44. Gayraud N T H, Rakotomamonjy A, Clerc M. Optimal transport applied to transfer learning for P300 detection. BCI 2017-7th Graz Brain-Computer Interface Conference, 2017, 6

45. Hsu W Y. EEG-based motor imagery classification using enhanced active segment selection and adaptive classifier. Computers in Biology and Medicine, 2011, 41(8): 633–639 DOI:10.1016/j.compbiomed.2011.05.014

46. Wu Z H, Lai Y X, Xia Y, Wu D, Yao D Z. Stimulator selection in SSVEP-based BCI. Medical Engineering & Physics, 2008, 30(8): 1079–1088 DOI:10.1016/j.medengphy.2008.01.004

47. Stamps K, Hamam Y. Towards inexpensive BCI control for wheelchair navigation in the enabled environment-A hardware survey. Brain Informatics, 2010, 336–345 DOI:10.1007/978-3-642-15314-3_32

48. Xu M P, Xiao X L, Wang Y J, Qi H Z, Jung T P, Ming D. A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Transactions on Biomedical Engineering, 2018, 65(5): 1166–1175 DOI:10.1109/tbme.2018.2799661

49. Diez P F, Mut V A, Laciar E, Perona E M A. Mobile robot navigation with a self-paced brain-computer interface based on high-frequency SSVEP. Robotica, 2014, 32(5): 695–709 DOI:10.1017/s0263574713001021

50. Farwell L A, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology, 1988, 70(6): 510–523 DOI:10.1016/0013-4694(88)90149-6

51. Nawroj A I, Wang S Y, Yu Y C, Gabel L. A brain-computer interface for robot navigation. In: 2012 38th Annual Northeast Bioengineering Conference (NEBEC). Philadelphia, PA, USA, IEEE, 2012: 15–16 DOI:10.1109/nebc.2012.6206941

52. Escolano C, Antelis J M, Minguez J. A telepresence mobile robot controlled with a noninvasive brain-computer interface. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(3): 793–804 DOI:10.1109/tsmcb.2011.2177968

53. Ron-Angevin R, Díaz-Estrella A, Velasco-Alvarez F. A two-class brain computer interface to freely navigate through virtual worlds. Biomedizinische Technik. Biomedical Engineering, 2009, 54(3): 126–133 DOI:10.1515/bmt.2009.014

54. Edlinger G, Holzner C, Guger C, Groenegress C, Slater M. Brain-computer interfaces for goal orientated control of a virtual smart home environment. In: 2009 4th International IEEE/EMBS Conference on Neural Engineering. Antalya, Turkey, IEEE, 2009, 463–465 DOI:10.1109/ner.2009.5109333

55. Gentiletti G G, Gebhart J G, Acevedo R C, Yáñez-Suárez O, Medina-Bañuelos V. Command of a simulated wheelchair on a virtual environment using a brain-computer interface. IRBM, 2009, 30(5/6): 218–225 DOI:10.1016/j.irbm.2009.10.006

56. Jayaram V, Alamgir M, Altun Y, Scholkopf B, Grosse-Wentrup M. Transfer learning in brain-computer interfaces. IEEE Computational Intelligence Magazine, 2016, 11(1): 20–31 DOI:10.1109/mci.2015.2501545

57. Kindermans P J, Schreuder M, Schrauwen B, Müller K R, Tangermann M. True zero-training brain-computer interfacing: an online study. PLoS One, 2014, 9(7): e102504 DOI:10.1371/journal.pone.0102504

58. Geng T, Dyson M, Tsui C S, Gan J Q. A 3-class asynchronous BCI controlling A simulated mobile robot. In: 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Lyon, France, IEEE, 2007, 2524–2527 DOI:10.1109/iembs.2007.4352842

59. Velasco-Álvarez F, Ron-Angevin R. Asynchronous brain-computer interface to navigate in virtual environments using one motor imagery. Bio-Inspired Systems: Computational and Ambient Intelligence, 2009, 698–705 DOI:10.1007/978-3-642-02478-8_87

60. Leeb R, Lee F, Keinrath C, Scherer R, Bischof H, Pfurtscheller G. Brain-computer communication: motivation, aim, and impact of exploring a virtual apartment. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2007, 15(4): 473–482 DOI:10.1109/tnsre.2007.906956

61. Friedman D, Leeb R, Guger C, Steed A, Pfurtscheller G, Slater M. Navigating virtual reality by thought: what is it like? Presence: Teleoperators and Virtual Environments, 2007, 16(1): 100–110 DOI:10.1162/pres.16.1.100

62. Zich C, Debener S, Kranczioch C, Bleichner M G, Gutberlet I, De Vos M. Real-time EEG feedback during simultaneous EEG-fMRI identifies the cortical signature of motor imagery. NeuroImage, 2015, 114, 438–447 DOI:10.1016/j.neuroimage.2015.04.020

63. Jeannerod M. Mental imagery in the motor context. Neuropsychologia, 1995, 33(11): 1419–1432 DOI:10.1016/0028-3932(95)00073-c

64. Pineda J A, Silverman D S, Vankov A, Hestenes J. Learning to control brain rhythms: making a brain-computer interface possible. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2003, 11(2): 181–184 DOI:10.1109/tnsre.2003.814445

65. Bayliss J D. Use of the evoked potential P3 component for control in a virtual apartment. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2003, 11(2): 113–116 DOI:10.1109/tnsre.2003.814438

66. Suh D, Cho H S, Goo J, Park K S, Hahn M. Virtual navigation system for the disabled by motor imagery. Advances in Computer, Information, and Systems Sciences, and Engineering. Springer, Dordrecht, 2006, 143–148 DOI:10.1007/1-4020-5261-8_24

67. Leeb R, Friedman D, Müller-Putz G R, Scherer R, Slater M, Pfurtscheller G. Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic. Computational Intelligence and Neuroscience, 2007, 1–8 DOI:10.1155/2007/79642

68. Tsui C S L, Gan J Q. Asynchronous BCI control of a robot simulator with supervised online training. Intelligent Data Engineering and Automated Learning, Springer, Berlin, Heidelberg, 2007, 125–134 DOI:10.1007/978-3-540-77226-2_14

69. Fujisawa J, Touyama H, Hirose M. EEG-based navigation of immersing virtual environment using common spatial patterns. In: 2008 IEEE Virtual Reality Conference. Reno, NV, USA, IEEE, 2008, 251–252 DOI:10.1109/vr.2008.4480786

70. Scherer R, Lee F, Schlogl A, Leeb R, Bischof H, Pfurtscheller G. Toward self-paced brain-computer communication: navigation through virtual worlds. IEEE Transactions on Biomedical Engineering, 2008, 55(2): 675–682 DOI:10.1109/tbme.2007.903709

71. Lu W, Wei Y N, Yuan J X, Deng Y M, Song A G. Tractor assistant driving control method based on EEG combined with RNN-TL deep learning algorithm. IEEE Access, 2020, 8, 163269–163279 DOI:10.1109/access.2020.3021051

72. Eleni A. Control of medical robotics and neurorobotic prosthetics by noninvasive Brain-Robot Interfaces via EEG and RFID technology. In: 2008 8th IEEE International Conference on BioInformatics and BioEngineering. Athens, Greece, IEEE, 2008, 1–4 DOI:10.1109/bibe.2008.4696838

73. Wang F, Zhou C C, Hao X, Wang S, Yang G D. BCI control system for humanoid robot based on motor imaginary. In: 2013 25th Chinese Control and Decision Conference (CCDC). Guiyang, China, IEEE, 2013, 5140–5143 DOI:10.1109/ccdc.2013.6561868

74. Chin Z Y, Ang K K, Wang C C, Guan C T. Navigation in a virtual environment using multiclass motor imagery Brain-Computer Interface. In: 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB). Singapore, IEEE, 2013, 152–157 DOI:10.1109/ccmb.2013.6609179

75. Scherer R, Friedrich E C V, Allison B, Pröll M, Chung M, Cheung W, Neuper C. Non-invasive brain-computer interfaces: Enhanced gaming and robotic control. In: International Work-Conference on Artificial Neural Networks. Springer, Berlin, Heidelberg, 2011, 362–369

76. Velasco-Álvarez F, Ron-Angevin R, da Silva-Sauer L, Sancha-Ros S, Blanca-Mena M J. Audio-cued SMR brain-computer interface to drive a virtual wheelchair. In: Advances in Computational Intelligence. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, 337–344 DOI:10.1007/978-3-642-21501-8_42

77. Müller S M T, Diez P F, Bastos-Filho T F, Sarcinelli-Filho M, Mut V, Laciar E, Avila E. Robotic wheelchair commanded by people with disabilities using low/high-frequency SSVEP-based BCI. In: World Congress on Medical Physics and Biomedical Engineering. Toronto, Canada, 2015, 1177–1180 DOI:10.1007/978-3-319-19387-8_285

78. Diez P F, Mut V A, Laciar E, Perona E M A. Mobile robot navigation with a self-paced brain-computer interface based on high-frequency SSVEP. Robotica, 2014, 32(5): 695–709 DOI:10.1017/s0263574713001021

79. Chen S C, Chen Y J, Zaeni I A E, Wu C M. A single-channel SSVEP-based BCI with a fuzzy feature threshold algorithm in a maze game. International Journal of Fuzzy Systems, 2017, 19(2): 553–565 DOI:10.1007/s40815-016-0289-3

80. Liu Y L, Li J J, Li Z J. An indoor navigation control strategy for a brain-actuated mobile robot. In: 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM). Singapore, Singapore, IEEE, 2018, 13–18 DOI:10.1109/icarm.2018.8610705

81. Yuan Y X, Li Z J, Liu Y L. Brain teleoperation of a mobile robot using deep learning technique. In: 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM). Singapore, Singapore, IEEE, 2018, 54–59 DOI:10.1109/icarm.2018.8610711

82. Liu Y, Li Z, Zhang T, Zhao S. Brain-robot interface-based navigation control of a mobile robot in corridor environments. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018, 50(8): 3047–3058 DOI:10.1109/TSMC.2018.2833857

83. Farmaki C, Krana M, Pediaditis M, Spanakis E, Sakkalis V. Single-channel SSVEP-based BCI for robotic car navigation in real world conditions. In: 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE). Athens, Greece, IEEE, 2019, 638–643 DOI:10.1109/bibe.2019.00120

84. Koo B, Lee H G, Nam Y, Choi S. Immersive BCI with SSVEP in VR head-mounted display. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Milan, Italy, IEEE, 2015, 1103–1106 DOI:10.1109/embc.2015.7318558

85. Bevilacqua V, Tattoli G, Buongiorno D, Loconsole C, Leonardis D, Barsotti M, Frisoli A, Bergamasco M. A novel BCI-SSVEP based approach for control of walking in Virtual Environment using a Convolutional Neural Network. In: 2014 International Joint Conference on Neural Networks (IJCNN). Beijing, China, IEEE, 2014, 4121–4128 DOI:10.1109/ijcnn.2014.6889955

86. Diez P F, Mut V A, Avila Perona E M, Laciar Leber E. Asynchronous BCI control using high-frequency SSVEP. Journal of NeuroEngineering and Rehabilitation, 2011, 8(1): 1–9 DOI:10.1186/1743-0003-8-39

87. Chung M, Cheung W, Scherer R, Rao R P N. Towards hierarchical BCIs for robotic control. In: 2011 5th International IEEE/EMBS Conference on Neural Engineering. Cancun, Mexico, IEEE, 2011, 330–333 DOI:10.1109/ner.2011.5910554

88. Legény J, Abad R V, Lécuyer A. Navigating in virtual worlds using a self-paced SSVEP-based brain-computer interface with integrated stimulation and real-time feedback. Presence: Teleoperators and Virtual Environments, 2011, 20(6): 529–544 DOI:10.1162/pres_a_00075

89. Annese V F, Mezzina G, De Venuto D. Wireless Brain-computer interface for wheelchair control by using fast machine learning and real-time hyper-dimensional classification. In: International Conference on Web Engineering. Springer, Cham, 2017, 61–74 DOI:10.1007/978-3-319-74433-9_5

90. Yu Y C, Nawroj A, Wang S Y, Gabel L. Mobile robot navigation through a brain computer interface. In: 2012 IEEE Signal Processing in Medicine and Biology Symposium (SPMB). New York, NY, USA, IEEE, 2012, 1–5 DOI:10.1109/spmb.2012.6469469

91. Curtin A, Ayaz H, Liu Y C, Shewokis P A, Onaral B. A P300-based EEG-BCI for spatial navigation control. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 3841–3844 DOI:10.1109/embc.2012.6346805

92. Lopes A C, Pires G, Vaz L, Nunes U. Wheelchair navigation assisted by Human-Machine shared-control and a P300-based Brain Computer Interface. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, CA, USA, IEEE, 2011, 2438–2444 DOI:10.1109/iros.2011.6094748

93. Cherubini A, Oriolo G, Macri F, Aloise F, Babiloni F, Cincotti F, Mattia D. Development of a multimode navigation system for an assistive robotics project. In: Proceedings 2007 IEEE International Conference on Robotics and Automation. Rome, Italy, IEEE, 2007, 2336–2342 DOI:10.1109/robot.2007.363668

94. Diez P F, Mut V A, Laciar E, Perona E M A. Mobile robot navigation with a self-paced brain-computer interface based on high-frequency SSVEP. Robotica, 2014, 32(5): 695–709 DOI:10.1017/s0263574713001021

95. Yuan Y X, Su W B, Li Z J, Shi G M. Brain-computer interface-based stochastic navigation and control of a semiautonomous mobile robot in indoor environments. IEEE Transactions on Cognitive and Developmental Systems, 2019, 11(1): 129–141 DOI:10.1109/tcds.2018.2885774

96. Zhang Z W, Wang W J, Song P P, Sheng S L, Xie L Y, Duan F, GuanSoo Y, Odagaki M. Design of an SSVEP-based BCI system with vision assisted navigation module for the cooperative control of multiple robots. In: 2017 IEEE 7th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). Honolulu, HI, USA, IEEE, 2017, 558–563 DOI:10.1109/cyber.2017.8446149

97. Nourmohammadi A, Jafari M, Zander T O. A survey on unmanned aerial vehicle remote control using brain-computer interface. IEEE Transactions on Human-Machine Systems, 2018, 48(4), 337–348 DOI:10.1109/thms.2018.2830647

98. Akce A, Johnson M, Bretl T. Remote teleoperation of an unmanned aircraft with a brain-machine interface: Theory and preliminary results. In: 2010 IEEE International Conference on Robotics and Automation. Anchorage, AK, USA, IEEE, 2010, 5322–5327 DOI:10.1109/robot.2010.5509671

99. LaFleur K, Cassady K, Doud A, Shades K, Rogin E, He B. Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain-computer interface. Journal of Neural Engineering, 2013, 10(4): 046003 DOI:10.1088/1741-2560/10/4/046003

100. Kim B H, Kim M, Jo S. Quadcopter flight control using a low-cost hybrid interface with EEG-based classification and eye tracking. Computers in Biology and Medicine, 2014, 51, 82–92 DOI:10.1016/j.compbiomed.2014.04.020

101. Royer A S, Doud A J, Rose M L, He B. EEG control of a virtual helicopter in 3-dimensional space using intelligent control strategies. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2010, 18(6): 581–589 DOI:10.1109/tnsre.2010.2077654

102. Doud A J, Lucas J P, Pisansky M T, He B. Continuous three-dimensional control of a virtual helicopter using a motor imagery based brain-computer interface. PLoS One, 2011, 6(10): e26322 DOI:10.1371/journal.pone.0026322

103. Alrajhi W, Hosny M, Al-Wabil A, Alabdulkarim A. Human factors in the design of BCI-controlled wheelchairs. In: Human-Computer Interaction. Advanced Interaction Modalities and Techniques. Cham: Springer International Publishing, 2014, 513–522 DOI:10.1007/978-3-319-07230-2_49

104. Al Zayer M, MacNeilage P, Folmer E. Virtual locomotion: a survey. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(6): 2315–2334 DOI:10.1109/tvcg.2018.2887379

105. Long J Y, Li Y Q, Wang H T, Yu T Y, Pan J H, Li F. A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2012, 20(5): 720–729 DOI:10.1109/tnsre.2012.2197221

106. Cao L, Li J, Ji H F, Jiang C J. A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control. Journal of Neuroscience Methods, 2014, 229, 33–43 DOI:10.1016/j.jneumeth.2014.03.011

107. Li J, Ji H, Cao L, Zang D, Gu R, Xia B, Wu Q. Evaluation and application of a hybrid brain computer interface for real wheelchair parallel control with multi-degree of freedom. International Journal of Neural Systems, 2014, 24(4): 1450014 DOI:10.1142/s0129065714500142

108. Fernández-Rodríguez Á, Velasco-Álvarez F, Bonnet-Save M, Ron-Angevin R. Evaluation of switch and continuous navigation paradigms to command a brain-controlled wheelchair. Frontiers in Neuroscience, 2018, 12, 438 DOI:10.3389/fnins.2018.00438

109. Pfurtscheller G, Allison B Z, Brunner C, Bauernfeind G, Solis-Escalante T, Scherer R, Zander T O, Mueller-Putz G, Neuper C, Birbaumer N. The hybrid BCI. Frontiers in Neuroscience, 2010, 4, 30 DOI:10.3389/fnpro.2010.00003

110. Li Y Q, Pan J H, Wang F, Yu Z L. A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control. IEEE Transactions on Biomedical Engineering, 2013, 60(11): 3156–3166 DOI:10.1109/tbme.2013.2270283

111. Pfurtscheller G, Neuper C, Brunner C, da Silva F L. Beta rebound after different types of motor imagery in man. Neuroscience Letters, 2005, 378(3): 156–159 DOI:10.1016/j.neulet.2004.12.034

112. Naito E, Kochiyama T, Kitada R, Nakamura S, Matsumura M, Yonekura Y, Sadato N. Internally simulated movement sensations during motor imagery activate cortical motor areas and the cerebellum. The Journal of Neuroscience, 2002, 22(9): 3683–3691

1. Cunbo LI, Ning LI, Yuan QIU, Yueheng PENG, Yifeng WANG, Lili DENG, Teng MA, Fali LI, Dezhong YAO, Peng XU, Multimodal collaborative BCI system based on the improved CSP feature extraction algorithm Virtual Reality & Intelligent Hardware 2022, 4(1): 22-37

2. Changlin HAN, Ming XU, Jingsheng TANG, Yadong LIU, Zongtan ZHOU, Giant magneto-impedance sensor with working point self-adaptation for unshielded human bio-magnetic detection Virtual Reality & Intelligent Hardware 2022, 4(1): 38-54

3. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120

4. Rui HAN, Zhiquan FENG, Jinglan TIAN, Xue FAN, Xiaohui YANG, Qingbei GUO, An intelligent navigation experimental system based on multi-mode fusion Virtual Reality & Intelligent Hardware 2020, 2(4): 345-353

5. Xiaohui ZHOU, Zimu YI, Yilin LIU, Kai HUANG, Hui HUANG, Survey on path and view planning for UAVs Virtual Reality & Intelligent Hardware 2020, 2(1): 56-69

6. Yang LI, Dong WU, Jin HUANG, Feng TIAN, Hong'an WANG, Guozhong DAI, Influence of multi-modality on moving target selection in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(3): 303-315

7. Yang LI, Jin HUANG, Feng TIAN, Hong-An WANG, Guo-Zhong DAI, Gesture interaction in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(1): 84-112

8. Athirah SYAMIMI, Yiwei GONG, Ryan LIEW, VR industrial applications―A singapore perspective Virtual Reality & Intelligent Hardware 2020, 2(5): 409-420

9. Jie GUO, Dongdong WENG, Yue LIU, Qiyong CHEN, Yongtian WANG, Analysis of teenagers' preferences and concerns regarding HMDs in education Virtual Reality & Intelligent Hardware 2021, 3(5): 369-382

10. Yijun LI, Miao WANG, Derong JIN, Frank STEINICKE, Qinping ZHAO, Effects of virtual environment and self-representations on perception and physical performance in redirected jumping Virtual Reality & Intelligent Hardware 2021, 3(6): 451-469

11. Susu HUANG, Daqing QI, Jiabin YUAN, Huawei TU, Review of studies on target acquisition in virtual reality based on the crossing paradigm Virtual Reality & Intelligent Hardware 2019, 1(3): 251-264

12. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289

13. Yuan CHANG, Guo-Ping WANG, A review on image-based rendering Virtual Reality & Intelligent Hardware 2019, 1(1): 39-54

14. Shiguang QIU, Shuntao LIU, Deshuai KONG, Qichang HE, Three-dimensional virtual-real mapping of aircraft autom-atic spray operation and online simulation monitoring Virtual Reality & Intelligent Hardware 2019, 1(6): 611-621

15. Xu PENG, Zhenyu GAO, Yitong DING, Dongfeng ZHAO, Xiaoyu CHI, Study of ghost image suppression in polarized catadioptric virtual reality optical systems Virtual Reality & Intelligent Hardware 2020, 2(1): 70-78

16. Zhiming HU, Sheng LI, Meng GAI, Temporal continuity of visual attention for future gaze prediction in immersive virtual reality Virtual Reality & Intelligent Hardware 2020, 2(2): 142-152

17. Lihui HUANG, Siti Faatihah Binte Mohd TAIB, Ryan Aung BA, Zhe An GOH, Mengshan XU, Virtual reality research and development in NTU Virtual Reality & Intelligent Hardware 2020, 2(5): 394-408

18. Stéphanie PHILIPPE, Alexis D. SOUCHET, Petros LAMERAS, Panagiotis PETRIDIS, Julien CAPORAL, Gildas COLDEBOEUF, Hadrien DUZAN, Multimodal teaching, learning and training in virtual reality: a review and case study Virtual Reality & Intelligent Hardware 2020, 2(5): 421-442

19. Jia Ming LEE, Xinxing XIA, Clemen OW, Felix CHUA, Yunqing GUAN, VEGO: A novel design towards customizable and adjustable head-mounted display for VR Virtual Reality & Intelligent Hardware 2020, 2(5): 443-453

20. Jingcheng QIAN, Yancong MA, Zhigeng PAN, Xubo YANG, Effects of Virtual-real fusion on immersion, presence, and learning performance in laboratory education Virtual Reality & Intelligent Hardware 2020, 2(6): 569-584

21. Hengwei XU, Siru LI, Wenpeng SONG, Jiajun SUN, Xinli WU, Xiaoqi WANG, Wenzhen YANG, Zhigeng PAN, Abdennour EI RHALIBI, Thermal perception method of virtual chemistry experiments Virtual Reality & Intelligent Hardware 2020, 2(4): 305-315

22. TJ MATTHEWS, Feng TIAN, Tom DOLBY, Interaction design for paediatric emergency VR training Virtual Reality & Intelligent Hardware 2020, 2(4): 330-344

23. Hongxin ZHANG, Jin ZHANG, Xue YIN, Kan ZHOU, Zhigeng PAN, Abdennour EI RHALIBI, Cloud-to-end rendering and storage management for virtual reality in experimental education Virtual Reality & Intelligent Hardware 2020, 2(4): 368-380

24. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329

25. Haoyu WANG, Jianhuang WU, A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction Virtual Reality & Intelligent Hardware 2021, 3(4): 302-314

26. Na ZHANG, Liwen TAN, Fengying LI, Bing HAN, Yifa XU, Development and application of digital assistive teaching system for anatomy Virtual Reality & Intelligent Hardware 2021, 3(4): 315-335

27. Daniel VANKOV, David JANKOVSZKY, Effects of using headset-delivered virtual reality in road safety research: A systematic review of empirical studies Virtual Reality & Intelligent Hardware 2021, 3(5): 351-368

28. Xiaolong LIU, Lili WANG, Redirected jumping in virtual scenes with alleys Virtual Reality & Intelligent Hardware 2021, 3(6): 470-483

29. Liming WANG, Xianwei CHEN, Tianyang DONG, Jing FAN, Virtual climbing: An immersive upslope walking system using passive haptics Virtual Reality & Intelligent Hardware 2021, 3(6): 435-450

30. Muhammad IRFAN, Muhammad MUNSIF, Deepdive: a learning-based approach for virtual camera in immersive contents Virtual Reality & Intelligent Hardware 2022, 4(3): 247-262

31. Dangxiao WANG, Yuan GUO, Shiyi LIU, Yuru ZHANG, Weiliang XU, Jing XIAO, Haptic display for virtual reality: progress and challenges Virtual Reality & Intelligent Hardware 2019, 1(2): 136-162

32. Aiguo SONG, Liyue FU, Multi-dimensional force sensor for haptic interaction: a review Virtual Reality & Intelligent Hardware 2019, 1(2): 121-135

33. Wenmin ZHU, Xiumin FAN, Yanxin ZHANG, Applications and research trends of digital human models in the manufacturing industry Virtual Reality & Intelligent Hardware 2019, 1(6): 558-579

34. Mohammad Mahmudul ALAM, S. M. Mahbubur RAHMAN, Affine transformation of virtual 3D object using 2D localization of fingertips Virtual Reality & Intelligent Hardware 2020, 2(6): 534-555

35. Mohib ULLAH, Sareer Ul AMIN, Muhammad MUNSIF, Muhammad Mudassar YAMIN, Utkurbek SAFAEV, Habib KHAN, Salman KHAN, Habib ULLAH, Serious games in science education: a systematic literature review Virtual Reality & Intelligent Hardware 2022, 4(3): 189-209

36. Yuan WEI, Dongdong GUAN, Qiuchen WANG, Xiangxian LI, Yulong BIAN, Pu QIN, Yanning XU, Chenglei YANG, Virtual fire drill system supporting co-located collaboration Virtual Reality & Intelligent Hardware 2019, 1(3): 290-302

37. Xiaoxiong FAN, Yun CAI, Yufei YANG, Tianxing XU, Yike Li, Songhai ZHANG, Fanglue ZHANG, Detection of scene-irrelevant head movements via eye-head coordination information Virtual Reality & Intelligent Hardware 2021, 3(6): 501-514

38. Hayat ULLAH, Sitara AFZAL, Imran Ullah KHAN, Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey Virtual Reality & Intelligent Hardware 2022, 4(3): 223-246