Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey

Abstract

The recent advancements in the field of Virtual Reality (VR) and Augmented Reality (AR) have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the next generation Software Technology (Soft Tech). VR and AR technology provide astonishing immersive contents with the help of high quality stitched panoramic contents and 360° imagery that widely used in the education, gaming, entertainment, and production sector. The immersive quality of VR and AR contents are greatly dependent on the perceptual quality of panoramic or 360° images, in fact a minor visual distortion can significantly degrade the overall quality. Thus, to ensure the quality of constructed panoramic contents for VR and AR applications, numerous Stitched Image Quality Assessment (SIQA) methods have been proposed to assess the quality of panoramic contents before using in VR and AR. In this survey, we provide a detailed overview of the SIQA literature and exclusively focus on objective SIQA methods presented till date. For better understanding, the objective SIQA methods are classified into two classes namely Full-Reference SIQA and No-Reference SIQA approaches. Each class is further categorized into traditional and deep learning-based methods and examined their performance for SIQA task. Further, we shortlist the publicly available benchmark SIQA datasets and evaluation metrices used for quality assessment of panoramic contents. In last, we highlight the current challenges in this area based on the existing SIQA methods and suggest future research directions that need to be target for further improvement in SIQA domain.

Keyword

Virtual reality ; Augmented reality ; Panoramic image ; Immersive contents ; Stitched image quality assessment ; Deep learning ; Convolutional neural networks

Cite this article

Hayat ULLAH, Sitara AFZAL, Imran Ullah KHAN. Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey. Virtual Reality & Intelligent Hardware, 2022, 4(3): 223-246 DOI:10.1016/j.vrih.2022.03.004

References

1. Li Y, Huang J, Tian F, Wang H A, Dai G Z. Gesture interaction in virtual reality. Virtual Reality & Intelligent Hardware, 2019, 1(1): 84–112 DOI:10.3724/sp.j.2096-5796.2018.0006

2. Zheng L Y, Liu X, An Z W, Li S F, Zhang R J. A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection. Virtual Reality & Intelligent Hardware, 2020, 2(1): 12–27 DOI:10.1016/j.vrih.2019.12.002

3. Zhang H X, Zhang J, Yin X, Zhou K, Pan Z G. Cloud-to-end rendering and storage management for virtual reality in experimental education. Virtual Reality & Intelligent Hardware, 2020, 2(4): 368–380 DOI:10.1016/j.vrih.2020.07.001

4. Qian J C, Ma Y C, Pan Z G, Yang X B. Effects of virtual-real fusion on immersion, presence, and learning performance in laboratory education. Virtual Reality & Intelligent Hardware, 2020, 2(6): 569–584 DOI:10.1016/j.vrih.2020.07.010

5. Tai Y H, Shi J S, Pan J J, Hao A M, Chang V. Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems. Virtual Reality & Intelligent Hardware, 2021, 3(4): 274–286 DOI:10.1016/j.vrih.2021.08.002

6. Wang H Y, Wu J H. A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction. Virtual Reality & Intelligent Hardware, 2021, 3(4): 302–314 DOI:10.1016/j.vrih.2021.08.004

7. Farley O R L, Spencer K, Baudinet L. Virtual reality in sports coaching, skill acquisition and application to surfing: a review. Journal of Human Sport and Exercise, 2019, 15(3): 535–548. DOI:10.14198/jhse.2020.153.06

8. Soltani P, Morice A H P. Augmented reality tools for sports education and training. Computers & Education, 2020, 155: 103923 DOI:10.1016/j.compedu.2020.103923

9. Syamimi A, Gong Y W, Liew R. VR industrial applications―A Singapore perspective. Virtual Reality & Intelligent Hardware, 2020, 2(5): 409–420 DOI:10.1016/j.vrih.2020.06.001

10. Zhu W M, Fan X M, Zhang Y X. Applications and research trends of digital human models in the manufacturing industry. Virtual Reality & Intelligent Hardware, 2019, 1(6): 558–579 DOI:10.1016/j.vrih.2019.09.005

11. Lyu W, Zhou Z, Chen L, Zhou Y. A survey on image and video stitching. Virtual Reality & Intelligent Hardware, 2019, 1(1): 55–83 DOI:10.3724/sp.j.2096-5796.2018.0008

12. Lee K Y, Sim J Y. Warping residual based image stitching for large parallax. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA, IEEE, 2020, 8195–8203 DOI:10.1109/cvpr42600.2020.00822

13. Brunet D, Vrscay E R, Wang Z. On the mathematical properties of the structural similarity index. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2012, 21(4): 1488–1499 DOI:10.1109/tip.2011.2173206

14. Kong Y Q, Cui L, Hou R. Full-reference IPTV image quality assessment by deeply learning structural cues. Signal Processing: Image Communication, 2020, 83: 115779 DOI:10.1016/j.image.2020.115779

15. Wang Z, Simoncelli E P, Bovik A C. Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. GrovePacific, CA, USA, IEEE, 2003, 1398–1402 DOI:10.1109/acssc.2003.1292216

16. Xue W F, Zhang L, Mou X Q, Bovik A C. Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2014, 23(2): 684–695 DOI:10.1109/tip.2013.2293423

17. Huang X, Wen D W, Xie J F, Zhang L P. Quality assessment of panchromatic and multispectral image fusion for the ZY-3 satellite: from an information extraction perspective. IEEE Geoscience and Remote Sensing Letters, 2014, 11(4): 753–757 DOI:10.1109/lgrs.2013.2278551

18. Liu H, Zhang Y, Zhang H, Fan C, Kwong S, Kuo C J, Fan X. Deep learning based picture-wise just noticeable distortion prediction model for image compression. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2019 DOI:10.1109/tip.2019.2933743

19. Moorthy A K, Bovik A C. Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2011, 20(12): 3350–3364 DOI:10.1109/tip.2011.2147325

20. Liu L X, Liu B, Huang H, Bovik A C. No-reference image quality assessment based on spatial and spectral entropies. Signal Processing: Image Communication, 2014, 29(8): 856–863 DOI:10.1016/j.image.2014.06.006

21. Mittal A, Soundararajan R, Bovik A C. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 2013, 20(3): 209–212 DOI:10.1109/lsp.2012.2227726

22. Moorthy A K, Bovik A C. A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters, 2010, 17(5): 513–516 DOI:10.1109/lsp.2010.2043888

23. Saad M A, Bovik A C, Charrier C. Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2012, 21(8): 3339–3352 DOI:10.1109/tip.2012.2191563

24. Yang L Y, Cheung G, Tan Z G, Huang Z. A content-aware metric for stitched panoramic image quality assessment. In: 2017 IEEE International Conference on Computer Vision Workshops. Venice, Italy, IEEE, 2017, 2487–2494 DOI:10.1109/iccvw.2017.293

25. Zhou X S, Zhang H Y, Wang Y J. A multi-image stitching method and quality evaluation. In: 2017 4th International Conference on Information Science and Control Engineering (ICISCE). Changsha, China, IEEE, 2017, 46–50 DOI:10.1109/icisce.2017.20

26. Xu M, Li C, Liu Y F, Deng X, Lu J X. A subjective visual quality assessment method of panoramic videos. In: 2017 IEEE International Conference on Multimedia and Expo. Hong Kong, China, IEEE, 2017, 517–522

27. Zhang B, Zhao J Z, Yang S, Zhang Y, Wang J, Fei Z S. Subjective and objective quality assessment of panoramic videos in virtual reality environments. In: 2017 IEEE International Conference on Multimedia & Expo Workshops. Hong Kong, China, IEEE, 2017, 163–168 DOI:10.1109/icmew.2017.8026226

28. Yang J C, Liu T L, Jiang B, Song H B, Lu W. 3D panoramic virtual reality video quality assessment based on 3D convolutional neural networks. IEEE Access, 2018, 6: 38669–38682 DOI:10.1109/access.2018.2854922

29. de A Azevedo R G, Birkbeck N, Janatra I, Adsumilli B, Frossard P. A viewport-driven multi-metric fusion approach for 360-degree video quality assessment. In: 2020 IEEE International Conference on Multimedia and Expo. London, UK, IEEE, 2020, 1–6 DOI:10.1109/icme46284.2020.9102936

30. Guo P, Shen Q, Ma Z, Brady D J, Wang Y. Perceptual Quality Assessment of Immersive Images Considering Peripheral Vision Impact. 2018

31. Chen S J, Zhang Y X, Li Y M, Chen Z Z, Wang Z. Spherical structural similarity index for objective omnidirectional video quality assessment. In: 2018 IEEE International Conference on Multimedia and Expo. San Diego, CA, USA, IEEE, 2018, 1–6 DOI:10.1109/icme.2018.8486584

32. Zhang Y X, Wang Y B, Liu F Y, Liu Z Z, Li Y M, Yang D Q, Chen Z Z. Subjective panoramic video quality assessment database for coding applications. IEEE Transactions on Broadcasting, 2018, 64(2): 461–473 DOI:10.1109/tbc.2018.2811627

33. Lim H T, Kim H G, Ra Y M. VR IQA NET: deep virtual reality image quality assessment using adversarial learning. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, AB, Canada, IEEE, 2018, 6737–6741 DOI:10.1109/icassp.2018.8461317

34. Li J, Yu K W, Zhao Y F, Zhang Y, Xu L. Cross-reference stitching quality assessment for 360° omnidirectional images. MM '19: Proceedings of the 27th ACM International Conference on Multimedia. 2019, 2360–2368 DOI:10.1145/3343031.3350973

35. Yu K W, Li J, Zhang Y, Zhao Y F, Xu L. Image quality assessment for omnidirectional cross-reference stitching. 2019

36. Li C, Xu M, Jiang L, Zhang S Y, Tao X M. Viewport proposal CNN for 360° video quality assessment. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA, IEEE, 2019, 10169–10178 DOI:10.1109/cvpr.2019.01042

37. Wu P, Ding W X, You Z X, An P. Virtual reality video quality assessment based on 3d convolutional neural networks. In: 2019 IEEE International Conference on Image Processing. Taipei, Taiwan, China, IEEE, 2019, 3187–3191 DOI:10.1109/icip.2019.8803023

38. Kim H G, Lim H T, Ro Y M. Deep virtual reality image quality assessment with human perception guider for omnidirectional image. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(4): 917–928 DOI:10.1109/tcsvt.2019.2898732

39. Yan W Q, Yue G H, Fang Y M, Chen H, Tang C, Jiang G Y. Perceptual objective quality assessment of stereoscopic stitched images. Signal Processing, 2020, 172: 107541 DOI:10.1016/j.sigpro.2020.107541

40. Zheng X L, Jiang G Y, Yu M, Jiang H. Segmented spherical projection-based blind omnidirectional image quality assessment. IEEE Access, 2020, 8: 31647–31659 DOI:10.1109/access.2020.2972158

41. Chen Z B, Xu J H, Lin C Y, Zhou W. Stereoscopic omnidirectional image quality assessment based on predictive coding theory. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(1): 103–117 DOI:10.1109/jstsp.2020.2968182

42. Yang J C, Liu T L, Jiang B, Lu W, Meng Q G. Panoramic video quality assessment based on non-local spherical CNN. IEEE Transactions on Multimedia, 2021, 23: 797–809 DOI:10.1109/tmm.2020.2990075

43. Wang X J, Chai X L, Shao F. Quality assessment for color correction-based stitched images via bi-directional matching. Journal of Visual Communication and Image Representation, 2021, 75: 103051 DOI:10.1016/j.jvcir.2021.103051

44. Leorin S, Lucchese L, Cutler R G. Quality assessment of panorama video for videoconferencing applications. In: 2005 IEEE 7th Workshop on Multimedia Signal Processing. Shanghai, China, IEEE, 2005, 1–4

45. Xu W, Mulligan J. Performance evaluation of color correction approaches for automatic multi-view image and video stitching. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA, IEEE, 2010, 263–270 DOI:10.1109/cvpr.2010.5540202

46. Yang L Y, Liu J, Gao C Q. An error-activation-guided blind metric for stitched panoramic image quality assessment. Computer Vision, 2017,256–268 DOI:10.1007/978-981-10-7302-1_22

47. Ling S Y, Cheung G, le Callet P. No-reference quality assessment for stitched panoramic images using convolutional sparse coding and compound feature selection. In: 2018 IEEE International Conference on Multimedia and Expo. San Diego, CA, USA, IEEE, 2018, 1–6 DOI:10.1109/icme.2018.8486545

48. Gandhe S T, Omkar S. Blind image quality evaluation of stitched image using novel hybrid warping technique. International Journal of Advanced Computer Science and Applications, 2019, 10(6): 384–389 DOI:10.14569/ijacsa.2019.0100649

49. Xia Y M, Wang Y F, Peng Y. Blind panoramic image quality assessment via the asymmetric mechanism of human brain. In: 2019 IEEE Visual Communications and Image Processing. Sydney, NSW, Australia, IEEE, 2019, 1–4 DOI:10.1109/vcip47243.2019.8965887

50. Yu S J, Li T S, Xu X Y, Tao H, Yu L, Wang Y X. NRQQA: a no-reference quantitative quality assessment method for stitched images. MMAsia '19: Proceedings of the ACM Multimedia Asia. 2019, 1–6 DOI:10.1145/3338533.3366563

51. Madhusudana P C, Soundararajan R. Subjective and objective quality assessment of stitched images for virtual reality. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2019, 28(11): 5620–5635 DOI:10.1109/tip.2019.2921858

52. Li J, Zhao Y F, Ye W H, Yu K W, Ge S M. Attentive deep stitching and quality assessment for 360° omnidirectional images. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(1): 209–221 DOI:10.1109/jstsp.2019.2953950

53. Hou J W, Lin W S, Zhao B Q. Content-dependency reduction with multi-task learning in blind stitched panoramic image quality assessment. In: 2020 IEEE International Conference on Image Processing. Abu Dhabi, United Arab Emirates, IEEE, 2020, 3463–3467 DOI:10.1109/icip40778.2020.9191241

54. Ullah H, Irfan M, Han K, Lee J W. DLNR-SIQA: deep learning-based no-reference stitched image quality assessment. Sensors, 2020, 20(22): 6457 DOI:10.3390/s20226457

55. Sun W, Min X K, Zhai G T, Gu K, Duan H Y, Ma S W. MC360IQA: a multi-channel CNN for blind 360-degree image quality assessment. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(1): 64–77 DOI:10.1109/jstsp.2019.2955024

56. Xu J H, Zhou W, Chen Z B. Blind omnidirectional image quality assessment with viewport oriented graph convolutional networks. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(5): 1724–1737 DOI:10.1109/tcsvt.2020.3015186

57. Poreddy A K R, Kara P A, Appina B, Simon A. A no-reference 3D virtual reality image quality assessment algorithm based on saliency statistics. In: Optics and Photonics for Information Processing XV. San Diego, USA, SPIE, 2021 DOI:10.1117/12.2597327

58. Ding W X, An P, Liu X, Yang C, Huang X P. No-reference panoramic image quality assessment based on adjacent pixels correlation. In: 2021 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. Chengdu, China, IEEE, 2021, 1–5 DOI:10.1109/bmsb53066.2021.9547132

59. Zhou W, Xu J H, Jiang Q P, Chen Z B. No-reference quality assessment for 360-degree images by analysis of multifrequency information and local-global naturalness. IEEE Transactions on Circuits and Systems for Video Technology, 1182, PP(99): 1 DOI:10.1109/tcsvt.2021.3081182

60. Zhang Y X, Liu Z Z, Chen Z Z, Xu X Z, Liu S. No-reference quality assessment of panoramic video based on spherical-domain features. In: 2021 Picture Coding Symposium (PCS). Bristol, United Kingdom, IEEE, 2021, 1–5 DOI:10.1109/pcs50896.2021.9477498

61. Li C, Xu M, Du X Z, Wang Z L. Bridge the gap between VQA and human behavior on omnidirectional video: a large-scale dataset and a deep learning model. MM '18: Proceedings of the 26th ACM International Conference on Multimedia. 2018, 932–940 DOI:10.1145/3240508.3240581

62. Xiao J X, Ehinger K A, Oliva A, Torralba A. Recognizing scene viewpoint using panoramic place representation. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA, IEEE, 2012, 2695–2702 DOI:10.1109/cvpr.2012.6247991

63. Duan H Y, Zhai G T, Min X K, Zhu Y C, Fang Y, Yang X K. Perceptual quality assessment of omnidirectional images. In: 2018 IEEE International Symposium on Circuits and Systems. Florence, Italy, IEEE, 2018, 1–5 DOI:10.1109/iscas.2018.8351786

64. Sun W, Gu K, Ma S W, Zhu W H, Liu N, Zhai G T. A large-scale compressed 360-degree spherical image database: from subjective quality evaluation to objective model comparison. In: 2018 IEEE 20th International Workshop on Multimedia Signal Processing. Vancouver, BC, Canada, IEEE, 2018, 1–6 DOI:10.1109/mmsp.2018.8547102

65. Chen M X, Jin Y Z, Goodall T, Yu X X, Bovik A C. Study of 3D virtual reality picture quality. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(1): 89–102 DOI:10.1109/jstsp.2019.2956408

66. Sendjasni A, Larabi M C, Cheikh F A. Perceptually-weighted cnn for 360-degree image quality assessment using visual scan-path and jnd. In: 2021 IEEE International Conference on Image Processing. Anchorage, AK, USA, IEEE, 2021, 1439–1443 DOI:10.1109/icip42928.2021.9506044

67. Tian C Z, Chai X L, Shao F. Stitched image quality assessment based on local measurement errors and global statistical properties. Journal of Visual Communication and Image Representation, 2021, 81: 103324 DOI:10.1016/j.jvcir.2021.103324

68. Zhou Y, Sun Y J, Li L D, Gu K, Fang Y M. Omnidirectional image quality assessment by distortion discrimination assisted multi-stream network. IEEE Transactions on Circuits and Systems for Video Technology, 1162, PP(99): 1 DOI:10.1109/tcsvt.2021.3081162

69. Zaragoza J, Chin T J, Brown M S, Suter D. As-projective-as-possible image stitching with moving DLT. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA, IEEE, 2013, 2339–2346 DOI:10.1109/cvpr.2013.303

70. Yan W Q, Hou C P, Lei J J, Fang Y M, Gu Z Y, Ling N. Stereoscopic image stitching based on a hybrid warping model. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(9): 1934–1946 DOI:10.1109/tcsvt.2016.2564838

71. Chang C H, Sato Y, Chuang Y Y. Shape-preserving half-projective warps for image stitching. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA, IEEE, 2014, 3254–3261 DOI:10.1109/cvpr.2014.422

72. Wallace G K. The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics, 1992, 38(1): xviii–xxxiv DOI:10.1109/30.125072

73. Wiegand T, Sullivan G J, Bjontegaard G, Luthra A. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(7): 560–576 DOI:10.1109/tcsvt.2003.815165

74. Sullivan G J, Ohm J R, Han W J, Wiegand T. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(12): 1649–1668 DOI:10.1109/tcsvt.2012.2221191

75. Benesty J, Chen J, Huang Y, Cohen I. Noise Reduction in Speech Processing. 2009

76. Sheskin D. Spearman's rank-order correlation coefficient. Handbook of Parametric and Nonparametric Statistical Procedures. 2007, 1353–1370

77. Brassington G. Mean absolute error and root mean square error: which is the better metric for assessing model performance? EGU General Assembly Conference Abstracts, 2017, 3574

78. Huang Y, Liu Z W, Jiang M H, Yu X, Ding X H. Cost-effective vehicle type recognition in surveillance images with deep active learning and web data. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(1): 79–86 DOI:10.1109/tits.2018.2888698

79. Feng C, Liu M Y, Kao C C, Lee T Y. Deep active learning for civil infrastructure defect detection and classification. In: ASCE International Workshop on Computing in Civil Engineering 2017. Seattle, Washington, Reston, VA, USA: American Society of Civil Engineers, 2017, 298–306 DOI:10.1061/9780784480823.036

80. Ji Y Z, Zhang H J, Jonathan Wu Q M. Salient object detection via multi-scale attention CNN. Neurocomputing, 2018, 322: 130–140 DOI:10.1016/j.neucom.2018.09.061

81. Xu Q, Xiao Y, Wang D Y, Luo B. CSA-MSO3DCNN: multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification. Remote Sensing, 2020, 12(1): 188 DOI:10.3390/rs12010188

82. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Houlsby N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. 2020

83. Zhou D, Kang B, Jin X, Yang L, Lian X, Hou Q, Feng J. DeepViT: Towards Deeper Vision Transformer. 2021

2. Yuanyuan SHI, Yunan LI, Xiaolong FU, Kaibin MIAO, Qiguang MIAO, Review of dynamic gesture recognition Virtual Reality & Intelligent Hardware 2021, 3(3): 183-206

3. Ziping ZHAO, Keru Wang, Zhongtian BAO, Zixing ZHANG, Nicholas CUMMINS, Shihuang SUN, Haishuai WANG, Jianhua TAO, Björn W. SCHULLER, Self-attention transfer networks for speech emotion recognition Virtual Reality & Intelligent Hardware 2021, 3(1): 43-54

4. Zike YAN, Hongbin ZHA, Flow-based SLAM: From geometry computation to learning Virtual Reality & Intelligent Hardware 2019, 1(5): 435-460

5. Rafik HAMZA, Minh-Son DAO, Privacy-preserving deep learning techniques for wearable sensor-based big data applications Virtual Reality & Intelligent Hardware 2022, 4(3): 210-222

6. Lianyu ZHENG, Xinyu LIU, Zewu AN, Shufei LI, Renjie ZHANG, A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection Virtual Reality & Intelligent Hardware 2020, 2(1): 12-27

7. Zhiyuan ZHANG, Yuchao DAI, Jiadai SUN, Deep learning based point cloud registration: an overview Virtual Reality & Intelligent Hardware 2020, 2(3): 222-246

8. Xiaojiao SONG, Jianjun ZHU, Jingfan FAN, Danni AI, Jian YANG, Topological distance-constrained feature descriptor learning model for vessel matching in coronary angiographies Virtual Reality & Intelligent Hardware 2021, 3(4): 287-301

9. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289

10. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120

11. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329

12. Mohib ULLAH, Sareer Ul AMIN, Muhammad MUNSIF, Muhammad Mudassar YAMIN, Utkurbek SAFAEV, Habib KHAN, Salman KHAN, Habib ULLAH, Serious games in science education: a systematic literature review Virtual Reality & Intelligent Hardware 2022, 4(3): 189-209

13. Wei LYU, Zhong ZHOU, Lang CHEN, Yi ZHOU, A survey on image and video stitching Virtual Reality & Intelligent Hardware 2019, 1(1): 55-83

14. Mengting XIAO, Zhiquan FENG, Xiaohui YANG, Tao XU, Qingbei GUO, Multimodal interaction design and application in augmented reality for chemical experiment Virtual Reality & Intelligent Hardware 2020, 2(4): 291-304

15. Mingxuan CHEN, Ping ZHANG, Zebo WU, Xiaodan CHEN, A multichannel human-swarm robot interaction system in augmented reality Virtual Reality & Intelligent Hardware 2020, 2(6): 518-533

16. Yonghang TAI, Junsheng SHI, Junjun PAN, Aimin HAO, Victor CHANG, Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems Virtual Reality & Intelligent Hardware 2021, 3(4): 274-286

17. Jinyu LI, Bangbang YANG, Danpeng CHEN, Nan WANG, Guofeng ZHANG, Hujun BAO, Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality Virtual Reality & Intelligent Hardware 2019, 1(4): 386-410

18. Xiaomei ZHAO, Fulin TANG, Yihong WU, Real-time human segmentation by BowtieNet and a SLAM-based human AR system Virtual Reality & Intelligent Hardware 2019, 1(5): 511-524

19. Chan QIU, Shien ZHOU, Zhenyu LIU, Qi GAO, Jianrong TAN, Digital assembly technology based on augmented reality and digital twins: a review Virtual Reality & Intelligent Hardware 2019, 1(6): 597-610

20. Wang LI, Junfeng WANG, Sichen JIAO, Meng WANG, Shiqi LI, Research on the visual elements of augmented reality assembly processes Virtual Reality & Intelligent Hardware 2019, 1(6): 622-634

21. Pengfei HAN, Gang ZHAO, A review of edge-based 3D tracking of rigid objects Virtual Reality & Intelligent Hardware 2019, 1(6): 580-596

22. Shenze WANG, Kaikai DU, Ningfang SONG, Dongfeng ZHAO, Di FENG, Zhengqian TU, Study on the adaptability of augmented reality smartglasses for astigmatism based on holographic waveguide grating Virtual Reality & Intelligent Hardware 2020, 2(1): 79-85

23. Lingfei ZHU, Qi CAO, Yiyu CAI, Development of augmented reality serious games with a vibrotactile feedback jacket Virtual Reality & Intelligent Hardware 2020, 2(5): 454-470

24. Jie REN, Chun YU, Yueting WENG, Chengchi ZHOU, Yuanchun SHI, Design and evaluation of window management operations in AR headset+smartphone interface Virtual Reality & Intelligent Hardware 2022, 4(2): 115-131

25. Yun-Han LEE, Tao ZHAN, Shin-Tson WU, Prospects and challenges in augmented reality displays Virtual Reality & Intelligent Hardware 2019, 1(1): 10-20

26. Yang LI, Dong WU, Jin HUANG, Feng TIAN, Hong'an WANG, Guozhong DAI, Influence of multi-modality on moving target selection in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(3): 303-315

27. Yang LI, Jin HUANG, Feng TIAN, Hong-An WANG, Guo-Zhong DAI, Gesture interaction in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(1): 84-112

28. Athirah SYAMIMI, Yiwei GONG, Ryan LIEW, VR industrial applications―A singapore perspective Virtual Reality & Intelligent Hardware 2020, 2(5): 409-420

29. Jie GUO, Dongdong WENG, Yue LIU, Qiyong CHEN, Yongtian WANG, Analysis of teenagers' preferences and concerns regarding HMDs in education Virtual Reality & Intelligent Hardware 2021, 3(5): 369-382

30. Yijun LI, Miao WANG, Derong JIN, Frank STEINICKE, Qinping ZHAO, Effects of virtual environment and self-representations on perception and physical performance in redirected jumping Virtual Reality & Intelligent Hardware 2021, 3(6): 451-469

31. Ahmed L ALYOUSIFY, Ramadhan J MSTAFA, AR-assisted children book for smart teaching and learning of Turkish alphabets Virtual Reality & Intelligent Hardware 2022, 4(3): 263-277

32. Susu HUANG, Daqing QI, Jiabin YUAN, Huawei TU, Review of studies on target acquisition in virtual reality based on the crossing paradigm Virtual Reality & Intelligent Hardware 2019, 1(3): 251-264

33. Yuan CHANG, Guo-Ping WANG, A review on image-based rendering Virtual Reality & Intelligent Hardware 2019, 1(1): 39-54

34. Shiguang QIU, Shuntao LIU, Deshuai KONG, Qichang HE, Three-dimensional virtual-real mapping of aircraft autom-atic spray operation and online simulation monitoring Virtual Reality & Intelligent Hardware 2019, 1(6): 611-621

35. Xu PENG, Zhenyu GAO, Yitong DING, Dongfeng ZHAO, Xiaoyu CHI, Study of ghost image suppression in polarized catadioptric virtual reality optical systems Virtual Reality & Intelligent Hardware 2020, 2(1): 70-78

36. Zhiming HU, Sheng LI, Meng GAI, Temporal continuity of visual attention for future gaze prediction in immersive virtual reality Virtual Reality & Intelligent Hardware 2020, 2(2): 142-152

37. Lihui HUANG, Siti Faatihah Binte Mohd TAIB, Ryan Aung BA, Zhe An GOH, Mengshan XU, Virtual reality research and development in NTU Virtual Reality & Intelligent Hardware 2020, 2(5): 394-408

38. Stéphanie PHILIPPE, Alexis D. SOUCHET, Petros LAMERAS, Panagiotis PETRIDIS, Julien CAPORAL, Gildas COLDEBOEUF, Hadrien DUZAN, Multimodal teaching, learning and training in virtual reality: a review and case study Virtual Reality & Intelligent Hardware 2020, 2(5): 421-442

39. Jia Ming LEE, Xinxing XIA, Clemen OW, Felix CHUA, Yunqing GUAN, VEGO: A novel design towards customizable and adjustable head-mounted display for VR Virtual Reality & Intelligent Hardware 2020, 2(5): 443-453

40. Jingcheng QIAN, Yancong MA, Zhigeng PAN, Xubo YANG, Effects of Virtual-real fusion on immersion, presence, and learning performance in laboratory education Virtual Reality & Intelligent Hardware 2020, 2(6): 569-584