Chinese
Adv Search
Home | Accepted | Article In Press | Current Issue | Archive | Special Issues | Collections | Featured Articles | Statistics

2019, 1(2): 136-162 Published Date:2019-4-20

DOI: 10.3724/SP.J.2096-5796.2019.0008

Haptic display for virtual reality: progress and challenges

Full Text: PDF (40) HTML (229)

Export: EndNote | Reference Manager | ProCite | BibTex | RefWorks

Abstract:

Immersion, interaction, and imagination are three features of virtual reality (VR). Existing VR systems possess fairly realistic visual and auditory feedbacks, and however, are poor with haptic feedback, by means of which human can perceive the physical world via abundant haptic properties. Haptic display is an interface aiming to enable bilateral signal communications between human and computer, and thus to greatly enhance the immersion and interaction of VR systems. This paper surveys the paradigm shift of haptic display occurred in the past 30 years, which is classified into three stages, including desktop haptics, surface haptics, and wearable haptics. The driving forces, key technologies and typical applications in each stage are critically reviewed. Toward the future high-fidelity VR interaction, research challenges are highlighted concerning handheld haptic device, multimodal haptic device, and high fidelity haptic rendering. In the end, the importance of understanding human haptic perception for designing effective haptic devices is addressed.
Keywords: Haptic display ; Virtual reality ; Desktop haptics ; Surface haptics ; Wearable haptics ; Multimodal haptics

Cite this article:

Dangxiao WANG, Yuan GUO, Shiyi LIU, Yuru ZHANG, Weiliang XU, Jing XIAO. Haptic display for virtual reality: progress and challenges. Virtual Reality & Intelligent Hardware, 2019, 1(2): 136-162 DOI:10.3724/SP.J.2096-5796.2019.0008

1. Sutherland I E. The ultimate display. Multimedia: From Wagner to virtual reality, 1965, 506−508

2. Goertz R C. Master-Slave Manipulator. Office of Scientific and Technical Information (OSTI), 1949 DOI:10.2172/1054625

3. van Dam A. Post-WIMP user interfaces. Communications of the ACM, 1997, 40(2): 63–67 DOI:10.1145/253671.253708

4. Salisbury K, Brock D, Massie T, Swarup N, Zilles C. Haptic rendering: programming touch interaction with virtual objects. In: Proceedings of the 1995 symposium on Interactive 3D graphics. Monterey, California, USA: ACM, 1995: 123–130 DOI:10.1145/199404.199426

5. Srinivasan M A, Basdogan C. Haptics in virtual environments: Taxonomy, research status, and challenges. Computers & Graphics, 1997, 21(4): 393–404 DOI:10.1016/s0097-8493(97)00030-7

6. von Thomas Ammann S, Ruspini D C, Kolarov K, Khatib O. The haptic display of complex graphical environments. In: The 24th annual Conference on Computer Graphics and Interactive Techniques, 1997, 345–352

7. Massie T, Salisbury K. The PHANToM Haptic Interface: A Device for Probing Virtual Objects. In: ASME International Mechanical Engineering Congress and Exhibition, 1994, 295–302

8. Puterbaugh U W M K. Six degree-offreedom haptic rendering using voxel sampling. In: ACM SIGGRAPH, 1999: 401–408

9. Lin M C, Otaduy M. Haptic Rendering: Foundations, Algorithms and Applications. In: CRC Press, 2008

10. Otaduy M A, Garre C, Lin M C. Representations and algorithms for force-feedback display. Proceedings of the IEEE, 2013, 101(9): 2068–2080 DOI:10.1109/jproc.2013.2246131

11. Ortega M, Redon S, Coquillart S. A six degree-of-freedom god-object method for haptic display of rigid bodies with surface properties. IEEE Transactions on Visualization and Computer Graphics, 2007, 13(3): 458–469 DOI:10.1109/tvcg.2007.1028

12. MacLean K E. Haptic interaction design for everyday interfaces. Reviews of Human Factors and Ergonomics, 2008, 4(1): 149–194 DOI:10.1518/155723408x342826

13. Wang D X, Shi Y J, Liu S, Zhang Y R, Xiao J. The influence of handle-avatar mapping uncertainty on torque fidelity of 6-DOF haptic rendering. World Haptics Conference (WHC). Daejeon, South Korea, 2013: 325–330 DOI:10.1109/WHC.2013.6548429

14. Wang D X, Shi Y J, Liu S, Zhang Y R, Xiao J. Haptic simulation of organ deformation and hybrid contacts in dental operations. IEEE Transactions on Haptics, 2014, 7(1): 48–60 DOI:10.1109/toh.2014.2304734

15. Yu G, Wang D X, Zhang Y R, Xiao J. Simulating sharp geometric features in six degrees-of-freedom haptic rendering. IEEE Transactions on Haptics, 2015, 8(1): 67–78 DOI:10.1109/toh.2014.2377745

16. Wang D X, Zhang Y R, Hou J X, Wang Y, Lv P, Chen Y G, Zhao H. Idental: A haptic-based dental simulator and its preliminary user evaluation. IEEE Transactions on Haptics, 2012, 5(4): 332–343 DOI:10.1109/toh.2011.59

17. Fukumoto M, Sugimura T. Active click: tactile feedback for touch panels. In: Extended Abstracts on Human Factors in Computing Systems. Seattle, Washington: ACM, 2001: 121–122 DOI:10.1145/634067.634141

18. Yatani K, Truong K N. SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices. In: Proceedings of the 22nd annual ACM symposium on User interface software and technology. Victoria, BC, Canada: ACM, 2009: 111–120 DOI:10.1145/1622176.1622198

19. Sawada H, Takeda Y. Tactile pen for presenting texture sensation from touch screen. 2015 8th International Conference on Human System Interaction (HSI). Warsaw, Poland, 2015, 334–339 DOI:10.1109/HSI.2015.7170689

20. Yang G H, Jin M S, Jin Y, Kang S. T-mobile: Vibrotactile display pad with spatial and directional information for hand-held device. 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, Taiwan, China: 2010, 5245–5250 DOI:10.1109/IROS.2010.5651759

21. Poupyrev I, Rekimoto J, Maruyama S. TouchEngine: a tactile display for handheld devices. In: Human Factors in Computing Systems. 2002, 20(25): 644–645

22. Kim S H, Sekiyama K, Fukuda T, Tanaka K, Itoigawa K. Development of dynamically Re-formable input device in tactile and visual interaction. In: International Symposium on Micro-NanoMechatronics and Human Science, Nagoya, Japan: 2007: 544–549 DOI:10.1109/MHS.2007.4420914

23. Kajimoto H. Enlarged electro-tactile display with repeated structure. In: IEEE World Haptics Conference, Istanbul, Turkey: IEEE, 2011: 575–579 DOI:10.1109/WHC.2011.5945549

24. Harrison C, Hudson S E. Providing dynamically changeable physical buttons on a visual display. In: the SIGCHI Conference on Human Factors in Computing Systems. Boston, MA, USA: 2009, 299–308

25. Kim S, Sekiyama K, Fukuda T. User-adaptive interface with reconfigurable keypad for in-vehicle information systems. In: International Symposium on Micro-NanoMechatronics and Human Science. Nagoya, Japan, 2008, 501–506 DOI:10.1109/MHS.2008.4752504

26. Watanabe T, Fukui S. A method for controlling tactile sensation of surface roughness using ultrasonic vibration. In: IEEE International Conference on Robotics and Automation. Nagoya, Japan: IEEE, 1995: 1134–1139 DOI:10.1109/ROBOT.1995.525433

27. Carter T, Seah S A, Long B, Drinkwater B, Subramanian S. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In: Proceedings of the 26th annual ACM symposium on User interface software and technology. St. Andrews, Scotland, United Kingdom: ACM, 2013: 505–514 DOI:10.1145/2501988.2502018

28. Winfield L, Glassmire J, Colgate J E, Peshkin M. T-PaD: tactile pattern display through variable friction reduction. In: Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. Tsukaba, Japan: 2007, 421–426 DOI:10.1109/WHC.2007.105

29. Marchuk N D, Colgate J E, Peshkin M A. Friction measurements on a large area TPaD. In: IEEE Haptics Symposium. Waltham, MA, USA: IEEE, 2010: 317–320 DOI:10.1109/HAPTIC.2010.5444636

30. Mullenbach J, Shultz C, Piper A M, Peshkin M, Colgate J E. Surface haptic interactions with a TPad tablet. In: Proceedings of the adjunct publication of the 26th annual ACM symposium on User interface software and technology. St. Andrews, Scotland, United Kingdom: ACM, 2013: 7–8 DOI:10.1145/2508468.2514929

31. Yang Y, Zhang Y R, Lemaire-Semail B, Dai X W. Enhancing the Simulation of Boundaries by Coupling Tactile and Kinesthetic Feedback. Haptics: Neuroscience, Devices, Modeling, and Applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2014: 19–26 DOI:10.1007/978-3-662-44196-1_3

32. Vezzoli E, Messaoud W B, Amberg M, Giraud F, Lemaire-Semail B, Bueno M A. Physical and perceptual independence of ultrasonic vibration and electrovibration for friction modulation. IEEE Transactions on Haptics, 2015, 8(2): 235–239 DOI:10.1109/toh.2015.2430353

33. Kim H, Kang J, Kim K D, Lim K M, Ryu J. Method for providing electrovibration with uniform intensity. IEEE Transactions on Haptics, 2015, 8(4): 492–496 DOI:10.1109/toh.2015.2476810

34. Mallinckrodt E, Hughes A L, Sleator W. Perception by the skin of electrically induced vibrations. Science, 1953, 118(3062): 277–278 DOI:10.1126/science.118.3062.277

35. Linjama J, Mäkinen V. E-sense screen: Novel haptic display with capacitive electrosensory interface. In: 4th Workshop for Haptic and Audio Interaction Design, 2009

36. Bau O, Poupyrev I, Israr A, Harrison C. TeslaTouch: electrovibration for touch surfaces. In: Proceedings of the 23nd annual ACM symposium on User interface software and technology. New York, New York, USA: ACM, 2010: 283–292 DOI:10.1145/1866029.1866074

37. Radivojevic Z, Beecher P, Bower C, Haque S, Andrew P, Hasan T, Bonaccorso F, Ferrari A C, Henson B. Electrotactile touch surface by using transparent graphene. In: Proceedings of the 2012 Virtual Reality International Conference. Laval, France: ACM, 2012: 1–3 DOI:10.1145/2331714.2331733

38. Vasudevan H, Manivannan M. Tangible images: runtime generation of haptic textures from images. 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. Reno, NE, USA, 2008: 357–360 DOI:10.1109/HAPTICS.2008.4479971

39. Saga S, Deguchi K. Lateral-force-based 2. 5-dimensional tactile display for touch screen. IEEE Haptics Symposium (HAPTICS), Vancouver, BC, Canada: IEEE, 2012: 15–22 DOI:10.1109/HAPTIC.2012.6183764

40. KimS-C, Israr A, Poupyrev I. Tactile rendering of 3D features on touch surfaces. In: Proceedings of the 26th annual ACM symposium on User interface software and technology. St. Andrews, Scotland, United Kingdom: ACM, 2013: 531−538 DOI:10.1145/2501988.2502020

41. Xu C, Israr A, Poupyrev I, Bau O, Harrison C. Tactile display for the visually impaired using TeslaTouch. In: Extended Abstracts on Human Factors in Computing Systems. Vancouver, BC, Canada: ACM, 2011, 317–322 DOI:10.1145/1979742.1979705

42. Israr A, Bau O, KimS-C, Poupyrev I. Tactile feedback on flat surfaces for the visually impaired. In: Human Factors in Computing Systems. Austin, Texas, USA: ACM, 2012: 1571–1576 DOI:10.1145/2212776.2223674

43. Hoggan E, Brewster S A, Johnston J. Investigating the effectiveness of tactile feedback for mobile touchscreens. In: the SIGCHI Conference on Human Factors in Computing Systems. Florence, Italy, 2008, 1573–1582

44. Levesque V, Oram L, MacLean K, Cockburn A, Marchuk N D, Johnson D, Colgate J E, Peshkin M A. Enhancing physicality in touch interaction with programmable friction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, BC, Canada: ACM; 2011: 2481–2490 DOI:10.1145/1978942.1979306

45. Lévesque V, Oram L, MacLean K. Exploring the design space of programmable friction for scrolling interactions. Haptics Symposium (HAPTICS). Vancouver, BC, Canada: IEEE, 2012, 23–30 DOI:10.1109/HAPTIC.2012.6183765

46. Shin H, Lim J M, Lee J U, Lee G, Kyung K U. Effect of tactile feedback for button GUI on mobile touch devices. ETRI Journal,2014,36(6):979–987 DOI:10.4218/etrij.14.0114.0028

47. Liu G H, Sun X Y, Wang D X, Liu Y, Zhang Y R. Effect of electrostatic tactile feedback on accuracy and efficiency of Pan gestures on touch screens. IEEE Transactions on Haptics, 2018, 11(1): 51–60 DOI:10.1109/toh.2017.2742514

48. Tran J J, Trewin S, Swart C, John B E, Thomas J C. Exploring pinch and spread gestures on mobile devices. In: Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services. Munich, Germany: ACM, 2013: 151–160 DOI:10.1145/2493190.2493221

49. Wang D X, Song M, Naqash A, Zheng Y K, Xu W L, Zhang Y R. Toward whole-hand kinesthetic feedback: A survey of force feedback gloves. IEEE Transactions on Haptics, 2018: 1 DOI:10.1109/toh.2018.2879812

50. Pacchierotti C, Sinclair S, Solazzi M, Frisoli A, Hayward V, Prattichizzo D. Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. IEEE Transactions on Haptics, 2017, 10(4): 580–600 DOI:10.1109/toh.2017.2689006

51. CyberGlove Systems LLC.http://www.cyberglovesystems. com/

52. HGlove-HAPTIONSA.https://www.haption.com/en/products-en/hglove-en.html

53. http://www.dextarobotics.com

54. HaptX|HapticglovesforVR training, simulation, anddesign. http://haptx.com/

55. Plexus/High-performanceVR/AR Gloves.http://plexus.im/

56. Sense Glove | Sense Glove.https://www.senseglove.com/

57. Avatar VR-NeuroDigital Technologies.http://www.neurodigital.es/avatarvr/

58. Contact CI.https://contactci.co/

59. Senso|Senso Glove|Senso Suit-probably the best controller for Virtual and Augmented reality.https: //senso. me/

60. Ultra-light gloves let users “touch” virtual objects". https://actu.epfl.ch/news/ultra-light-gloves-let-users-touch-virtual-objects/

61. VRgluv-Force Feedback Gloves for Virtual Reality. https://vrgluv.com/#contact

62. Zheng Y K, Wang D X, Wang Z Q, Zhang Y, Zhang Y R, Xu W L. Design of a lightweight force-feedback glove with a large workspace. Engineering, 2018, 4(6): 869–880 DOI:10.1016/j.eng.2018.10.003

63. Zhang Y, Wang D X, Wang Z Q, Wang Y P, Wen L, Zhang Y R. A two-fingered force feedback glove using soft actuators. Haptics Symposium (HAPTICS), San Francisco, CA, USA: IEEE, 2018: 186–191 DOI:10.1109/HAPTICS.2018.8357174

64. Gabardi M, Solazzi M, Leonardis D, Frisoli A. A new wearable fingertip haptic interface for the rendering of virtual shapes and surface features. Haptics Symposium (HAPTICS). Philadelphia, PA, USA: IEEE, 2016: 140–146 DOI:10.1109/HAPTICS.2016.7463168

65. Schorr S B, Okamura A M. Three-dimensional skin deformation as force substitution: Wearable device design and performance during haptic exploration of virtual environments. IEEE Transactions on Haptics, 2017, 10(3): 418–430 DOI:10.1109/toh.2017.2672969

66. Prattichizzo D, Chinello F, Pacchierotti C, Malvezzi M. Towards wearability in fingertip haptics: A 3-DoF wearable device for cutaneous force feedback. IEEE Transactions on Haptics, 2013, 6(4): 506–516 DOI:10.1109/toh.2013.53

67. Talvas A, Marchal M, Duriez C, Otaduy M A. Aggregate constraints for virtual manipulation with soft fingers. IEEE Transactions on Visualization and Computer Graphics, 2015, 21(4): 452–461 DOI:10.1109/tvcg.2015.2391863

68. Jacobs J, Stengel M, Froehlich B. A generalized God-object method for plausible finger-based interactions in virtual environments. IEEE Symposium on 3D User Interfaces (3DUI). Costa Mesa, CA, USA: 2012, 43–51 DOI:10.1109/3DUI.2012.6184183

69. Cui T, Song A G, Xiao J. Modeling global deformation using circular beams for haptic interaction. RSJ International Conference on Intelligent Robots and Systems. St. Louis, MO, USA: IEEE, 2009, 1743–1748 DOI:10.1109/IROS.2009.5354710

70. Pacchierotti C, Salvietti G, Hussain I, Meli L, Prattichizzo D. The hRing: A wearable haptic device to avoid occlusions in hand tracking. Haptics Symposium (HAPTICS), Philadelphia, PA, USA, IEEE, 2016: 134–139 DOI:10.1109/HAPTICS.2016.7463167

71. Maisto M, Pacchierotti C, Chinello F, Salvietti G, de Luca A, Prattichizzo D. Evaluation of wearable haptic systems for the fingers in augmented reality applications. IEEE Transactions on Haptics, 2017, 10(4): 511–522 DOI:10.1109/toh.2017.2691328

72. Zachmann G, Rettig A. Natural and robust interaction in virtual assembly simulation. In: Eighth ISPE International Conference on Concurrent Engineering: Research and Applications (ISPE/CE2001), 2001, 425–434

73. Moehring M, Froehlich B. Pseudo-physical interaction with a virtual car interior in immersive environments. In: the 11th Eurographics conference on Virtual Environments. Aalborg, Denmark: 2005, 181−–189

74. Holz D, Ullrich S, Wolter M, Kuhlen T. Multi-Contact Grasp Interaction for Virtual Environments. Journal of Virtual Reality and Broadcasting, 2008, 5

75. Borst C W, Indugula A P. Realistic virtual grasping. In: VR 2005, Virtual Reality. Bonn, Germany: IEEE, 2005, 91–98 DOI:10.1109/VR.2005.1492758

76. Ott R, de Perrot V, Thalmann D, Vexo F. MHaptic: a haptic manipulation library for generic virtual environments. In: International Conference on Cyberworlds. Hannover, Germany, 2007: 338–345 DOI:10.1109/CW.2007.54

77. Gourret J P, Thalmann N M, Thalmann D. Simulation of object and human skin formations in a grasping task. ACM SIGGRAPH Computer Graphics, 1989, 23(3): 21–30 DOI:10.1145/74334.74335

78. Garre C, Hernández F, Gracia A, Otaduy M A. Interactive simulation of a deformable hand for haptic rendering. In: IEEE World Haptics Conference. Istanbul, Turkey: IEEE, 2011, 239−244 DOI:10.1109/WHC.2011.5945492

79. Okamoto S, Nagano H, Yamada Y. Psychophysical dimensions of tactile perception of textures. IEEE Transactions on Haptics, 2013, 6(1): 81–93 DOI:10.1109/toh.2012.32

80. Lederman S J, Klatzky R L. Haptic perception: A tutorial. Attention, Perception & Psychophysics, 2009, 71(7): 1439–1459 DOI:10.3758/app.71.7.1439

81. Rossignac J, Allen M, Book W J, Glezer A, Ebert-Uphoff I, Shaw C, Rosen D, Askins S, Bai J, Bosscher P, Gargus J, Kim B M, Llamas I, Nguyen A, Yuan G, Zhu H H. Finger sculpting with Digital Clay: In: 3D shape input and output through a computer-controlled real surface. 2003 Shape Modeling International. Seoul, South Korea: 2003, 229–231 DOI:10.1109/SMI.2003.1199620

82. Stanley A A, Okamura A M. Deformable model-based methods for shape control of a haptic jamming surface. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(2): 1029–1041 DOI:10.1109/tvcg.2016.2525788

83. Lu L, Zhang Y, Guo X. A Surface Texture Display for Flexible Virtual Objects. In: AsiaHaptics Conference, 2018

84. Fani S, Ciotti S, Battaglia E, Moscatelli A, Bianchi M. W-FYD: A wearable fabric-based display for haptic multi-cue delivery and tactile augmented reality. IEEE Transactions on Haptics, 2018, 11(2): 304–316 DOI:10.1109/toh.2017.2708717

85. Tong Q, Yuan Z, Liao X, Zheng M, Yuan T, Zhao J. Magnetic Levitation Haptic Augmentation for Virtual Tissue Stiffness Perception. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(12): 3123−3136

86. Adel A, Micheal M M, Self M A, Abdennadher S, Khalil I S M, Rendering of Virtual Volumetric Shapes Using an Electromagnetic-Based Haptic Interface. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: 2018, 1–9

87. Mosadegh B, Polygerinos P, Keplinger C, Wennstedt S, Shepherd R F, Gupta U, Shim J, Bertoldi K, Walsh C J, Whitesides G M. Pneumatic networks for soft robotics that actuate rapidly. Advanced Functional Materials, 2014, 24(15): 2163–2170 DOI:10.1002/adfm.201303288

88. Ge Q, Sakhaei A H, Lee H, Dunn C K, Fang N X, Dunn M L. Multimaterial 4D printing with tailorable shape memory polymers. Scientific Reports, 2016, 6: 31110 DOI:10.1038/srep31110

89. Ware T H, McConney M E, Wie J J, Tondiglia V P, White T J. Voxelated liquid crystal elastomers. Science, 2015, 347(6225): 982–984 DOI:10.1126/science.1261019

90. Culbertson H, López Delgado J J, Kuchenbecker K J. One hundred data-driven haptic texture models and open-source methods for rendering on 3D objects. IEEE Haptics Symposium (HAPTICS), Houston, TX, USA: 2014, 319–325 DOI:10.1109/HAPTICS.2014.6775475

91. Strese M, Lee J Y, Schuwerk C, Han Q F, Kim H G, Steinbach E. A haptic texture database for tool-mediated texture recognition and classification. In: International Symposium on Haptic, Audio and Visual Environments and Games (HAVE) Proceedings. Richardson, TX, USA: IEEE, 2014, 118–123 DOI:10.1109/HAVE.2014.6954342

92. Strese M, Boeck Y, Steinbach E. Content-based surface material retrieval. World Haptics Conference (WHC). Munich, Germany: IEEE, 2017, 352–357 DOI:10.1109/WHC.2017.7989927

93. Zheng H T, Fang L, Ji M Q, Strese M, Ozer Y, Steinbach E. Deep learning for surface material classification using haptic and visual information. IEEE Transactions on Multimedia, 2016, 18(12): 2407–2416 DOI:10.1109/tmm.2016.2598140

94. Stevens J C. Thermal intensification of touch sensation: Further extensions of the Weber phenomenon. Sensory Processes, 1979, 3(3), 240–248

95. Charpentier A. Analyse experimentale de quelques elements de la sensation de poids. Archive de Physiologie normale et pathologiques, 1981, 3, 122–135

96. Ellis R R, Lederman S J. The material-weight illusion revisited. Perception & Psychophysics, 1999, 61(8): 1564–1576 DOI:10.3758/bf03213118

97. Ellis R R, Lederman S J. The golf-ball illusion: Evidence for top-down processing in weight perception. Perception, 1998, 27(2): 193–201 DOI:10.1068/p270193

98. Craig A, Bushnell M. The thermal grill illusion: Unmasking the burn of cold pain. Science, 1994, 265(5169): 252–255 DOI:10.1126/science.8023144

99. Leung A Y, Wallace M S, Schulteis G, Yaksh T L. Qualitative and quantitative characterization of the thermal grill. Pain, 2005, 116(1): 26–32 DOI:10.1016/j.pain.2005.03.026

email E-mail this page

Articles by authors

VRIH