Special Issue - Virtual Reality & Intelligent Hardware

Haptic interaction

Touch is an important channel for human beings to communicate with the external world and plays a crucial role in human interaction with nature. Information such as softness, friction, texture and warmth, or more complicated human emotional communication can only be perceived by the act of touching. Despite its importance, it is striking that haptic feeback in human-machine interaction is still in its infancy. For example, a human user can enjoy realistic visual or auditory experiences facilitated by a computer through devices such as a head-mounted display or a stereo headset. However, haptic experiences that the user can obtain is extremely limited during such interactions. Haptic interaction technology enables users to touch virtual objects and experience tactile sensations, including contact force, softness, texture, weight, vibration, and temperature. These sensations can significantly enhance users' interactive experience of virtual-reality systems. Haptic interaction technology can be applied in many fields, such as surgical simulation/ training, virtual assembly and maintenance of complex mechanical equipment such as aircraft engines, interactive entertainment experiences such as movies and computer games.

All

View Abstracts

Download Citations

Download E-Journal

EndNote

Reference Manager

ProCite

BiteTex

RefWorks

Editorial

Haptic feedback for virtual reality

Dangxiao Wang

2019, 1(2) : 1-2

DOI：10.3724/SP.J.2096-5796.2019.0000

PDF (49)

HTML (1325)

Review

Multi-dimensional force sensor for haptic interaction: a review

Aiguo SONG, Liyue FU

2019, 1(2) : 121-135

DOI：10.3724/SP.J.2096-5796.2019.0016

Abstract (2502)

PDF (74)

HTML (2145)

Haptic interaction plays an important role in the virtual reality technology, which let a person not only view the 3D virtual environment but also realistically touch the virtual environment. As a key part of haptic interaction, force feedback has become an essential function for the haptic interaction. Therefore, multi-dimensional force sensors are widely used in the fields of virtual reality and augmented reality. In this paper, some conventional multi-dimensional force sensors based on different measurement principles, such as resistive, capacitive, piezoelectric, are briefly introduced. Then the mechanical structures of the elastic body of multi-dimensional force sensors are reviewed. It is obvious that the performance of the multi-dimensional force sensor is mainly dependent upon the mechanical structure of elastic body. Furthermore, the calibration process of the force sensor is analyzed, and problems in calibration are discussed. Interdimensional coupling error is one of the main factors affecting the measurement precision of the multi-dimensional force sensors. Therefore, reducing or even eliminating dimensional coupling error becomes a fundamental requirement in the design of multi-dimensional force sensors, and the decoupling state-of-art of the multi-dimensional force sensors are introduced in this paper. At last, the trends and current challenges of multi-dimensional force sensing technology are proposed.

Haptic display for virtual reality: progress and challenges

Dangxiao WANG, Yuan GUO, Shiyi LIU, Yuru ZHANG, Weiliang XU, Jing XIAO

2019, 1(2) : 136-162

DOI：10.3724/SP.J.2096-5796.2019.0008

Abstract (3006)

PDF (101)

HTML (3907)

Immersion, interaction, and imagination are three features of virtual reality (VR). Existing VR systems possess fairly realistic visual and auditory feedbacks, and however, are poor with haptic feedback, by means of which human can perceive the physical world via abundant haptic properties. Haptic display is an interface aiming to enable bilateral signal communications between human and computer, and thus to greatly enhance the immersion and interaction of VR systems. This paper surveys the paradigm shift of haptic display occurred in the past 30 years, which is classified into three stages, including desktop haptics, surface haptics, and wearable haptics. The driving forces, key technologies and typical applications in each stage are critically reviewed. Toward the future high-fidelity VR interaction, research challenges are highlighted concerning handheld haptic device, multimodal haptic device, and high fidelity haptic rendering. In the end, the importance of understanding human haptic perception for designing effective haptic devices is addressed.

Electrostatic tactile representation in multimedia mobile terminal

Xuezhi YAN, Qiushuang WU, Xiaoying SUN

2019, 1(2) : 201-218

DOI：10.3724/SP.J.2096-5796.2019.0003

Abstract (2993)

PDF (48)

HTML (4183)

Electrostatic tactile representation technology can enhance the authenticity and immersion of human-computer interaction by perception of tactile features such as the shape and texture of visual objects in touch screen in naked finger. Focusing on the application in multimedia terminal of electrostatic force tactile representation technology, this paper summarizes the typical devices of electrostatic tactile representation, tactile rendering model-driven and data-driven algorithm, driving signal loading method, tactile representation effect evaluation method and so on. The author's view on the development status and future prospects of this technology is presented as follows: (1) Electrostatic tactile representation technology is an optimization scheme for implementing the low power and bare finger tactile representation function on the surface of the multimedia terminal; (2) The rendering dynamic range of electrostatic tactile representation technology is still insufficient, and the rendering effect of rough texture is better, but there is no effective algorithm for the rendering of fine texture. Multiple fusion technology may be one of the solutions; (3) From the perspective of evaluation of tactile representation effect, which shows that there is still considerable room for improvement both in theoretical models and applied algorithms; (4) Electrostatic tactile representation technology is essentially an applied science. Mutual promotion and benign promotion of theoretical research and commercial application is the only way for its progress. Therefore, a more mature prototype of principle is urgently needed to be popularized and applied in commerce.

Article

Haptic interface using tendon electrical stimulation with consideration of multimodal presentation

Akifumi TAKAHASHI, Kenta TANABE, Hiroyuki KAJIMOTO

2019, 1(2) : 163-175

DOI：10.3724/SP.J.2096-5796.2019.0011

Abstract (4043)

PDF (82)

HTML (1822)

Background

Our previous studies have shown that electrical stimulation from the skin surface to the tendon region (Tendon Electrical Stimulation: TES) can elicit a force sensation, and adjusting the current parameters can control the amount of the sensation. TES is thought to present a proprioceptive force sensation by stimulating receptors or sensory nerves responsible for recognizing the magnitude of the muscle contraction existing inside the tendon, so it can be a proprioceptive module of a small-size, low-cost force feedback device. But there is also suspect that TES presents only strong, noisy skin sensation. From previous study, it was found that TES has some limitation on varying sensations.

Methods

In this study, in addition to characterizing the proprioceptive sensation induced by TES, we constructed a multimodal presentation system reproducing a situation in which force is applied to the hand was offered, so as to investigate whether TES contributed to the reproduction of haptics cooperating with other modalities, rather than disturbing them. Specifically, we used vibration to present a cutaneous sensation and a visual head mounted display (HMD) system to present simultaneous images. Using this system, we also evaluated the efficacy of TES itself and that of the multimodal system involving TES.

Results

We found that TES, along with visual and vibration stimulation, contributed to the perception of a certain force.

Conclusions

Thus, TES appears to be an effective component of multimodal force sense presentation systems.

Eye-around vibration haptics on VR immersion improvement

Tao ZENG, Keyu WEI, Yanlin YU, Yi ZHAO

2019, 1(2) : 176-184

DOI：10.3724/SP.J.2096-5796.2018.0014

Abstract (2197)

PDF (52)

HTML (1526)

Due to the inherent shortcomings of the hardware, the immersion of visual interaction between the user and the virtual reality (VR) equipment is greatly reduced. In this paper, effects of eye-around vibration haptics on improving the VR immersion were studied. The vibration was generated by flexible vibrators whose performance was evaluated by a laser vibrometer. Fitting the vibrators on the human eye area at different positions and derived by different waveforms and frequencies of the input signal, the effects of vibration on the human vision and comfort of the users were verified. Then, with the selected input signals and fitting locations, different kinds of vibration were applied on the eye area cooperating with virtual reality images or videos to evaluate the changes of immersion. Research results provide references to the modeling of eye tactile feedback and the design of relevant tactile device in improving the VR immersion.

Psychophysics of wearable haptic/tactile perception in a multisensory context

Xiao LEI, Tingwei ZHANG, Kun CHEN, Jue ZHANG, Yue TIAN, Fang FANG, Lihan CHEN

2019, 1(2) : 185-200

DOI：10.3724/SP.J.2096-5796.2018.0012

Abstract (2649)

PDF (48)

HTML (1979)

Multisensory lab based in Peking University, has carried out basic studies in multisensory space and time processing, intersensory binding and haptic/tactile perception. We exploited a typical paradigm of multisensory illusion-temporal ventriloquist effect and applied it in a wide range of multisensory interactions (mainly focused on temporal processing). In this work, we summarized how the tactile stimuli were exploited to compose tactile cues and as tactile apparent motion to interface with other sensory stimuli (visual and auditory stimuli) to examine the underlying perceptual organization in a multisensory context. Moreover, we introduced two examples of wearable haptic/tactile perception in our lab, by using two customized tactile devices and discussed the potential applications in this field.

An immersive space liquid bridge experiment system with gesture interaction and vibrotactile feedback

Meng SONG, Shiyi LIU, Ge YU, Lili GUO, Dangxiao WANG

2019, 1(2) : 219-232

DOI：10.3724/SP.J.2096-5796.2019.0001

Abstract (3002)

PDF (45)

HTML (3003)

Space liquid bridge experiment is a typical space telescience experiment, which has the potential to synchronize the interaction between the human users and remote space laboratory equipment, and has high requirements on manipulation accuracy and real-time interactivity. Typical tasks in space liquid bridge experiment include bridge pulling, and clearance process etc. In order to achieve the effect of natural and immersive interaction for these tasks, a control strategy based on Leap Motion gesture interaction control and vibrotactile feedback is developed. A gesture recognition algorithm and fuzzy control algorithm based on position control are developed, and the experimental results show that the algorithms have a small memory load and a high recognition accuracy of 97.68%. We perform comparative experiments on single visual feedback and visual-tactile feedback, which verifies that the interactive effect of the visual-tactile feedback is superior to the single visual feedback. Experimental results validate that by combining Leap Motion multi-finger recognition, vibrotactile feedback and immersive display, the proposed syatem is able to achieve the goal of natural interaction for space telescience experiment.

A study of multimodal interaction design based on touch and audition for the visually impaired

Yang JIAO, Jiangtao GONG, Yingqing XU

2019, 1(2) : 233-239

DOI：10.3724/SP.J.2096-5796.2019.0002

Abstract (2762)

PDF (62)

HTML (5313)

Background

It studies multimodal interaction design based on touch and audition for the visually impaired and related assistive devices.

Methods

With the consideration of traditional multimodal hypothesis, cognition and multimodal interaction design, this study proposes a multimodal interaction modal based on touch and auditions for the visually impaired, illustrates information flows from tactile and auditory interaction interfaces, and conducts a blind user experiment under multimodal scenario to analyze the ability of user perception and cognition.

Results

Under the multimodal interaction scenario, the visually impaired users achieved better user experience and higher recognition efficiency.

Conclusions

Discusses the design principles of blind assistive devices and their interactions by this interaction modal and user experience, guiding the explorations of blind assistive devices design. The visually impaired can enjoy the internet like us.

A tactile perception technology of character in digital images

Xinli WU, Jiali LUO, Minxiong ZHANG, Chunyan KANG, Wenzhen YANG

2019, 1(2) : 240-249

DOI：10.3724/SP.J.2096-5796.2018.0013

Abstract (2827)

PDF (24)

HTML (2741)

For the blind, it is difficult to appreciate calligraphy. A tactile generation technology of text image is developed, which can enable the blind to recognize text in digital images by tactile recognition. This paper presents a Gauss difference algorithm based on tangential flow of edges, which can extract the contour of text precisely, and get the smooth contour of text with remarkable edge features. A tactile generation algorithm based on five elements is proposed to generate stable and continuous tactile sense for perceiving the contour of text. The experimental results show that through some adaptive training, the blind can better recognize some characters with simple contours, which provides a possibility for tactile appreciation of calligraphy.