Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board

Articles

    Editorial

  • Hand and gesture
    Xilin CHEN

    DOI:10.3724/SP.J.2096-5796.2021.03.03

    2021, 3(3) : 1-2

    PDF (7) HTML (141)
  • Review

  • Review of dynamic gesture recognition
    Yuanyuan SHI, Yunan LI, Xiaolong FU, Kaibin MIAO, Qiguang MIAO

    DOI:10.1016/j.vrih.2021.05.001

    2021, 3(3) : 183-206

    Abstract(233) PDF (9) HTML (172)
    In recent years, gesture recognition has been widely used in the fields of intelligent driving, virtual reality, and human-computer interaction. With the development of artificial intelligence, deep learning has achieved remarkable success in computer vision. To help researchers better understanding the development status of gesture recognition in video, this article provides a detailed survey of the latest developments in gesture recognition technology for videos based on deep learning. The reviewed methods are broadly categorized into three groups based on the type of neural networks used for recognition: two-stream convolutional neural networks, 3D convolutional neural networks, and Long-short Term Memory (LSTM) networks. In this review, we discuss the advantages and limitations of existing technologies, focusing on the feature extraction method of the spatiotemporal structure information in a video sequence, and consider future research directions.
  • Survey on depth and RGB image-based 3D hand shape and pose estimation
    Lin HUANG, Boshen ZHANG, Zhilin GUO, Yang XIAO, Zhiguo CAO, Junsong YUAN

    DOI:10.1016/j.vrih.2021.05.002

    2021, 3(3) : 207-234

    Abstract(215) PDF (6) HTML (126)
    The field of vision-based human hand three-dimensional (3D) shape and pose estimation has attracted significant attention recently owing to its key role in various applications, such as natural human-computer interactions. With the availability of large-scale annotated hand datasets and the rapid developments of deep neural networks (DNNs), numerous DNN-based data-driven methods have been proposed for accurate and rapid hand shape and pose estimation. Nonetheless, the existence of complicated hand articulation, depth and scale ambiguities, occlusions, and finger similarity remain challenging. In this study, we present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras. Related RGB-D cameras, hand datasets, and a performance analysis are also discussed to provide a holistic view of recent achievements. We also discuss the research potential of this rapidly growing field.
  • Article

  • Adaptive cross-fusion learning for multi-modal gesture recognition
    Benjia ZHOU, Jun WAN, Yanyan LIANG, Guodong GUO

    DOI:10.1016/j.vrih.2021.05.003

    2021, 3(3) : 235-247

    Abstract(156) PDF (13) HTML (123)
    Background
    Gesture recognition has attracted significant attention because of its wide range of potential applications. Although multi-modal gesture recognition has made significant progress in recent years, a popular method still is simply fusing prediction scores at the end of each branch, which often ignores complementary features among different modalities in the early stage and does not fuse the complementary features into a more discriminative feature.
    Methods
    This paper proposes an Adaptive Cross-modal Weighting (ACmW) scheme to exploit complementarity features from RGB-D data in this study. The scheme learns relations among different modalities by combining the features of different data streams. The proposed ACmW module contains two key functions: (1) fusing complementary features from multiple streams through an adaptive one-dimensional convolution; and (2) modeling the correlation of multi-stream complementary features in the time dimension. Through the effective combination of these two functional modules, the proposed ACmW can automatically analyze the relationship between the complementary features from different streams, and can fuse them in the spatial and temporal dimensions.
    Results
    Extensive experiments validate the effectiveness of the proposed method, and show that our method outperforms state-of-the-art methods on IsoGD and NVGesture.
  • Teaching Chinese sign language with a smartphone
    Yanxiao ZHANG, Yuecong MIN, Xilin CHEN

    DOI:10.1016/j.vrih.2021.05.004

    2021, 3(3) : 248-260

    Abstract(193) PDF (7) HTML (144)
    Background
    There is a large group of deaf-mutes worldwide, and sign language is a major communication tool in this community. It is necessary for deaf-mutes to be able to communicate with others who are capable of hearing, and hearing people also need to understand sign language, which produces a great demand for sign language tuition. Even though there have already been a large number of books written about sign language, it is inefficient to learn sign language through reading alone, and the same can be said on watching videos. To solve this problem, we developed a smartphone-based interactive Chinese sign language teaching system that facilitates sign language learning.
    Methods
    The system provides a learner with some learning modes and captures the learner's actions using the front camera of the smartphone. At present, the system provides a vocabulary set with 1000 frequently used words, and the learner can evaluate his/her sign action by subjective or objective comparison. In the mode of word recognition, the users can play any word within the vocabulary, and the system will return the top three retrieved candidates; thus, it can remind the learners what the sign is.
    Results
    This system provides interactive learning to enable a user to efficiently learn sign language. The system adopts an algorithm based on point cloud recognition to evaluate a user's sign and costs about 700ms of inference time for each sample, which meets the real-time requirements.
    Conclusion
    This interactive learning system decreases the communication barriers between deaf-mutes and hearing people.
  • Review

  • Data-driven simulation in fluids animation: A survey
    Qian CHEN, Yue WANG, Hui WANG, Xubo YANG

    DOI:10.1016/j.vrih.2021.02.002

    2021, 3(2) : 87-104

    Abstract(370) PDF (32) HTML (333)
    The field of fluid simulation is developing rapidly, and data-driven methods provide many frameworks and techniques for fluid simulation. This paper presents a survey of data-driven methods used in fluid simulation in computer graphics in recent years. First, we provide a brief introduction of physical-based fluid simulation methods based on their spatial discretization, including Lagrangian, Eulerian, and hybrid methods. The characteristics of these underlying structures and their inherent connection with data-driven methodologies are then analyzed. Subsequently, we review studies pertaining to a wide range of applications, including data-driven solvers, detail enhancement, animation synthesis, fluid control, and differentiable simulation. Finally, we discuss some related issues and potential directions in data-driven fluid simulation. We conclude that the fluid simulation combined with data-driven methods has some advantages, such as higher simulation efficiency, rich details and different pattern styles, compared with traditional methods under the same parameters. It can be seen that the data-driven fluid simulation is feasible and has broad prospects.
  • Article

  • Affine particle-in-cell method for two-phase liquid simulation
    Luan LYU, Wei CAO, Enhua WU, Zhixin YANG

    DOI:10.1016/j.vrih.2020.12.003

    2021, 3(2) : 105-117

    Abstract(319) PDF (34) HTML (318)
    Background
    The interaction of gas and liquid can produce many interesting phenomena, such as bubbles rising from the bottom of the liquid. The simulation of two-phase fluids is a challenging topic in computer graphics. To animate the interaction of a gas and liquid, MultiFLIP samples the two types of particles, and a Euler grid is used to track the interface of the liquid and gas. However, MultiFLIP uses the fluid implicit particle (FLIP) method to interpolate the velocities of particles into the Euler grid, which suffer from additional noise and instability.
    Methods
    To solve the problem caused by fluid implicit particles (FLIP), we present a novel velocity transport technique for two individual particles based on the affine particle-in-cell (APIC) method. First, we design a weighed coupling method for interpolating the velocities of liquid and gas particles to the Euler grid such that we can apply the APIC method to the simulation of a two-phase fluid. Second, we introduce a narrowband method to our system because MultiFLIP is a time-consuming approach owing to the large number of particles.
    Results
    Experiments show that our method is well integrated with the APIC method and provides a visually credible two-phase fluid animation.
    Conclusions
    The proposed method can successfully handle the simulation of a two-phase fluid.
  • Helmholtz decomposition-based SPH
    Zhongyao YANG, Maolin WU, Shiguang LIU

    DOI:10.1016/j.vrih.2021.01.003

    2021, 3(2) : 118-128

    Abstract(311) PDF (10) HTML (267)
    Background
    SPH method has been widely used in the simulation of water scenes. As a numerical method of partial differential equations, SPH can easily deal with the distorted and complex boundary. In addition, the implementation of SPH is relatively simple, and the results are stable and not easy to diverge. However, SPH method also has its own limitations. In order to further improve the performance of SPH method and expand its application scope, a series of key and difficult problems restricting the development of SPH need to be improved.
    Methods
    In this paper, we introduce the idea of Helmholtz decomposition into the framework of smoothed particle hydrodynamics (SPH) and propose a novel velocity projection scheme for three-dimensional water simulation. First, we apply Helmholtz decomposition to a three-dimensional velocity field and decompose it into three orthogonal subspaces. Then, our method combines the idea of spatial derivatives in SPH to obtain a discrete Poisson velocity equation. Finally, the conjugate gradient (CG) is utilized to efficiently solve the Poisson equation.
    Results
    The experimental results show that the proposed scheme is suitable for various situations and has higher efficiency than the current SPH projection scheme.
    Conclusion
    Compared with the previous projection scheme, our solution does not need to modify the particle velocity indirectly by pressure projection, but directly by velocity field projection. The new scheme can be well integrated into the existing SPH framework, and can be applied to the interaction of water with static and dynamic obstacles, even for viscous fluid.

Aims & Scope

Virtual Reality & Intelligent Hardware (VRIH) is an open access journal that aims to showcase and promote distinguished research in the field of virtual reality and intelligent hardware. It provides a global publishing and academic exchange platform for researchers, professionals and industry practitioners. The journal offers high-quality single-blind peer review and is published bimonthly in English.

Special Issues

  • Hand and gesture

    150 Browse

    Hand and gesture

    Intro:Hands play an important role in our daily life. We use our hands for manipulation in working, emphasi...

    2021 Vol. 3 No. 3

  • Simulation and interaction of fluid and solid dynamics

    343 Browse

    Simulation and interaction of fluid and solid dynamics

    Intro:Fluid and solid simulation is to generate a realistic simulation of fluids and solids, in particular ...

    2021 Vol. 3 No. 2

  • Emotion recognition for human-computer interaction

    489 Browse

    Emotion recognition for human-computer interaction

    Intro:Emotion recognition is to quantify,describe and recognize different emotional states through thebehav...

    2021 Vol. 3 No. 1

  • VR/AR research and commercial applications in Singapore

    478 Browse

    VR/AR research and commercial applications in Singapore

    Intro:The COVID-19 pandemic has virtually changed every aspect of our lives and society globally. This spec...

    2020 Vol. 2 No. 5

  • VR and experiment simulation

    484 Browse

    VR and experiment simulation

    Intro:In this special issue,the theme is on the application of VR/AR to experiments and training.We include...

    2020 Vol. 2 No. 4

  • 3D visual processing and reconstruction

    1242 Browse

    3D visual processing and reconstruction

    Intro:3D sensing represents the main channel through which humans, or robotics agents, understand and inter...

    2020 Vol. 2 No. 3