Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board

Virtual reality and augmented reality in medical simulation

Virtual reality/augmented reality (VR/AR) technologies have been widely used in medical fields, such as surgical simulation, healthcare, surgical navigation, computer assisted diagnosis, medical visualization and translational medicine. In this special issue, we include 6 research papers on the application of VR/AR to medical simulation, which can be divided into two sets, including surgical simulation and medical modelling/analysis.
View Abstracts Download Citations pdf Download E-Journal

EndNote

Reference Manager

ProCite

BiteTex

RefWorks

Editorial

Virtual reality and augmented reality in medical simulation

2021, 3(4) : 1-1

DOI:10.3724/SP.J.2096-5796.2021.03.04

PDF (15) HTML (264)

Article

Orthodontic simulation system with force feedback for training complete bracket placement procedures

2021, 3(4) : 261-273

DOI:10.1016/j.vrih.2021.08.001

Abstract (310) PDF (14) HTML (317)
Background
A virtual system that simulates the complete process of orthodontic bracket placement can be used for pre-clinical skill training to help students gain confidence by performing the required tasks on a virtual patient.
Methods
The hardware for the virtual simulation system is built using two force feedback devices to support bi-manual force feedback operation. A 3D mouse is used to adjust the position of the virtual patient. A multi-threaded computational methodology is adopted to satisfy the requirements of the frame rate. The computation threads mainly consist of the haptic thread running at a frequency of >1000Hz and the graphic thread at >30Hz. The graphic thread allows the graphics engine to effectively display the visual effects of biofilm removal and acid erosion through texture mapping. Using the haptic thread, the physics engine adopts the hierarchy octree collision-detection algorithm to simulate the multi-point and multi-region interaction between the tools and the virtual environment. Its high efficiency guarantees that the time cost can be controlled within 1 ms. The physics engine also performs collision detection between the tools and particles, making it possible to simulate paint and removal of colloids. The surface-contact constraints are defined in the system; this ensures that the bracket will not divorce from or embed into the tooth during the adjustment of the bracket. Therefore, the simulated adjustment is more realistic and natural.
Results
A virtual system to simulate the complete process of orthodontic bracket bonding was developed. In addition to bracket bonding and adjustment, the system simulates the necessary auxiliary steps such as smearing, acid etching, and washing. Furthermore, the system supports personalized case training.
Conclusions
The system provides a new method for students to practice orthodontic skills.
Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems

2021, 3(4) : 274-286

DOI:10.1016/j.vrih.2021.08.002

Abstract (239) PDF (8) HTML (262)
Background
Compared with traditional thoracotomy, video-assisted thoracoscopic surgery (VATS) has less minor trauma, faster recovery, higher patient compliance, but higher requirements for surgeons. Virtual surgery training simulation systems are important and have been widely used in Europe and America. Augmented reality (AR) in surgical training simulation systems significantly improve the training effect of virtual surgical training, although AR technology is still in its initial stage. Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice.
Methods
This study proposed an immersive AR lobectomy within a thoracoscope surgery training system, using visual and haptic modeling to study the potential benefits of this critical technology. The content included immersive AR visual rendering, based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling. Furthermore, we designed an AR haptic rendering systems, whose model architecture consisted of multi-touch interaction points, including kinesthetic and pressure-sensitive points. Finally, based on the above theoretical research, we developed an AR interactive VATS surgical training platform.
Results
Twenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system. Face, content, and construct validation methods were used to assess the tactile sense, visual sense, scene authenticity, and simulator performance.
Conclusions
The results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time. The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices.
Topological distance-constrained feature descriptor learning model for vessel matching in coronary angiographies

2021, 3(4) : 287-301

DOI:10.1016/j.vrih.2021.08.003

Abstract (264) PDF (2) HTML (224)
Background
Feature matching technology is vital to establish the association between virtual and real objects in virtual reality and augmented reality systems. Specifically, it provides them with the ability to match a dynamic scene. Many image matching methods, of which most are deep learning-based, have been proposed over the past few decades. However, vessel fracture, stenosis, artifacts, high background noise, and uneven vessel gray-scale make vessel matching in coronary angiography extremely difficult. Traditional matching methods perform poorly in this regard.
Methods
In this study, a topological distance-constrained feature descriptor learning model is proposed. This model regards the topology of the vasculature as the connection relationship of the centerline. The topological distance combines the geodesic distance between the input patches and constrains the descriptor network by maximizing the feature difference between connected and unconnected patches to obtain more useful potential feature relationships.
Results
Matching patches of different sequences of angiographic images are generated for the experiments. The matching accuracy and stability of the proposed method is superior to those of the existing models.
Conclusions
The proposed method solves the problem of matching coronary angiographies by generating a topological distance-constrained feature descriptor.
A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction

2021, 3(4) : 302-314

DOI:10.1016/j.vrih.2021.08.004

Abstract (278) PDF (2) HTML (269)
Background
Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia that can cause severe heart problems. Catheter ablation is one of the most ideal procedures for the treatment of AF. Physicians qualified to perform this procedure need to be highly skilled in manipulating the relevant surgical devices. This study proposes an interactive surgical simulator with high fidelity to facilitate efficient training and low-cost medical education.
Methods
We used a shared centerline model to simulate the interaction between multiple surgical devices. An improved adaptive deviation-feedback approach is proposed to accelerate the convergence of each iteration. The periodical beating of the human heart was also simulated in real time using the position-based dynamics (PBD) framework to achieve higher fidelity. We then present a novel method for handling the interaction between the devices and the beating heart mesh model. Experiments were conducted in a homemade simulator prototype to evaluate the robustness, performance, and flexibility of the proposed method. Preliminary evaluation of the simulator was performed by medical students, residents, and surgeons.
Results
The interaction between surgical devices, static vascular meshes, and beating heart mesh was stably simulated in a frame rate suitable for interaction.
Conclusion
Our simulator is capable of simulating the procedure of catheter ablation with high fidelity and provides immersive visual experiences and haptic feedback.
Development and application of digital assistive teaching system for anatomy

2021, 3(4) : 315-335

DOI:10.1016/j.vrih.2021.08.005

Abstract (275) PDF (4) HTML (252)
Background
Anatomy is a required course for all medicine-related industries. In recent decades, the teaching quality and effect of anatomy have been compromised by factors including a decrease in human body specimens, dampened enthusiasm for the discipline, reduced teaching hours of anatomy, scale expansion of medical education, and obstacles in performing field autopsies and observations.
Methods
Based on China's digitalized visible human research achievements, this article extracts the boundary information of anatomic structures from tomographic images, constructs three-dimensional (3D) digital anatomical models with authentic texture information, and develops an anatomy assistive teaching system for teachers and students based on the knowledge points of anatomy, to meet the anatomy teaching requirements of different majors at various levels.
Results
This scientific, complete, and holistic system has produced over 6000 3D digital anatomical models, 5000 anatomy knowledge points, 50 anatomical operation videos, and 150 micro demonstration classes, with teaching contents for different majors and levels, such as systematic anatomy, topographic anatomy, sectional anatomy, anatomy of motion, and virtual anatomical operation table. Ranging from network terminals, desktops, touchscreen 3D displays, desktops, and projection 3D volumetric displays to augmented reality, its diversified interactive forms meet the requirements for a learning environment in different settings.
Conclusions
With multiple teaching and learning links covered, such as teaching environment, teaching resources, instructional slides, autonomous learning, and learning effect evaluation, this novel teaching system serves as a vital component and a necessary resource in anatomy teaching and functions as an important supplement to traditional anatomy teaching. Applied and promoted in most medical colleges and schools in China, this system has been recognized and approved by anatomy teachers and students, and plays a positive role in guaranteeing the effect and quality of anatomy teaching.
A marching cube algorithm based on edge growth

2021, 3(4) : 336-349

DOI:10.1016/j.vrih.2021.08.006

Abstract (222) PDF (2) HTML (243)
The marching cube algorithm is currently one of the most popular three-dimensional (3D) reconstruction surface rendering algorithms. It forms cube voxels based on an input image and then uses 15 basic topological configurations to extract isosurfaces from the voxels. The algorithm processes each cube voxel in a traversal-based manner, but it does not consider the relationship between the isosurfaces in adjacent cubes. Owing to ambiguity, the final reconstructed model may have holes. In this paper, we propose a marching cube algorithm based on edge growth. The algorithm first extracts seed triangles, grows these seed triangles, and then reconstructs the entire 3D model. According to the position of the growth edge, we propose 17 topological configurations with isosurfaces. The reconstruction results showed that the algorithm can reconstruct the 3D model well. When only the main contour of the 3D model is required, the algorithm performs well. In addition, when there are multiple scattered parts in the data, the algorithm can extract only the 3D contours of the parts connected to the seed by setting the region selected based on the seed.