Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board

TABLE OF CONTENTS

2020,  2 (2):   104 - 118

Published Date:2020-4-20 DOI: 10.1016/j.vrih.2020.04.001

Abstract

Background
This study proposes a series of geometry and physics modeling methods for personalized cardiovascular intervention procedures, which can be applied to a virtual endovascular simulator.
Methods
Based on personalized clinical computed tomography angiography (CTA) data, mesh models of the cardiovascular system were constructed semi-automatically. By coupling 4D magnetic resonance imaging (MRI) sequences corresponding to a complete cardiac cycle with related physics models, a hybrid kinetic model of the cardiovascular system was built to drive kinematics and dynamics simulation. On that basis, the surgical procedures related to intervention instruments were simulated using specially-designed physics models. These models can be solved in real-time; therefore, the complex interactions between blood vessels and instruments can be well simulated. Additionally, X-ray imaging simulation algorithms and realistic rendering algorithms for virtual intervention scenes are also proposed. In particular, instrument tracking hardware with haptic feedback was developed to serve as the interaction interface of real instruments and the virtual intervention system. Finally, a personalized cardiovascular intervention simulation system was developed by integrating the techniques mentioned above.
Results
This system supported instant modeling and simulation of personalized clinical data and significantly improved the visual and haptic immersions of vascular intervention simulation.
Conclusions
It can be used in teaching basic cardiology and effectively satisfying the demands of intervention training, personalized intervention planning, and rehearsing.

Content

1 Introduction and motivations
Cardiovascular disease (CVD) remains the leading cause of mortality both in China and worldwide[1,2]. Percutaneous coronary intervention (PCI) is an effective treatment for CVD. The clinical treatment procedure of PCI includes: inserting a guidewire and catheter into the coronary artery via the femoral artery or radial artery, using a balloon to remove the occlusion in the coronary artery, and placing stents to improve the blood circulation[3,4]. All of these procedures are performed with guidance from real-time X-ray imaging. The number of PCI clinical cases in China has kept growing in recent years, and there were over 750,000 cases in 2017[5]. However, the amount of existing interventional cardiologists cannot meet the demand for treatment. A virtual intervention system, which combines virtual reality and medical techniques based on clinical image data, can provide an immersive virtual operating environment and process and help shorten the training cycle of cardiologists and improve training. Additionally, the virtual intervention system plays a significant role in decreasing life-threatening complications of both training and clinical treatment. Moreover, utilizing the virtual intervention system can effectively achieve intervention planning and rehearsal, which can further improve the quality of medical services[6,7,8].
The keys to developing a virtual intervention system lie in patient-specific blood vessel geometric and physical modeling and dynamic simulation of the vessels interacting with instruments during the operation. The modeling accuracy of blood vessels and instruments has a direct impact on the effectiveness and credibility of intervention planning and rehearsal. The dynamic simulation also plays a decisive role in the immersion of the virtual intervention system. There are many works dedicated to the modeling and simulation methods of the cardiovascular system and other human organs[9,10,11,12,13]. This study proposes a 4D modeling and simulation method for the cardiovascular system. Patient-specific blood vessel geometric and dynamic models are established from the clinical CTA data. Operating instruments are simulated via physics-based modeling and real-time resolution for their interactions with blood vessels. In addition, an operation instrument tracker with a haptic feedback function is developed and integrated into the virtual intervention system. The proposed method and integrated system, with high reality in visualization and haptic feedback, are significant for the promotion of immersive and interactive PCI simulation, and can well meet the demand for intervention training and rehearsal.
2 Related works
Virtual reality technologies have been gradually applied in the field of medicine. The development of a virtual intervention system can dramatically improve the effects of clinical medical diagnosis and reduce the risks and costs of surgical training. Research into virtual intervention systems stems from the United States and Sweden, and there are already mature intervention simulator products available. In 2004, Gallagher et al. proposed that cardiovascular interventions needed a new training mode urgently to deal with the risks of using patients for surgical training, and the outstanding role of virtual reality technology in laparoscopic surgery training at that time could well fill that need[14]. In 2007, Dawson et al. developed a training system for vascular intervention which simulated the operating process of angiography and stent placement[15]. Simbionics and Mentice developed several generations of intervention simulators in the years since 2007. The ANGIO MENTOR simulator made by Simbionics is the first commercial simulator that uses photoelectric sensors to detect the movement of surgical instruments and can be used for training and evaluation of vascular intervention. Mentice's VIST-LAB simulator has several surgical training modules, and can be used for training for operations on angiography and vascular intervention of various organs and tissues. Both of the vascular intervention simulators can provide one-dimensional haptic feedback and real-time X-ray image simulation, but they are unable to simulate three-dimensional anatomical human organs and tissues.
The studies on the virtual intervention systems of cardiovascular intervention start much later. Chang et al. developed a surgical simulation system for cerebrovascular intervention, which included visual feedback and haptic feedback, and were used for the training on basic operating skills of cerebrovascular intervention surgery[16]. Yang et al. used Autodesk 3ds Max to model and simulate human soft tissue organs such as bladder and liver and proposed the basic method for soft tissue simulation based on Autodesk 3ds Max [10]. However, such a modeling method is inefficient and unlikely to be widely used in processing personalized clinical image data. Xie et al. developed a haptic feedback device for cardiovascular intervention[17]. At present, although there are some research reports on the technologies related to cardiovascular intervention, the modeling and simulation methods for the whole operating process and PCI simulators with versatile functions have not yet been reported.
3 4D hybrid modeling for personalized cardiovascular systems
Due to the diversity of individual vascular structures and lesion locations, extracting a patient-specific cardiovascular model from clinical image data is the basis of conducting intervention training and rehearsal. To achieve this goal, we propose novel vessel geometric and dynamic modeling methods.
3.1 Geometric modeling based on personalized clinical image data
CTA is a common preoperative examination program for patients. Since X-rays cannot penetrate the contrast medium, angiographic images can be used to diagnose the location and type of vascular lesions. Figure 1 shows the workflow of a patient-specific vessel mesh model extracted from CTA. Given a sequence of clinical CTA slices, the images are first transferred to 3D volume data. Then, a graphics processing unit (GPU)-based level-set segmentation algorithm is employed to conduct semi-automatic segmentation on the volume data[18]. In this segmentation phase, user-specified seeds are placed at the region of interest (ROI) in the images to initialize the level set field, which is updated at all active elements according to the equation below:
Φ x , t = Φ x , t - Δ t + Δ t F x , t Φ x , t - Δ t
Where
F x , t
is the speed function,
x
denotes the coordinate in the image,
t
is the current time, and the scalar field
Φ x , t = 0
represents the 3D level set surface. For each vascular branch, a set of vessel section boundary contours can be automatically obtained with the segmentation algorithm after the seed points are set along its central line. Although this segmentation method is efficient, it cannot completely separate the blood vessels from the tissues or organs that they attach to. For this reason, we use spline fitting on contours and provide control vertices to allow manual adjustment. The adjusted contours can be further used to generate a single vessel mesh model by lofting. Finally, all the obtained vessel mesh models give rise to a complete cardiovascular system mesh model.
3.2 Motion modeling and simulation for personalized cardiovascular system
The cardiac system keeps beating during the whole process of intervention. To promote the fidelity and accuracy of virtual intervention, we develop a dynamic cardiovascular simulation system to recover and simulate the highly realistic beating motion of the whole cardiac system by integrating Position Based Dynamics (PBD)[19] and mass-spring models[20].
Given the reconstructed mesh model of the cardiac system, we first generate a tetrahedral model based on Delaunay tetrahedralization[21]. Then, elastic spheres[22] are placed at the vertices of each tetrahedron to represent the volumetric properties, and each pair of these spheres are connected with 3D springs, which control elongation, flexion, and torsion, as illustrated in Figure 2. The filling-sphere model of the cardiac system decouples local deformation and global deformation, namely, the applied forces will generate a local deformation first, and then propagate to the global model, which can well guarantee the system's stability.
Different from the mesh model of the cardiac system, the personalized 4D CTA data, which contains temporal and spatial information, is hard to obtain during clinical diagnosis and treatment procedures due to the difficulty and the clinical necessity of acquisition. To seek a universal 4D modeling method for the cardiovascular system, we extract the ventricle’s motion features from 4D MRI sequences consisting of a complete cardiac cycle. The extracted ventricle motion features are used as a template to drive personalized cardiac motion model in a visually-plausible way. As shown in Figure 3, ventricle mesh models are extracted from each frame of a complete cardiac cycle (Figure 3a-e). A bi-ventricular model for smooth, cardiac cycle motion is then obtained by interpolating linearly between frames based on the deformation fields. We then build a relationship between the personalized cardiac system model and the ventricular template model (Figure 3f), so that the ventricular motion can drive the kinematic motion modeling. Considering that the personalized cardiac system model has the same topological structure as the ventricular template model, we divide them into small regions and keep the correspondence of topological structure. Gaussian mapping is used to match two regions from different mesh models, and we model the personalized dynamic cardiac motion with a mass-spring system. In our implementation, the number of the divided sub-regions for each mesh is 700.
4 Modeling and interaction simulating for surgical instruments
Vascular intervention uses various kinds of guidewires and catheters as the major surgical instruments. The foundation of real-time simulation for the interactive motion process of the guidewire/catheter in the blood vessel is the physical modeling of the guidewire/catheter. In this study, a continuous physical guidewire model is constructed based on the continuous Cosserat[23] theory, so that the guidewire/catheter has the freedom of both forward/backward movement and torsion movement. The physical model is further discretized to make it more operable. We define constraints and the energy-constrained Lagrangian motion equation to satisfy the physical model of the guidewire/catheter. The interaction between the guidewire/catheter and the blood vessel is calculated with a collision detection algorithm. To realize the simulation of the guidewire/catheter during the operation, the displacements and motion states of the guidewire/catheter are updated via solving the motion equation in an implicit Euler method.
4.1 Physical modeling for the guidewire/catheter
The center lines can approximate the spatial shape of the guidewire/catheter, and the volume of the guidewire/catheter can be neglected. To realize the continuous spatial shape description of the thin elastic rod, the center lines can be expressed with a curve
r ( σ ) = ( r x ( σ ) , r y ( σ ) , r z ( σ ) ) T
. In the curve expression
r ( σ ) : [ 0,1 ] R 3
represents the position specified by line element
σ [ 0,1 ]
on the center line of a thin elastic rod. The spatial position can only describe the bending and tension of the thin elastic rods but is not capable in describing the torsion of the cross-section of the thin elastic rods. For this purpose, an orthogonal coordinate system is established at each point on the center line of the thin elastic rod. The coordinate axis directions are
d 1 ( σ )
,
d 2 ( σ )
and
d 3 ( σ )
. The orthogonal coordinate system is used to help represent the forwarding, bending, and torsion of the thin elastic rods. Particularly,
d 1 ( σ )
parallel to the tangent direction of the guidewire/catheter center line, which means the following formula should be satisfied:
r ' r ' - d 1 = 0
Here
r '
represents the spatial derivative of the center line. This constraint couples the center line and direction information of the guidewire/catheter. Figure 4 shows a continuous physics-based model for the guidewire and catheter.
The norm
r '
express stretch forming of the center line
r
in position
σ
, and the spatial derivatives in directions
d k ,   k = 2 ,   3
represent spatial bending and torsion of the guidewire/catheter. To simplify motion description and simulation calculation, quaternion is used to represent rotation instead of a 3D coordinate system. The physical model of the continuous guidewire/catheter is discretized to solve the motion and deformation equations of the guidewire/catheter. The center line is divided into N discrete spatial control points
r i = r ( σ i , t ) R 3 ,   i [ 1 , N ]
. Afterward,
r i r i + 1
is an element on the center line. Its length can be modified. The center line element has its direction and is used to describe the bending and rotation of the guidewire/catheter. The directions of the center line elements are expressed by quaternions
q j ,   j [ 1 , N - 1 ]
. The discretized physics-based model of guidewire/catheter is shown in Figure 5.
After discretization, the spatial derivative of the center line is obtained by:
r i ' = r i + 1 - r i r i + 1 - r i
If the tensile stiffness coefficient of the guidewire is very large, the spatial derivative of the center line can be obtained:
r i ' 1 l i ( r i + 1 - r i )
Of which,
  l i = r i + 1 0 - r i 0
, and
r i 0
is the initial position of the control vertex
r i
.
The spatial derivative of the direction of the central line element can be approximately expressed as:
q j ' 1 l j ( q i + 1 - q i )
Where
l j = 1 2 ( r i + 1 0 - r i 0 + r i + 2 0 - r i + 1 0 )
.
Since the discretized physical model only records the coordinates and direction information at the control points, it is necessary to interpolate the points between the center line elements, which can greatly simplify the expression and calculation of the guidewire/catheter motion.
4.2 Representation of guidewire/catheter motion
Based on the physical model of the guidewire/catheter, the motion of the guidewire/catheter is described by the Lagrangian motion equation with constraint conditions. This kinematics model needs two integrity constraints; Eq.6 couples the position of the particle on the guidewire/catheter center line with the direction of the central line element, and Eq.7 ensures that quaternions can represent the rotation correctly.
C p = r ' r ' - d 1 = 0
C q = q - 1 = 0
By introducing the Lagrange multiplier, the motion prototype expression of the guidewire/catheter becomes
d d t T g ˙ i - T g i + V g i + D g ˙ i + λ C p g i + μ C q g i = 0 1 F e d σ
In this equation,
T
is the kinetic energy of the guidewire/catheter that can be calculated from translational kinetic energy and cross-sectional kinetic energy of the center line.
V
is the potential energy of guidewire/catheter and is calculated from the potential energy of tensile and bending deformation.
D
is dissipation energy and is calculated with the internal friction coefficient.
F e
is the combined external force and moment, and
g i { r x , r y , r z , q 1 , q 2 , q 3 , q 4 }
is the generalized coordinate.
λ
and
μ
are constraint coefficients. By solving the equation, the motion state and displacement of the guidewire/catheter can be updated, and the motion simulation results of the guidewire/catheter can be obtained.
4.3 Interaction simulation between instrument and blood vessel
It is necessary to simulate the interaction between the guidewire/catheter and the blood vessel, as well as the physical deformation and movement of the guidewire/catheter itself. In cardiovascular intervention, the guidewire/catheter maintains its motion in vessels under the constraints of the vessel wall. Therefore, the simulation process is realized with the collision detection and response between the guidewire/catheter and the vessel wall.
Collision detection between the guidewire/catheter and vessels has the following characteristics. First, during the insertion process of the guidewire/catheter, the basic geometric unit topological shape of the guidewire/catheter will change. Second, the problem of self-collision is not involved, because the guidewire/catheter travels in narrow vessels. Based on the characteristics above, considering that the collision detection with the spatial hash grid does not need to update the bounding box in real-time as the method is based on a hierarchical bounding box, which can avoid the inefficiency of the method caused by the change of geometric topological shape, our method adopts a collision detection method based on a spatial hash[24] collision grid.
Figure 6 shows the flow chart of the collision detection algorithm based on a hash grid. The unit side length of the spatial hash grid is selected first. Then the spatial hash grid is established, and the collision detection unit is mapped into the hash grid. The index of the collision unit is built to detect whether there is a tetrahedron intersection and to record the collision point. A GPU is used for parallel processing in building the axis-aligned bounding box[25], calculating the tetrahedral hash value and inverse matrix of the tetrahedron, and resolving collision elements in the hash grid.
When two colliding objects with different grid densities collide at one vertex, their penetration vectors are different. The interaction between the guidewire/catheter and vessels involve different mesh densities, which inevitably result in different penetration vectors. The direction of the vascular penetration vector is the normal direction of the intersection plane, and the direction of the guidewire/catheter penetration vector is the tangent direction of the guidewire/catheter in the penetration part. The calculation formula of the response force is as follows:
F = ( E 1 + E 2 ) × i = 1 n δ i n i j = 1 m δ j n j i = 1 n δ i n i + j = 1 m δ j n j
Here
E 1
and
E 2
are collision rigidity coefficients of the blood vessel and guidewire/catheter,
δ i
and
δ j
are the penetration depth of the blood vessel and guidewire/catheter,
n i
and
n j
are the penetration vectors of blood vessel and guidewire/catheter. Based on the collision detection and force calculation between the guidewire/catheter and the vessel, the complete simulation of motion and interaction of the guidewire/catheter in vessels is shown in Figure 7. First, the physical model of the guidewire/catheter is established, and the Lagrangian motion equation is constructed. The collision detection between the guidewire/catheter and the vessel is calculated, and the force is substituted into the Lagrangian motion equation. The implicit Euler method is used to solve the motion equation to update the state of the guidewire/catheter. Repeat the process until the whole guidewire/catheter enters the vessel and interacts with it.
5 X-Ray imaging simulation
5.1 Blending deep maps of the organs and tissues
Real-time X-ray fluoroscopy is a prime reference for doctors to trace and diagnose during intervention. The accuracy and fidelity of X-ray imaging simulation in a virtual intervention system are crucial for surgical training and rehearsal. The mesh models of the vessels and related organs used in the virtual intervention system are based on clinical CT data. Considering the differences of individual blood vessels, the trend of vessels, and organ characteristics, there is inevitable occlusion and intersection between the meshes of vessels and different organs. This creates a great challenge in sorting the vessels in the direction of radiation and in calculating radiation energy attenuation, which affects the real-time simulation of cardiovascular X-ray images. We propose a sequence-independent depth map blending algorithm, and to use the blending results to calculate the tissue thickness of the branches and organs in the direction of radiation.
As shown in Figure 8, to set the parameters of the scene camera in the 3D virtual scene, the location of the ray source is taken as the position of the scene camera, and the direction of light emission is taken as the direction of observation. The projection coordinates of each vertex are calculated by using the projection matrix of the world viewpoint in the vertex shading pipeline. The third component of the projection coordinates is the depth value related to the ray source of the vertex. In the vertex shading pipeline, the depth values of all the vertices are normalized to [0, 1]. The smaller the depth values are, the closer the vertex is to the ray source. Conversely, when the depth approaches 1 it indicates that the vertex is far from the source. Therefore, the depth value can be used to judge the occlusion relationship of vertices in the ray direction and the thickness of vessels and organs in the ray direction represented by the mesh model. By mapping the depth value to the color value and eliminating the triangular patches with larger depth value, the frontal depth map of the 3D scene can be obtained. By eliminating triangular patches with smaller depth values, the backward depth map of the scene can be obtained.
To obtain the thickness of vessels and organs in the ray direction with high efficiency, depth maps are used for sequence-independent blending operations. The algorithm is as follows. The linked list structure is constructed and stored in the Fragment Shader. The linked list elements record the attributes of the elements contributing to the final color of the pixel, including color, alpha value, and depth value. The total number of fragments added to the list is then recorded. For each pixel, a separate linked list is generated, which contains all the element attributes contributing to the pixel. All pixels are stored in the same buffer image. Each pixel needs to save its header pointer and store it in the 2D image with the same size of the frame buffer. An atomic operation is used to update the header pointer, and atomic exchange operation is used to ensure that multiple Shaders do not add operations to the same list, so it will not affect the results of the others. After the construction of the linked list data structure, in the second rendering process, the Fragment Shader traverses the linked list corresponding to the pixels and ranks the fragment metadata in the linked list in depth. When all the elements are arranged in depth order, the blending operation can be completed from back to front. Finally, the thickness of the vessel and organ model corresponding to the projection position can be obtained by calculating the difference between the front and back depth.
5.2 X-ray imaging simulation based on energy attenuation
Bones contain much calcium, which due to its relatively high atomic number absorbs X-rays efficiently. This reduces the number of X-rays that reach the detector in the shadow of the bones, making them clearly visible on the radiograph. Other organs and tissue types also absorb rays to different extents. In order to generate real-time virtual X-ray images, we use the Lambert-Beer law to simulate the process of X-ray penetration and calculate intensity attenuation. When a narrow-beam monochrome X-ray passes through a medium with uniform density and atomic number, assuming the thickness of the medium is
L
, and the absorption coefficient of the medium is
μ x
, the Lambert-Beer law describes the relationship between the intensity
I
of the transmitted ray and the intensity
I 0
of the incident ray via:
I = I 0 e - μ x L
In our implementation, we simulate the X-ray penetration for each vessel and organ separately. We consider that the density and atomic number of a tissue type are approximately uniform. If an X-ray passes through multiple objects, the effect of attenuation is cumulative. Based on the results of depth maps blending, we simulate rays traversing the sorted sequences from the ray source to the final synthetic image. In this process, the final intensity is calculated according to the Lambert-Beer law. Generated virtual X-ray images are shown in Figure 9.
6 Design of instrument tracker with haptic feedback
Tracking equipment with a haptic feedback function for surgical instruments is developed so that the trainees can use real surgical instruments to interact with the virtual vascular intervention and experience the operating process vividly. The instruments related to displacement and haptic interaction in vascular intervention mainly include guidewire, catheter, balloon guidewire, and stent guidewire. Therefore, the virtual intervention system of vascular intervention should be able to collect the following feedback information: real-time acquisition of the guidewire/catheter movement (including forward and backward movement and rotation movement) and real-time feedback of the resistance of the doctor's operation caused by friction and collision of the guidewire/catheter in the blood vessel (including forward and backward resistance and rotation resistance). Trainees operate the guidewire/catheter through the blood vessel by pushing, pulling, and twisting. The guidewire/catheter has two types of free motions: forward and backward motion along the axis and circumferential rotation motion, i.e., feedback force, including push-pull force and twist force.
The instrument tracker designed in this study can track the motion of the guidewire/catheter and provide 2-DOF motion feedback and 2-DOF haptic feedback at any position along the axis of the guidewire/catheter. As shown in Figure 10, the instrument tracker is composed of a displacement tracking module, haptic feedback mechanism, drive motor, motor driver, guide rail, controller, etc. During the operation of virtual intervention, the detection equipment is connected with the simulation computer. Through the main/auxiliary tracking module, the tracker recognizes the surgical instruments entering the pipe with a machine vision method and follows the guidewire/catheter along the axis direction. The information on the surgical instruments and the motion parameters, such as type, position, velocity, and acceleration, are transmitted to the computer. According to the above information, a virtual guidewire/catheter model is constructed and simulated, and the simulated image is rendered in real-time. At the same time, the feedback force of the interaction between the guidewire/catheter and different parts of the virtual cardiovascular system is calculated, which is decomposed into the push-pull feedback force and twist feedback force. The control signal is obtained by calculation, and the controller transmits it to the motor driver to drive the motor and make the feedback module work. The haptic feedback module and the main/auxiliary tracking module do not interfere with each other and can move along the guide rail to exert feedback force at any part of the guidewire/catheter. The guidewire/catheter transfers the virtual feedback force to the trainees to achieve the training purpose of the virtual intervention.
7 System integration and result analysis
As shown in Figure 11, the virtual intervention system for vascular intervention is built mainly with the proposed cardiovascular 4D modeling and simulation method, surgical instrument modeling, and simulation method. Instrument trackers with haptic feedback function and hardware-in-the-loop simulation modules such as air pumps and injectors are also involved. The software was deployed on a desktop computer with an Intel Core processor clocked at 3.7 GHz, with 8 GB RAM and an NVIDIA GTX 1060 GPU. The integrated virtual vascular intervention system supports personalized clinical data 4D modeling, uses real surgical instruments for interactive operation, provides a real-time simulation of the whole process of vascular intervention with haptic feedback, and can carry out effective surgical training, preoperative planning and rehearsing.
7.1 Result analysis of cardiovascular 4D modeling
Figure 12 shows the 4D modeling results of the cardiovascular system and motion simulation results during cardiac contraction and diastole. We demonstrate the visualization effect of the extracted vascular mesh model. The vessels can follow the cardiac deformation movement in the cardiac cycle. On the one hand, the simulation results can be used to simulate the angiographic images in the system. On the other hand, as a real-time linkage between 3D anatomical views and 2D X-ray angiographic images, the simulation results can guide the trainees to establish a corresponding relationship between 2D images and 3D space in operation training. Cardiovascular X-ray simulation results based on a deformable mesh model are shown in Figure 13, with the location of cardiovascular lesions marked. The rendering method of the angiographic effect presented in this paper has been validated on some personalized data. It can vividly and robustly render the filling and dissipation process of contrast media at all levels of blood vessels or lesions.
Table 1 shows the data scale and the processing time required for the extraction of the personalized vascular model from clinical CT data. Level set segmentation and lofting of multi-branch blood vessels are performed on two sets of different CT data. The cumulative time of automatic processing is 2 to 3 minutes. The proposed model extraction method can process preoperative exam data in real-time and has strong practicability in surgical planning and rehearsing.
Statistics of the personalized vessel mesh model construction from CT data
CT data No.1 CT data No.2
Volume data size 512×512×276 512×512×224
Number of vascular branches involved in modeling 28 24
Number of extracted contour lines of blood vessels 341 309
Number of triangular patches after lofting 205804 437602
Cumulative automatic processing time (s) 141.85 188.912
Average manual processing time (minutes) 15.2 19.5
7.2 X-ray imaging simulation results
Figure 14 shows the X-ray image simulation results of surgical instruments used in vascular intervention. Figure 15 shows the X-ray image simulation of the treatment process, from left to right: balloon (not dilated), balloon (after dilation), and angiography effect after smashing lesions, and finally, the dissipation of contrast media. The performance of the X-ray image simulation algorithm is shown in Figure 16. When the number of triangular facets in the 3D scene exceeds 200,000, the frame rate of the X-ray simulation image can reach 125 FPS, which means the high computation efficiency and processing ability of the simulation method meets the demand of real-time interactive performance in the virtual intervention system.
7.3 Functional analysis of vascular interv-ention simulation system
The usability of the virtual vascular intervention system is evaluated by 55 cardiologists. Statistical results are shown in Figure 17. Most of the participants have the consensus that our system can well contribute to cardiologists training. Research about virtual intervention systems has been carried out in the United States, Canada, Sweden, and some mature vascular intervention training systems have been put into practical application as products. Representative systems include VIST-LAB and ANGIO MENTOR. We compare our virtual vascular intervention system with these two systems in functions of X-ray imaging simulation, 3D anatomical view, personalized data processing, and training effect evaluation, which are closely related to the virtual intervention objectives. The results are shown in Table 2.
Function comparison among several vascular intervention simulation systems
Functions VIST-LAB ANGIO MENTOR Our system
Personalized modeling data interface × ×
X-ray imaging simulation √(Pleural only)
3D dynamic simulation of organs × ×
Real-time linkage between 3D and 2D × ×
Operation record
Evaluation of virtual intervention
Compared with the virtual intervention system of vascular intervention, which has been put into application, our virtual intervention system has obvious advantages in personalized data-based modeling, real-time 3D dynamic simulation of human organs and tissues, X-ray imaging, and 3D simulation linkage. Besides, it achieves or surpasses the level of similar systems in X-ray imaging simulation, operation recording, and operation evaluation of trainees. The system supports instant modeling and simulation of personalized clinical data to meet the demands of virtual intervention rehearsal. It can be used in teaching basic cardiology, surgical skills training, personalized surgical planning and rehearsal, surgical effect evaluation, new surgical technology experiments, etc. The cardiovascular intervention system is now being used in Peking Union Medical College Hospital, Beijing Anzhen Hospital, Beijing Chaoyang Hospital, and other hospitals in China.
8 Conclusions
We proposed a series of geometric and physics modeling methods for the simulation of full operating procedures of personalized cardiovascular intervention, which were applied in a virtual intervention simulator. Based on personalized clinical CTA data, the cardiovascular geometric model and the dynamic model were built semi-automatically. Operating instruments were simulated using physics-based models, and the interactions between instruments and vessels were resolved in real-time with efficient collision detection. Additionally, we proposed a simulation algorithm for cardiovascular X-ray imaging using a depth map blending and a realistic rendering method based on the results of 4D modeling, simulation of the cardiovascular system, and the motion simulation of surgical instruments. An operating instrument tracker with a haptic feedback function was also developed and integrated into the virtual intervention system. The proposed method and integrated system have high reality in visualization and haptic feedback, which can significantly improve the visual and haptic immersion of vascular intervention simulation, and effectively satisfy the demands of intervention training, personalized intervention planning, and rehearsing.

Reference

1.

Chen W W, Gao R L, Liu L S. China cardiovascular disease report synopsis. Chinese Circulation Journal, 2018, 33(1): 1–8

2.

Benjamin E J, Blaha M J, Chiuve S E, Cushman M, Das S R, Deo R, de Ferranti S D, Floyd J, Fornage M, Gillespie C, Isasi C R, Jiménez M C, Jordan L C, Judd S E, Lackland D, Lichtman J H, Lisabeth L, Liu S, Longenecker C T, Mackey R H, Matsushita K, Mozaffarian D, Mussolino M E, Nasir K, Neumar R W, Palaniappan L, Pandey D K, Thiagarajan R R, Reeves M J, Ritchey M, Rodriguez C J, Roth G A, Rosamond W D, Sasson C, Towfighi A, Tsao C W, Turner M B, Virani S S, Voeks J H, Willey J Z, Wilkins J T, Wu J H, Alger H M, Wong S S, Muntner P. Heart disease and stroke statistics-2017 update: a report from the American Heart Association. Circulation, 2017, 135(10): e146–e603 DOI:10.1161/CIR.0000000000000485

3.

George A S. Percutaneous Coronary Intervention (PCI): practice essentials, background, indications. https://emedicine.medscape.com/article/161446-overview

4.

Serruys P W, Morice M C, Kappetein A P, Colombo A, Holmes D R, Mack M J, Ståhle E, Feldman T E, van den Brand M, Bass E J, van Dyck N, Leadley K, Dawkins K D, Mohr F W. Percutaneous coronary intervention versus coronary-artery bypass grafting for severe coronary artery disease. New England Journal of Medicine, 2009, 360(10): 961–972 DOI:10.1056/nejmoa0804626

5.

Liu H S. Data on interventional treatment of cardiovascular diseases in China in 2017. China Medical News, 2018, 33(9):22–23

6.

Hooshiar A, Najarian S, Dargahi J. Haptic telerobotic cardiovascular intervention: a review of approaches, methods, and future perspectives. IEEE Reviews in Biomedical Engineering, 2019: 1 DOI:10.1109/rbme.2019.2907458

7.

Gillio R G. Virtual surgery system. US Patent: 5882206, 1999

8.

Deng Z L, Xie L, Luo M S. Virtual surgery contrast development simulation with cardiovascular intervention. China Digital Medicine, 2015, 10(2): 29–31 DOI:10.3969/j.issn1673-7571.2015.2.007

9.

Zhang J G. Research on the soft tissue deformation modeling methods in virtual surgery. Harbin Engineering University, 2016

10.

Yang Z. A study on modeling and simulation of human soft tissue organs based on 3DsMAX. Air Force Medical University, 2015

11.

Grund-Pedersen J. Interventional simulator system. US Patent: 8083524, 2011

12.

Tang D L. Multi-modality image-based modeling approach for cardiovascular disease: simulation, assessment, prediction, and virtual surgery. Molecular & Cellular Biomechanics, 2019, 16(s1): 11 DOI:10.32604/mcb.2019.05170

13.

Li S, Xie Z J, Xia Q, Hao A M, Qin H. Hybrid 4D cardiovascular modeling based on patient-specific clinical images for real-time PCI surgery simulation. Graphical Models, 2019, 101: 1–7 DOI:10.1016/j.gmod.2018.11.002

14.

Gallagher A G, Cates C U. Virtual reality training for the operating room and cardiac catheterisation laboratory. The Lancet, 2004, 364(9444): 1538–1540 DOI:10.1016/s0140-6736(04)17278-4

15.

Dawson D L, Meyer J, Lee E S, Pevec W C. Training with simulation improves residents' endovascular procedure skills. Journal of Vascular Surgery, 2007, 45(1): 149–154 DOI:10.1016/j.jvs.2006.09.003

16.

Chang H. Research on simulation system of cerebrovascular interventional surgery. Harbin Institute of Technology, 2008

17.

Xie L, Shen X L, Wu Z L, Deng Z L, Gu L X, Cai P, Hao A M. The simulator development of cardiovascular interventional virtual surgery with force feedback. Journal of Jiangxi Normal University (Natural Sciences Edition), 2017, 41(4): 331–337 DOI:10.16357/j.cnki.issn1000-5862.2017.04.01

18.

Roberts M, Packer J, Sousa M C, Mitchell J R. A work-efficient gpu algorithm for level set segmentation. Proceedings of the Conference on High Performance Graphics, Switzerland, 2010, 123–132

19.

Müller M, Heidelberger B, Hennix M, Ratcliff J. Position based dynamics. Journal of Visual Communication and Image Representation, 2007, 18(2): 109–118 DOI:10.1016/j.jvcir.2007.01.005

20.

Mollemans W, Schutyser F, van Cleynenbreugel J, Suetens P. Tetrahedral mass spring model for fast soft tissue deformation. Proceedings of Surgery Simulation and Soft Tissue Modeling. Berlin: Springer, 2003: 145–154 DOI:10.1007/3-540-45015-7_14

21.

Si H. TetGen, a delaunay-based quality tetrahedral mesh generator. ACM Transactions on Mathematical Software, 2015, 41(2): 1–36 DOI:10.1145/2629697

22.

Conti F, Khatib O, Baur C. Interactive rendering of deformable objects based on a filling sphere modeling approach. In: 2003 IEEE International Conference on Robotics and Automation. Taipei, 2003DOI:10.1109/robot.2003.1242167

23.

Rubin M B. Cosserat theories: shells, rods and points. Springer Netherlands, 2000 DOI:10.1007/978-94-015-9379-3

24.

Teschner M, Heidelberger B, Müller M. Optimized spatial hashing for collision detection of deformable objects. In: International Symposium on Vision, Modeling, and Visualization. Munich, 2003, 19–21

25.

van den Bergen G. Efficient collision detection of complex deformable models using AABB trees. Journal of Graphics Tools, 1997, 2(4): 1–13 DOI:10.1080/10867651.1997.10487480