Study on the adaptability of augmented reality glasses to astigmatism based on holographic waveguide grating
In order to meet the requirement that astigmatism users can use AR smart glasses without wearing a diopter lens, a cylindrical lens waveguide grating is designed based on the principle of holographic waveguide grating. First, a cylindrical lens waveguide substrate is constructed to deflect the external light to satisfy the astigmatism users' normal viewing of the real world. Then a variable period grating structure is established based on the cylindrical lens waveguide substrate to make the light from the virtual world in the optical machine emit normally to the human eyes. Finally, the structure parameters of the grating are optimized to improve the diffraction efficiency. The results show that the structure of cylindrical lens waveguide grating can realize the function of wearing AR smart glasses directly for astigmatism users. The total light utilization rate can reach 90% and the imaging uniformity is excellent. The brightness difference is less than 0.92% and the vertical field of view angle is 10°. This research has a guiding contribution to AR product design for people with far/near vision and promotes the expansion of product types.
Survey on path and view planning for UAVs
In recent decades, unmanned aerial vehicles (UAVs) have developed rapidly and been widely used in many domains, such as photography, reconstruction, monitoring, search, and rescue. In these missions, one key issue is the path and view planning, which tells UAVs where exactly to fly and how to look. With specific consideration on three popular UAV applications (scene reconstruction, environment exploration and aerial cinematography), we provide a survey, that will assist researchers to better position and evaluate their works in the context of existing solutions, and also help newcomers and practitioners in related fields to quickly gain an overview of the vast literature. In addition to the current research status, both advantages and disadvantages together with the potential explorative trends of each application domain are elaborated upon and analyzed.
A smart assistance system for cable assembly combining wearable augmented reality with portable visual inspection
The assembly task guided by paper document is the most widespread form in aircraft cable assembly process by far. This form is very complicated and requires high level skills of assembly workers. Technologies of wearable Augmented Reality (AR) and portable visual inspection provide an effective way to improve the efficiency and the quality of cable assembly. This paper proposed a smart assistance system for cable assembly combining wearable AR with portable visual inspection. Specifically, a portable visual device based on binocular vision and deep learning is developed to realize fast detection and recognition of cable brackets which is installed on aircraft airframe. Then a Convolutional Neural Network (CNN) is constructed for reading the texts on cable after gathering image from the camera on wearable AR device. An authoring tool developed for creating and managing the assembly process is proposed for realizing the visual guidance of cable assembly based on wearable AR device. The system is applied to the cable assembly on an aircraft bulkhead prototype. The results show that the system can recognize the number, types and locations of brackets, and can read the text of aircraft cable correctly. The authoring tool can help the user who has no professional programming experience to establish a process plan, i.e. assembly outline based on AR for cable assembly quickly and easily. The system is able to realize quick assembly guidance to aircraft cable with text, image and 3D model. It is beneficial for reducing the dependency on the paper document, the labor intensity and the error rate.
View synthesis from multi-view rgb data using multi-layered representation and volumetric estimation
Aiming at free-view exploration of complicated scene, this paper presents a method for interpolate views among multi RGB cameras. In this paper, we combine the idea of both cost volumes which represent the 3D information, and 2D semantic segmentation of the scene, to accomplish view synthesis of complicated scenes. We use the idea of cost volume to estimate the depth and confidence map of the scene, as well as using the multi-layer representation and resolution of the data to optimize the view synthesis of the main object. With different treatment of different layers of the volume, we can handle complicated scene containing multiple person and plentiful occlusion. We also propose the view-interpolation——multi-view reconstruction ——view interpolation pipeline to iteratively optimize the result. We test our method on varying data of multi-view scene and generate decent results.
A two-phase real-time rendering method for realistic water refraction
Realistic rendering has been an important goal of many interactive applications, which requires an efficient virtual simulation of many special effects that are common in the real world. However, refraction is often ignored in these applications. Rendering the refraction effect is quite complicated and time-consuming. In this study, a simple efficient and fast rendering technique of water refraction effects is proposed. This technique consists of a broad phase and a narrow phase. In the broad phase, the water surface is considered flat. The vertices of underwater meshes will be transformed based on Snell’s Law. In the following narrow phase, influences of waves on the water surface are involved. Every pixel on the water surface mesh is collected through a screen-space method with an extra GPU pass. The broad phase guarantees redirection of most pixels that need to be recalculated in the narrow phase to the pixels in the rendering buffer. Experiment results show the realism and efficiency of the proposed technique.
Personalized cardiovascular intervention simulation system
This paper proposes a series of geometry and physics modeling methods for the full operation procedures of personalized cardiovascular intervention, which in all have been applied to a virtual endovascular simulator. Given personalized clinical computed tomography angiography (CTA) data, mesh models of the cardiovascular system are firstly constructed in a semi-automatic way. By coupling 4D magnetic resonance imaging (MRI) sequences corresponding to a complete cardiac cycle with related physics models, a hybrid kinetic model of the cardiovascular system is built to drive kinematics and dynamics simulation. On that basis, the operation procedures related to intervention instruments are simulated based on specially-designed physics models, which can be solved in real-time, so that the complex interactions between vessels and instruments can be well simulated. Meanwhile, X-ray imaging simulation algorithms and realistic rendering algorithms for virtual intervention scenes are also proposed. Especially, an instrument tracking hardware with haptic feedback is developed to serve as the interaction interface of real instruments and the virtual intervention system. Finally, a personalized cardiovascular intervention simulation system is developed by integrating the aforementioned techniques, which can significantly improve the visual and haptic immersions of vascular intervention simulation, effectively satisfy the demands of intervention training, personalized intervention planning and rehearsing.
The research of ghost suppression method in polarized catadioptric virtual reality optical system
In this paper, a polarized catadioptric virtual reality optical system is introduced. Aiming at the problem of serious ghost image in the system, the causes are analyzed based on the design principles and optical structure. The distribution of stray light is simulated by using Lighttools, and three major ghost paths are selected by using the area of diffuse spot, and the energy ratio of stray light, K as the evaluation ways. A method to restrain ghost image by optimizing the structure of optical system by controlling the focal power of ghost image path is proposed. The results show that the of ghost image path increases by 40% and K decreases by 40% compared with before optimization. Ghost image is effectively suppressed, which provides theoretical basis and technical support for ghost suppression in virtual reality optical system.