Background　 Combining the use of an AR headset and a smartphone can provide wider display and precise touch input simultaneously; it can redefine the way we use applications today. Unfortunately, users are deprived of such benefit because of the independence of two devices. There lacks a kind of intuitive and direct interactions across them. In this paper, we conduct a formative study to understand the window management requirements and interaction preferences of using an AR headset and a smartphone simultaneously and report the insights we gained. Also, we introduce an example vocabulary of window management operations in AR headset + smartphone interface. It allows users to manipulate windows in virtual space and shift windows between devices efficiently and seamlessly.

Background　 With an increasing number of products becoming digital, mobile, and networked, paying attention to the quality of interactions with such products is also becoming more relevant. Although the quality of such interactions has been addressed in several scientific studies, little attention has been paid to their implementation in real life and everyday contexts. Methods This paper describes the development of a novel office phone prototype, called YPhone, which demonstrates the application of a specific set of Generation Y interaction qualities (instantaneous, playful, collaborative, expressive, responsive, and flexible) in the context of office work. The working prototype supports office workers in experiencing new types of interactions. It was set out in practice through a series of evaluations. Results We found that the playful, expressive, responsive, and flexible qualities incur greater trust than the instantaneous and collaborative qualities. Such qualities can be grouped, although this may differ for different evaluated products, and researchers must be cautious about generalizations. Conclusions The overall evaluation was deemed positive, with some valuable suggestions provided regarding its user interactions and features.

Background In this study, we propose a novel 3D scene graph prediction approach for scene understanding from point clouds. Methods It can automatically organize the entities of a scene in a graph, where objects are nodes and their relationships are modeled as edges. More specifically, we employ the DGCNN to capture the features of objects and their relationships in the scene. A Graph Attention Network (GAT) is introduced to exploit latent features obtained from the initial estimation to further refine the object arrangement in the graph structure. A one loss function modified from cross entropy with a variable weight is proposed to solve the multi-category problem in the prediction of object and predicate. Results Experiments reveal that the proposed approach performs favorably against the state-of-the-art methods in terms of predicate classification and relationship prediction and achieves comparable performance on object classification prediction. Conclusions The 3D scene graph prediction approach can form an abstract description of the scene space from point clouds

Background The personality and feedback of an animated pedagogical agent (APA) are vital social-emotional features that render the agent perceptually believable. Their effects on learning during virtual training need to be examined. Methods In this paper, an explanation model is proposed to clarify the underlying mechanism of how these two features affect learners. Two studies were conducted to investigate the model. In Study 1, the effect of the APA's personality type and feedback strategy on flow experience and performance was reexamined, revealing significant effects of the feedback strategy on flow and performance and a marginally significant effect of the personality type on performance. To explore the mechanism behind these effects, a theoretical model is proposed by distinguishing between intrinsic and extrinsic motivation effects. In Study 2, the model was evaluated, and the APA's personality type was found to significantly influence the factors in the path of the extrinsic motivation effect rather than those in the path of the intrinsic motivation effect. Results In contrast, the feedback strategy affected factors in the path of the intrinsic motivation effect. Conclusions These results validated the proposed model. Further distinguishing the two motivation effects is necessary to understand the respective effects of an APA's personality and feedback features on learning experiences and outcomes.

Brain Computer Interface (BCI) provides the possibility of bypassing the peripheral nervous system and directly communicating with surrounding devices. The navigation technology using BCI has gone through the process of exploring the prototype paradigm in the virtual environment to accurately completing the locomotion intention of the operator in the form of a powered wheelchair or mobile robot in the real environment. This paper gives a brief overview of BCI navigation applications that have been used in both real and virtual environments in the past 20 years. Horizontal comparison is conducted between various paradigms applied to BCI and their unique signal processing methods. In view of the shift in control mode from synchronous to asynchronous, the development trend of navigation applications in the virtual environment is also reviewed. The contradiction between high-level commands and low-level commands is introduced as the main line to review the two major applications of BCI navigation in the real environment: mobile robot and Unmanned Aerial Vehicles. Finally, toward the popularization of BCI navigation applications to scenarios outside the laboratory, research challenges including human factors in navigation application interaction design and the feasibility of hybrid BCI for BCI navigation are discussed in detail.

Eye tracking technology for mobile devices has made considerable progress. However, due to limited computing capacity and the complexity of context, the traditional image feature-based technology can not extract features accurately and thus impact the performance. This paper proposes a novel approach by fusing appearance-based and feature-based eye tracking methods. Face and eye region detection were conducted to extract features, which were then used as an input to the appearance model to detect the feature points. These feature points were used to generate feature vectors, such as corner center-pupil center, by which the gaze fixation coordinates were calculated. In order to find the feature vectors with the best performance, we conducted a comparison across different vectors under different image resolution and illumination conditions, and the results showed that the average gaze fixation accuracy was achieved with 1.93 degrees of visual angle, when the image resolution was 96×48 pixels, with light sources illuminating from the front of the eye. Compared with the current methods, our method improved the accuracy of gaze fixation and was more usable.

Currently, many simulator systems for medical procedures are under development. These systems can provide new solutions for training, planning, and testing medical practices, improve performance, and optimize the time of the exams. Some premises must be followed and applied to the model under development, such as usability, control, graphics realism, and interactive and dynamic gamification, to make the best of these technologies. This study presents a simulation system of a medical examination procedure in the nasal cavity for training and research, using a patient’s accurate computed tomography (CT) as a reference. The pathologies that are used as a guide for the development of the system are highlighted. Furthermore, an overview of current studies covering bench medical mannequins, 3D printing, animals, hardware, software, and software that use hardware to boost user interaction, is given. Finally, a comparison with similar state-of-the-art works is made. The main result of this work is interactive gamification techniques to propose an experience of simulation of an immersive exam by identifying pathologies present in the nasal cavity such as hypertrophy of turbinates, septal deviation adenoid hypertrophy, nasal polyposis, and tumor.

The problem of visualizing a hierarchical dataset is an important and useful  technical in many real happened situations. Folder system, stock market, and other hierarchical related  dataset can use this technical for better understanding the structure, dynamic variation of the dataset. Traditional space-filling(square) based methods have advantages of compact space usage, node size  showing compared to diagram based methods. While space-filling based methods have two main  research directions—static and dynamic performance. We present a treemapping method based on  balanced partitioning that enables in one variant very good aspect ratios, in another good temporal  coherence for dynamic data and in the third a good compromise between these two aspects. To layout  a treemap, we divide all children of a node into two groups. These groups are further divided until we  reach groups of single elements. Then these groups are combined to form the rectangle representing  the parent node. This process is performed for each layer of a given hierarchical dataset. In one variant  of our partitioning we sort child elements first and built two as equal as possible sized groups from big  and small elements(size-balanced partition), which achieves good aspect ratios for the rectangles, but  less good temporal coherence(dynamic). The second variant takes the sequence of children and creates  the as equal as possible groups with-out sorting(sequence-based, good compromise between aspect  ratio and temporal coherency). The third variant splits the children sets always into two groups of  equal cardinality regardless of their size(number-balanced, worse aspect ratios but good temporal  coherence). We evaluate aspect ratios and dynamic stability of our methods and propose a new metric  that measures the visual difference between rectangles during their movement for representing  temporally changing inputs. We demonstrate that our treemapping via balanced partitioning out  performs state-of-the-art methods for a number of real-world datasets.

Background  Social distancing is an effective way to reduce the spread of the SARS-Covid2 virus. Many students and researchers have already attempted to use computer vision technology to automatically detect human beings in the field of view of a camera and help enforce social distancing. However, because of the present lockdown measures in several countries, the validation of computer vision systems using large-scale datasets is a challenge. Methods  In this paper, a new method is proposed for generating customized datasets and validating deep-learning-based computer vision models using virtual reality (VR) technology. Using VR, we modeled a digital twin (DT) of an existing office space and used it to create a dataset of individuals in different postures, dresses, and locations. To test the proposed solution, we implemented a convolutional neural network (CNN) model for detecting people in a limited-sized dataset of real humans and a simulated dataset of humanoid figures. Results  We detected the number of persons in both the real and synthetic datasets with more than 90% accuracy, and the actual and measured distances were significantly correlated (r=0.99). Finally, we used intermittent-layer- and heatmap-based data visualization techniques to explain the failure modes of a CNN. Conclusions  A new application of DTs is proposed to enhance workplace safety by measuring the social distance between individuals. The use of our proposed pipeline along with a DT of the shared space for visualizing both environmental and human behavior aspects preserves the privacy of individuals and improves the latency of such monitoring systems because only the extracted information is streamed.

Background Electroencephalography (EEG) has gained popularity in various types of biomedical applications as a signal source that can be easily acquired and conveniently analyzed. However, owing to a complex scalp electrical environment, EEG is often polluted by diverse artifacts, with electromyography artifacts being the most difficult to remove. In particular, for ambulatory EEG devices with a restricted number of channels, dealing with muscle artifacts is a challenge. Methods In this study, we propose a simple but effective novel scheme that combines singular spectrum analysis (SSA) and canonical correlation analysis (CCA) algorithms for single-channel problems and then extend it to a fewchannel case by adding additional combining and dividing operations to channels. Results We evaluated our proposed framework on both semi-simulated and real-life data and compared it with some state-of-theart methods. The results demonstrate this novel framework's superior performance in both single-channel and few-channel cases. Conclusions This promising approach, based on its effectiveness and low time cost, is suitable for real-world biomedical signal processing applications.

Background Compared with traditional biomagnetic field detection devices, such as superconducting quantum interference devices (SQUIDs) and atomic magnetometers, only giant magnetoimpedance (GMI) sensors can be applied for unshielded human brain biomagnetic detection, and they have the potential for application in next-generation wearable equipment for brain-computer interfaces (BCIs). Achieving a better GMI sensor without magnetic shielding requires the stimulation of the GMI effect to be maximized and environmental noise interference to be minimized. Moreover, the GMI effect stimulated in an amorphous filament is closely related to its working point, which is sensitive to both the external magnetic field and the drive current of the filament. Methods In this paper, we propose a new noisereducing GMI gradiometer with a dual-loop self-adapting structure. Noise reduction is realized by a direction-flexible differential probe, and the dual-loop structure optimizes and stabilizes the working point by automatically controlling the external magnetic field and drive current. This dual-loop structure is fully program controlled by a micro control unit (MCU), which not only simplifies the traditional constantparameter sensor circuit, saving the time required to adjust the circuit component parameters, but also improves the sensor performance and environmental adaptation. Results In the performance test, within 2 min of self-adaptation, our sensor showed a better sensitivity and signal-to-noise ratio (SNR) than those of the traditional designs and achieved a background noise of 12 pT/√Hz at 10 Hz and 7 pT/√Hz at 200 Hz. Conclusion To the best of our knowledge, our sensor is the first to realize self-adaptation of both the external magnetic field and the drive current.

Background　As a novel approach for people to directly communicate with an external device, the study of brain-computer interfaces (BCIs) has become well-rounded. However, similar to the real-world scenario, where individuals are expected to work in groups, the BCI systems should be able to replicate group attributes. Methods　We proposed a 4-order cumulants feature extraction method (CUM4-CSP) based on the common spatial patterns (CSP) algorithm. Simulation experiments conducted using motion visual evoked potentials (mVEP) EEG data verified the robustness of the proposed algorithm. In addition, to freely choose paradigms, we adopted the mVEP and steady-state visual evoked potential (SSVEP) paradigms and designed a multimodal collaborative BCI system based on the proposed CUM4-CSP algorithm. The feasibility of the proposed multimodal collaborative system framework was demonstrated using a multiplayer game controlling system that simultaneously facilitates the coordination and competitive control of two users on external devices. To verify the robustness of the proposed scheme, we recruited 30 subjects to conduct online game control experiments, and the results were statistically analyzed. Results　The simulation results prove that the proposed CUM4-CSP algorithm has good noise immunity. The online experimental results indicate that the subjects could reliably perform the game confrontation operation with the selected BCI paradigm. Conclusions　The proposed CUM4-CSP algorithm can effectively extract features from EEG data in a noisy environment. Additionally, the proposed scheme may provide a new solution for EEG-based group BCI research.