# Trajectory prediction model for crossing-based target selection

1. Beijing Key Lab of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

2. State Key Lab of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Abstract

Keywords： Target selection ; Crossing-based selection ; Trajectory prediction

Content

^{[1]}. An ideal model for the entire target-selection motion process may help computer designers make predictions about interaction results during the process of target selection rather than at the end of the process.

^{[2]}. Due to the elongated-rectangle target, users prefer to cross the target on its longer side, with the main impact on trajectory being the bending of the path

^{[3]}. Control theory is commonly used to model trajectories for target selection tasks. However, the existing trajectory prediction model is not able to model the trajectory for crossing-based selection well due to the lack of consideration of the shape of the target. Moreover, control theory and dynamic models are usually used to simulate a reaching movement. Both models quantify the location and velocity at each instant according to the control signal being dynamically adjusted by the position and velocity feedback.

**2.1**

**Crossing-based target selection**

^{[4]}is the most robust and successful model for human motion behavior, as it accurately predicts the time to complete target acquisition. An early crossing-based selection experiment referring to a “goal passing task” was performed by Accot and Zhai, which laid the foundation for the Steering Law

^{[5]}. Apitz et al.

^{[6]}specified the six task conditions for indirect stylus input. Luo and Vogel

^{[7]}found generalizable and empirical support for the application of crossing-based selection to touch input through analysis of the six task conditions.

^{[8]}found that the average interaction time for crossing-based target selection is 16% faster than that for indirect stylus input. Taking into consideration the influence of the target’s shape, Dixon et al.

^{[9]}tested crossing target density and orientation. Using a direct stylus input device, they found that crossing-based target selection is faster than pointing for dialog boxes while also remaining spatially efficient. Cockburn et al.

^{[10]}and Buxton et al.

^{[11]}tested direct stylus input and indirect stylus input respectively, especially the friction force between finger and surface while dragging, and indicated that the input mode appeared to affect the performance of target selection. However, most basic research has focused on the duration and the endpoint of the task while no attention has been paid to the trajectory.

**2.2**

**Trajectory prediction model**

^{[12]}. In addition to predicting average behavior, the optimal control model can also simulate the feedback of unexpected changes in the real environment. This kind of model can therefore reflect the uncertainty, delay and unstable fluctuation during the process of selecting targets and adjust according to feedback.

^{[13]}built a target-selection motion model based on a linear-quadratic-Gaussian optimal feedback control (OFC) mechanism. The OFC model can simulate a static or moving circular target, while there is no thought for modeling the movement time. Quinn and Zhai

^{[14]}developed a production model which can predict users’ timing performance while typing using word-gesture keyboards. However, due to the specific application, other factors such as semantic information would be helpful to the model, and it may be not suitable for general target selection tasks. In crossing-based target selection, the specifics of the target’s shape results in a more curved path and more challenges to trajectory modeling, which is difficult for the existing model to simulate.

**2.3**

**Dynamic models**

^{[15]}used a dynamic model to simulate the reaching movement of octopus arms. Tahara et al.

^{[16]}constructed a musculoskeletal redundant arm model to simulate the reaching movement of human arms. However, target selection is a much more microscopic movement, where the environment and users’ slight psychological fluctuations may lead to large changes of trajectory. Oulasvirta et al.

^{[17]}attempted to use neuromechanics to model the process in which users press a button, which also provided a solution for more elaborate interactions. One case of dynamic models that had been applied in trajectory prediction is known as social force model (SFM). Helbing and Molnar first introduced the term of “social forces”

^{[19]}and presented how they used them to simulate the motion of pedestrians by measuring the internal motivations of the individuals to perform certain movements

^{[18,21]}.

^{[18]}. The main forces that affect the motion of particle

*p*are as follows:

**3.1**

**Desired force**

^{[19]}, we introduce the concept of desired direction and desired velocity. The particle

*p*of mass ${m}_{p}$ tends to move with a desired speed ${v}_{p}^{0}$ towards the moving target with a changeable direction vector $\text{}\overrightarrow{{e}_{p\alpha}(t)}$ , and therefore the particle is likely to correspondingly alter its actual velocity $v\left(t\right)$ with a relaxation time $\text{}{\tau}_{p}$ . The desired force of the particle can be described by an acceleration term of the form.

**3.2**

**Inertial losses**

^{[20]}. For this reason, we introduce a linear inertial loss function to simulate the phenomenon of inertial losses. It is given by:

**3.3**

**Boundary force**

*A, B*are system parameters.

**3.4**

**Interactive force**

^{[21]}, we use angle to control both the bending of the trajectory and the insertion angle as influenced by psychological factors.

^{[7]}.

**4.1**

**Tasks**

^{[7]}. Participants used a mouse to cross a moving target with one of two different orientations (Ori), one of eight directions of movement (Dir), width of 96 pixels and a velocity of 192 pixels per second. The initial distance between target and start point was 960 pixels. The crossing-based target selection tasks are illustrated in Figure 1.

**4.2**

**Participants and apparatus**

**4.3**

**Optimization parameters**

*A*,

*B*,

*C*,

*D*,

*E*,

*F*,

*G*, $H,\text{}I,\text{}{w}_{ini},\text{}{w}_{end},\text{}{\theta}_{v},\text{}{\theta}_{i}]$ is selected according to the model defined in Section 3, which may signifi-cantly affect the similarity between the real and simulated data. We defined the sum total Euler distance per frame as the cost function in order to estimate the similarity value. We developed a genetic algorithm to obtain the most appropriate parameter set. With the population size set at 50, the optimization parameter set is shown in Table 1.

Parameter | Values |
---|---|

A | 1847.5 |

B | 1157.4 |

C | $\left\{\begin{array}{c}-2099.5,\text{}<{\displaystyle \overrightarrow{{e}_{p}(t)}},\text{}{\displaystyle \overrightarrow{{e}_{\alpha}(t)}}>\le {\theta}_{v}\\ 1916.1,\text{}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right. $ |

D | $\left\{\begin{array}{c}-0.4793,\text{}<{\displaystyle \overrightarrow{{e}_{p}(t)}},\text{}{\displaystyle \overrightarrow{{e}_{\alpha}(t)}}>\le {\theta}_{v}\\ 2.4787,\text{}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right. $ |

E | $\left\{\begin{array}{c}4.9962,\text{}<{\displaystyle \overrightarrow{{e}_{p}(t)}},\text{}{\displaystyle \overrightarrow{{e}_{\alpha}(t)}}>\le {\theta}_{v}\\ 4.8709,\text{}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right. $ |

F | 1220.3 |

G | $\left\{\begin{array}{c}-1282.4,\text{}<{\displaystyle \overrightarrow{{e}_{p\alpha}(t)}},\text{}{\displaystyle \overrightarrow{{n}_{\alpha ori}}}>\le {\theta}_{i}\\ 1231.8,\text{}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right. $ |

H | $\left\{\begin{array}{c}2.1322,\text{}<{\displaystyle \overrightarrow{{e}_{p\alpha}(t)}},\text{}{\displaystyle \overrightarrow{{n}_{\alpha ori}}}>\le {\theta}_{i}\\ 9.8106,\text{}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right. $ |

I | $\left\{\begin{array}{c}0.4991,\text{}<{\displaystyle \overrightarrow{{e}_{p\alpha}(t)}},\text{}{\displaystyle \overrightarrow{{n}_{\alpha ori}}}>\le {\theta}_{i}\\ -1.5923,\text{}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right. $ |

${w}_{ini}$ | 0.9 |

${w}_{end}$ | 0.7 |

${\theta}_{v}$ | 0.1948 |

${\theta}_{i}$ | -0.1907 |

$Ori(\xb0)$ | $Dir(\xb0)$ | $({p}_{x}\left({t}_{0}\right),\text{}{p}_{y}\left({t}_{0}\right))$ | $({\dot{p}}_{x}\left({t}_{0}\right),\text{}{\dot{p}}_{y}\left({t}_{0}\right))$ | ${\tau}_{0}$ | ${v}_{p}^{0}$ |
---|---|---|---|---|---|

0 | -135 | (841.8, -380.5) | (162.7, 522.1) | 0.045 | 1558.4 |

0 | -90 | (881.7, -335.2) | (167.1, 526.7) | 0.029 | 1542.6 |

0 | -45 | (873.2, -634.5) | (164.3, 516.9) | 0.039 | 1785.6 |

0 | 0 | (853.2, -525.0) | (162.6, 522.7) | 0.020 | 1734.6 |

0 | 45 | (897.9, -403.3) | (166.4, 530.2) | 0.069 | 1734.3 |

0 | 90 | (869.0, -627.1) | (165.3, 518.8) | 0.034 | 1712.3 |

0 | 135 | (929.0, -434.1) | (166.5, 520.0) | 0.015 | 1580.3 |

0 | 180 | (830.1, -305.7) | (164.2, 526.1) | 0.023 | 1508.0 |

90 | -135 | (1333.0, 42.8) | (188.7, 543.4) | 0.027 | 1540.6 |

90 | -90 | (1311.6, 48.3) | (189.5, 545.1) | 0.025 | 1535.1 |

90 | -45 | (1350.2, 59.5) | (184.0, 541.5) | 0.040 | 1765.5 |

90 | 0 | (1369.9, 54.0) | (190.0, 545.8) | 0.020 | 1808.6 |

90 | 45 | (1292.5, 38.6) | (187.0, 544.7) | 0.026 | 1704.9 |

90 | 90 | (1401.8, 19.0) | (191.6, 542.0) | 0.027 | 1669.5 |

90 | 135 | (1322.2, 82.8) | (187.5, 546.5) | 0.028 | 1534.6 |

90 | 180 | (1285.3, 7.8) | (186.9, 545.2) | 0.051 | 1521.4 |

**5.1**

**Comparison of trajectories**

**5.2**

**Results of endpoints**

$Ori(\xb0)$ | $Dir(\xb0)$ | $Offset\text{}of\text{}Endpoints(\mathrm{m}\mathrm{m})$ |
---|---|---|

0 | -135 | 4.13 |

0 | -90 | 3.52 |

0 | -45 | 2.18 |

0 | 0 | 3.30 |

0 | 45 | 1.94 |

0 | 90 | 3.03 |

0 | 135 | 2.95 |

0 | 180 | 3.65 |

90 | -135 | 2.30 |

90 | -90 | 1.41 |

90 | -45 | 1.98 |

90 | 0 | 3.10 |

90 | 45 | 1.23 |

90 | 90 | 1.75 |

90 | 135 | 2.84 |

90 | 180 | 4.31 |

**5.3**

**Results of time fitting**

**6.1**

**Curved paths**

**6.2**

**Influence of orientation**

**6.3**

**Dynamic model for target selection**

**6.4**

**Model evaluation**

**6.5**

**Design in interfaces**

Reference

St Amant R, Riedl M O. A perception/action substrate for cognitive modeling in HCI. International Journal of Human-Computer Studies, 2001, 55(1): 15–39 DOI:10.1006/ijhc.2001.0470

Accot J. Les tâches trajectorielles en interaction homme-machine: cas des tâches de navigation. Toulouse 1, 2001

Apitz G, Guimbretière F, Zhai S M. Foundations for designing and evaluating user interfaces based on the crossing paradigm. ACM Transactions on Computer-Human Interaction, 2010, 17(2): 1–42 DOI:10.1145/1746259.1746263

Fitts P M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology: General, 1992, 121(3): 262–269 DOI:10.1037//0096-3445.121.3.262

Accot J, Zhai S. Beyond Fitts' law: models for trajectory-based HCI tasks. In: Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. ACM, 1997, 295–302

Apitz G, Guimbretière F, Zhai S M. Foundations for designing and evaluating user interfaces based on the crossing paradigm. ACM Transactions on Computer-Human Interaction, 2010, 17(2): 1–42 DOI:10.1145/1746259.1746263

Luo Y, Vogel D. Crossing-based selection with direct touch input. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2014, 2627–2636 DOI:10.1145/2556288.2557397

Forlines C, Balakrishnan R. Evaluating tactile feedback and direct vs. indirect stylus input in pointing and crossing selection tasks. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Florence, Italy, ACM, 2008, 1563–1572 DOI:10.1145/1357054.1357299

Dixon M, Guimbretière F, Chen N. Optimal parameters for efficient crossing-based dialog boxes. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Florence, Italy, ACM, 2008, 1623–1632 DOI:10.1145/1357054.1357307

Cockburn A, Ahlström D, Gutwin C. Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse. International Journal of Human-Computer Studies, 2012, 70(3): 218–233 DOI:10.1016/j.ijhcs.2011.11.002

Buxton W, Hill R, Rowley P. Issues and techniques in touch-sensitive tablet input. ACM SIGGRAPH Computer Graphics, 1985, 19(3): 215–224 DOI:10.1145/325165.325239

Todorov E. Optimality principles in sensorimotor control. Nature Neuroscience, 2004, 7(9): 907–915 DOI:10.1038/nn1309

Huang J, Peng X L, Tian F, Wang H, Dai G Z. Modeling a target-selection motion by leveraging an optimal feedback control mechanism. Science China Information Sciences, 2018, 61(4): 044101 DOI:10.1007/s11432-017-9326-8

Quinn P, Zhai S M. Modeling gesture-typing movements. Human–Computer Interaction, 2018, 33(3): 234–280 DOI:10.1080/07370024.2016.1215922

Yekutieli Y, Sagiv-Zohar R, Aharonov R, Engel Y, Hochner B, Flash T. Dynamic model of the octopus arm. I. biomechanics of the octopus reaching movement. Journal of Neurophysiology, 2005, 94(2): 1443–1458 DOI:10.1152/jn.00684.2004

Tahara K, Arimoto S, Sekimoto M, Luo Z W. On control of reaching movements for musculo-skeletal redundant arm model. Applied Bionics and Biomechanics, 2009, 6(1): 11–26 DOI:10.1080/11762320902789848

Oulasvirta A, Kim S, Lee B. Neuromechanics of a Button Press. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Montreal QC, Canada, ACM, 2018, 1–13 DOI:10.1145/3173574.3174082

Helbing D, Molnár P. Social force model for pedestrian dynamics. Physical Review E, 1995, 51(5): 4282 DOI:10.1103/physreve.51.4282

Helbing D, Farkas I, Vicsek T. Simulating dynamical features of escape panic. Nature, 2000, 407(6803): 487–490 DOI:10.1038/35035023

Hajri A A, Fels S, Miller G, Ilich M. Moving Target Selection in 2D Graphical User Interfaces. In: Human-Computer Interaction–INTERACT 2011. Berlin, Heidelberg, Springer Berlin Heidelberg, 2011, 141–161 DOI:10.1007/978-3-642-23771-3_12

Helbing D, Johansson A. Pedestrian, crowd and evacuation dynamics. Encyclopedia of Complexity and Systems Science, 2010, 16: 697–716