Goal-Directed Movements in Menu Selection in Computer-User Interfaces[1]
Hans-Leo TEULINGS2 and Arend W.A. VAN GEMMERT3
2NeuroScript, Tempe AZ,
HLTeulings@NeuroScriptSoftware.com
3Motor Control Laboratory, Department of Kinesiology,
Arizona State University, VanGemmert@asu.edu
Abstract: This study examined goal-directed movements towards menu items in a drop-down menu of 16 items and compared the movements with those in a menu list arranged in a 4 x 4 square. The movements were measured using a pen on a digitizer and a mouse. Results show that total movement duration and secondary submovement duration follow the expected pattern as a function of target precisions and index of difficulty. Pen movements towards menu items in the square layout menu were faster than towards items in the normal dropdown menu. However, mouse movements did not show this advantage. A preliminary explanation is that mouse movements can optimally controlled in vertical movements while pen movements can be controlled in all directions.
1. Introduction
Goal-directed movements are used in human-computer interaction. Computer users produce commands by moving the mouse to a target, which can be a button, hyperlink, or menu item, followed by a mouse click. Menus can have as many as 16 items and sometimes even more. This number of items poses a challenge for the motor control system when fast movements are required. The difficulty of the goal-directed movement towards such a goal can be expressed by the "index of difficulty" (ID=2Log(2*Distance to the target / Width of the target)). Movement time increases with ID (Fitts, 1954). In the far most Menu Item in the drop-down menu when target width kept equal for all items ID = 2log(2*16). It is hypothesized that the increase in movement time is the result of an increase of duration of the secondary submovements. This study examined movements towards menu items in a drop-down menu, and compares the movement characteristics with those in an alternative menu lay-out where the menu items are arranged according to a 4x4 square lay-out. In the latter case, 4 different distances, in addition to 4 different directions, have to be selected by the user. In the far most Menu Item in the square menu, ID = 2log(2*4), which is less and should therefore result in shorter movement times, but at the cost of selecting one of 4 different directions. However, direction selection may not form a limiting factor since the direction categories are relatively coarse, so that it would not overrule with the distance selection process (e.g., Bohan et al., 2003). Alternative menus have been proposed in the past, e.g., the floating menu (Kyota and Teulings, 1999) showing benefits in time and accuracy but may not be as acceptable due to their unfamiliarity.
The production of an accurate goal-directed movement with a pen or mouse to a visible target as fast and precise as possible requires that the visuo-motor system preprograms the initial part of the movement, with subsequent fine adjustments of amplitude and direction based on visual feedback. Successive corrections when the target is approached has been one of the oldest theories to explain the relation between distance, accuracy and duration (Crossman & Goodeve, 1963). If the movement requires a change in direction or increase in amplitude, a small deceleration and acceleration of the pen movement may be measured. The initial ballistic movement towards the vicinity of the target is called the primary submovement. The feedback-controlled adjustment(s) are called secondary submovements. The ballistic submovement is fast but inaccurate. The feedback-controlled adjustments are accurate, but consume time. The optimized submovement model (OSM, Meyer et al., 1988) proposes that the motor system minimizes total movement duration through planning an optimal combination of primary and secondary submovements. This optimization was verified in pen movements (Teulings, in prep.).
2. Experiment
Participants. Eleven adult university students and teachers (7 women, 4 males, and ages 21 to 44 years, all right–hand dominant) participated after informed consent.
Equipment. Stimuli were presented using a Princeton E0900, 48-cm CRT display (visible 45 cm) set at 1600x1200 dots and 72 Hz vertical refresh rate. The display was vertical in front of the participant at 50 cm distance. Writing movements were recorded using a Wacom Intuos2, XD-0608-U display with an active area of 20.32 cm x 15.24 cm. The sampling rate was 102 Hz. The resolution was 0.001 cm and the RMS error <0.1 cm. Movements were also recorded using a normal mouse (Mouse Port Compatible Mouse with a ball). The resolution of the mouse was 0.022 cm meaning that 1 bit due to a mouse movement compared to 0.022 cm on the screen. At the same time, the mouse moved physically 0.0025 cm. Thus an 8-cm target size requires only a 1-cm movement by the mouse. In general, physical movements of the mouse over the table are miniscule compared to the actual movement they produce on the computer screen. The sampling rate of the mouse was estimated at 50 Hz. The sampling rate of the mouse is actually dependent upon the mouse speed. When the mouse is standing still, the sampling rate is 0 and when the mouse moves slowly, the sampling rate is 50 Hz, increasing to a sampling rate of 100 Hz when the mouse speed increases. The sampling rate for the mouse was estimated by arbitrarily requiring that the overall movement duration by the mouse is equal to that of the pen (0.610 s), because no comparative data were known between pen and mouse. Data were collected on a 2.5 GHz Intel4 PC with Windows XP. Recording and analysis of the pen movements were conducted using MovAlyzeR (Teulings, 2003).
Stimuli. The stimulus was presented in real-size on the computer. It consisted of a 2-s precue of the home position of 2 cm wide and 1 cm high. The imperative stimulus consisting of the very home position plus the representation of a menu 0.25 cm below it appeared after a 2-s latency. The menu consisted of 16 items inside a box of 5 cm wide and 8 cm high.
Conditions. There were 2 layouts for the menu.
(a) Drop-down: One row of 16 items, numbered from top to bottom (See Figure 1, left).
(b) Square: 4 rows of 4 items numbered top to bottom and left to right (See Figure 1, right).
At the beginning of each trial one of four targets: 2, 8, 9, and 14, was presented. The subject was instructed to move the pen or the mouse as quickly as possible to the corresponding target in the menu shown. Thus the total number of conditions was 8 (See Table 1). The average vertical size of the movement to these targets was 4.5 cm in the dropdown menu and 4.75 cm in the square layout menu. Due to the particular choices of the Menu Items, the movements required for the square layout menu was slightly greater. The Index of Difficulty (ID) was estimated per condition (See also Figure 2). The sequence of menu types and targets was random. The experiment was conducted once using the pen and digitizer and once using the mouse in random sequence.
Procedure. The participant sat at a desk with the digitizer on top and the computer monitor display in front of them. They were instructed to place the pen on the digitizer, or the mouse with the left key clicked, in the middle of the home position as soon as it was visible as a precue. Above the home position, one of the four Item Numbers: 2, 8, 9, 14 was shown. Upon presentation of the imperative stimulus, they were required to draw a line to the Menu Item indicated. Recording started immediately upon appearance of the imperative stimulus. Each condition was repeated 8 times. Trials were in random order. The participant initially performed the experiment as practice run, during which the experimenter could instruct the proper execution of each condition. The movement had to be performed as rapidly and accurately as possible. Each trial was processed immediately after its recording so the experimenter could monitor whether the trials were consistent. When the subject produced a sufficient number of consistent trials, the training series was aborted and the recording of the experimental data started. One session took 15 minutes.
Analysis. Writing patterns were lowpass filtered at 10 Hz (transition band 5 Hz till 16 Hz), differentiated, and velocity and acceleration curves were estimated. Only the vertical component of the movement was analyzed. Stroke segmentation was at points where the vertical velocity passed through zero. Subsequent zero crossings within 0.04 seconds were discarded. A stroke was further segmented into a primary and a secondary submovement by the first negative-to-positive zero crossing of the vertical acceleration after the absolute peak velocity (since the movement is downward signs need to be reversed in this statement). Per submovement and for the entire stroke, duration and vertical size were estimated. The duration of the primary submovement was defined as T1/T and S1/S, respectively, where T1 and S1 are the duration and size of the primary submovement and S and T the size and duration of the entire stroke, respectively.
Trials were automatically discarded if they were not consistent, e.g., if the amplitude and direction of the stroke were not within boundaries. For example, in the drop-down menu, the movement to Menu Item 9 had to be 5+-0.5cm long with a direction of 270+-22.5degrees. In total 10% of the 704 trials were discarded in the pen and digitizer recordings and 15% of the704 trials in the mouse recordings. The higher error rate is attributed to the lesser accuracy of the mouse for movement recording, resulting in noise in addition to motor errors.
3. Results
Figure 3 shows examples of a goal directed movement to Item 9 in the normal dropdown menu with the vertical velocity pattern segmented into primary and secondary submovements, recorded using a pen and digitizer. Figure 4 (left) shows total stroke duration and the durations of primary and secondary submovements. The goal-directed movements in the dropdown menu are on average 0.680 s, and in the square layout menu 0.528 s (t(10)=5.57, p<0.001). It is noticeable that the pattern of total stroke durations follows the pattern of the indices of difficulty (ID) (Figure 2). Post-hoc analysis shows that the duration of the total movement was shorter in the square layout menu than in the drop down menu in Menu Items 2, 8, and 14 (t(10)<2.76, p<0.05) while there was no difference in Menu Item 9. This corroborates with the fact that the difference in ID between drop-down and square menu is relatively small for this menu item. Similarly to the total movement duration, the duration of the secondary submovement is smaller in the square layout menu than in the dropdown menu (t(10)=5.12, p<0.001). Remarkable is that primary submovement duration is largely independent upon the size of the movement
When inspecting size of the total movement and of the primary
and secondary submovements, we find, that the average sizes of the total movements
in both menu types was not different (t(10)=1.37, p>0.2). In agreement
with the expectations, the primary submovement was longer, and thus the secondary
submovement was shorter in the square layout menu than in the dropdown menu
(t(10)<2.78, p<0.05).
 |
Figure 5 shows the corresponding results when using the mouse instead of the pen and digitizer for the goal-directed movements. No differences were found between both menu types for the durations (t(10)<1.87, p>0.05) or sizes (t(10)<1, p>0.2) for the total movement or the primary or secondary submovements.
When comparing, within pen or mouse movements and within each menu type a striking similarity can be observed between the pattern of indices of difficulty for the various distances and the total movement durations.
4. Discussion
Fast and accurate goal-directed movements have been recorded towards menu items in two different menu layouts, a typical dropdown menu and an alternative menu layout where the menu items are organized in a square. Movements were performed using a pen on a digitizer or using a mouse. Total movement duration by pen and mouse follows the pattern of indices of difficulty of each movement condition. While the pen movements show the expected benefits in the square layout menu because of the lesser amplitude precision required, the mouse movements did not show any difference between both menu layouts. A preliminary explanation may be that the mouse movements are more geared towards vertical displacements without direction choices as commonly used in human computer interfaces while pen movements are more geared towards movements in all directions.
5. References
Bohan M, Longstaff MG, Van Gemmert AW, Rand MK, Stelmach GE. (2003). Effects of target height and width on 2D pointing movement duration and kinematics. Motor Control, 7, 278-289.
Crossman, E.R.F.W., & Goodeve, P.J., (1963). Feedback control of hand movement and Fitts' law. Proceedings of the Experimental Society, Oxford.
Fitts, P.M., (1954). The information capacity of the human motor system in controlling amplitude of movement. Journal of Experimental Psychology, 47, 381-391.
Kiyota, K., Teulings, H.L. (1999). A new menu system based on human motor control knowledge. Proceedings of the 9th Biennial Conference of the International Graphonomics Society (pp. 259-261). Singapore: Nanyang Technological University.
Meyer, D.E., Abrams, R.A., Kornblum, S., Wright, C.E., & Smith, J.E.K. (1988). Optimality in human motor performance: Ideal control of rapid aimed movements. Psychological Review, 95, 340-370.
Teulings, H.L. (in prep.). Optimization of Movement Duration in Handwriting Strokes in Different Directions in Young, Elderly, and Parkinsonian Subjects.
Teulings, H.L. (2003). MovAlyzeR2.6 [Computer program]. http://www.neuroscriptsoftware.com.