Eye-Tracking in Software Usability Evaluation

By Andrea Pagotto

Introduction

Usability is a very important factor to consider in software development, as it is often key to the success for interactive applications. One emerging method of assessing usability of a software interface is called eye tracking. Eye tracking can capture data from the user’s eye direction while the user is using the interface, and this data can be processed in many different ways to come up with quantitative measures to characterize the usability issues of the user interface. This blog post will discuss how eye tracking works, how it is being used in recent software usability studies, and what’s next for this technique.

How does it work?

Psychological Basis:  The psychological basis behind eye tracking is the concept that the eyes are the “window to the soul”, and gaze can give hints towards a user’s thoughts and intentions [12]. According to the “Eye-mind” theory, eye movement and direction can provide dynamic information about where a user’s attention is being focused.

Detection: Images can be used to detect where a person is looking, or “point of regard”, most commonly by detecting the corneal reflections [12],  or a video of the face. This method requires good quality cameras with good resolution and high capture rates, which can be quite expensive [7].

Primary measurements: There are two primary measurements, called “fixations” and “saccades”, which are drawn from the direct eye-tracking output data. Fixations are measured by the frequency and amount of time spent looking at a target. Saccades are rapid movements between fixation points [14].

Data Visualization: The main techniques used are: heat maps, gaze plots and areas of interest (AOIs).

·     Heat Maps: Heat maps, shown in figure 1, are a visual representation of which areas were the most fixated on. This can help to interpret the results, and also to present results to show that a design change is affecting where the user’s attention is [8, 14]

·    Gaze Plots: Gaze plots or scan paths, shown in figure 2,  are derived from saccade-fixation-saccade sequences. These types of plots are used to identify which areas of the screen the user’s attention is drawn to, as well as the order they were looked at [8, 14].

·    Areas of Interest: This data interpretation method, shown in figure 3, allows the researcher to define block sections on the page, and evaluate the amount of time the user spent looking at each block.

Figure 1 - Heat Map [8]

Figure 2 - Gaze Plot [8]

Figure 3 - Areas of Interest [8]

How is eye tracking being used in recent usability evaluations?

Several applications where eye tracking can provide value have been identified from recent studies:

1)    A method to enhance other usability analysis techniques

2)    A quantitative way of comparing layouts

3)    To assess usability with children users

4)    To assess novel interfaces

 

1. Eye Tracking to Enhance Other Evaluation Methods

Eye tracking is often used to enhance the results provided from other methods of usability evaluation. Some commonly used techniques that are often used in combination with eye tracking are mouse-clicking, think aloud testing, and surveys. In the literature search of recent studies, all of these methods have been enhanced by the use of eye tracking to provide further clarification and quantification of the results. The main observations that can be drawn from these studies are that eye-tracking can enhance studies by providing quantitative data to back up verbal or written feedback, as well as provide additional insights into what the user is thinking in addition to mouse click data.

With Surveys: A study that shows the value of eye tracking with surveys is seen in a study comparing two recommender schemes for e-commerce [1]. The results were first analyzed based on eye-tracking data including AOIs, heat plots and gaze plots to determine which recommender schemes were drawing the user’s attention the most. This was followed by surveys to assess the user’s opinions of the usefulness and ease of use of each option. In this case, the most useful choice also coincided with the option that was drawing the user’s attention the most. Therefore eye tracking was able to provide data to quantify the usability of one option over the other, to back up the survey data.

With Think Aloud: Eye tracking with Think Aloud were used to in a study to assess the ease of use of a Volunteered Geographic Information system [5]. One of the main aspects important to this system is learnability, because it is important to obtain new users that they will be able to easily understand the system. Eye tracking was effective in this assessment, because it was able to quantify how long and where the users were looking, to quantitatively corroborate their verbal feedback.

With Mouse Tracking: In a recent study on geographic mapping software, eye tracking was used to draw comparisons on the ease of use of the user interface. The aspect being compared was the mechanism of zooming in on the map [9]. Eye tracking was used to generate eye fixation counts while operating the different zoom techniques, to quantify the amount of effort required by the user for each type of zoom. This provided a quantitative comparison of these techniques that enabled the researchers to draw conclusions about which zooming technique is the best in terms of usability.

 

2. Eye Tracking in Layout Comparisons

Eye tracking has been effectively applied to enable direct comparison in user interface layouts, and compare the effect of specific layout differences on usability. This enables the researchers to provide quantitative values for each layout to draw a statistically significant conclusion about which is better.

Comparing Efficiency: Eye tracking was applied to assess the efficiency of a software store [3]. The primary metric of efficiency is time required to complete a task, however this was enhanced with eye-tracking data by also measuring fixation time and fixation duration for specific areas. Associating time with a particular region of interest allows researchers to better assess the usability issues.

Comparing Cognitive Load: Eye tracking was used for comparing different three different layouts of agricultural web pages [18]. A hot spot distribution was used to compare the websites, which enabled the researchers to draw conclusions about the user’s cognitive load associated with each interface.

3. Benefits with Children Users

Historically, one of the main uses of eye tracking is to assess situations where users many not be able to communicate their thoughts effectively through verbal or written interactions, such in studies with children. In recent studies involving eye tracking, children were a predominant user group being assessed, and eye tracking was able to effectively provide insight into the usability of the interfaces. In assessing a mobile application for preschoolers [10], a scan path plot was analyzed, which showed that the children had a low learning curve and good intuition towards how to use this interface. In another study, eye tracking was applied to assess the motivation of young children in completing tasks using a software user interface [13]. Eye tracking was effectively applied in this study, in combination with electroencephalography to analyze the children’s mental state, to perform calculations of Fitts’ law for each user and determine measures of “motivation”, which was related back to design aspects of the user interface to improve it. Eye tracking was also employed to assess a tablet game [6]. The results from this study indicated that the children were often missing relevant information on the interface.

4. User Interfaces on Different Devices

In recent studies, eye tracking has been employed to assess the usability of interfaces on several different types of platforms. Recently assessed interfaces have been smart watches [17], an interactive white board [15], and a smart TV [16]. In novel platforms with new user interface designs, eye tracking can be of use providing an overview of how the user interface is drawing the user’s attention. These were all preliminary studies to gauge the success of new types of user interface designs in these developing technologies.

Smart TV: A user interface assessment was done based on mock-ups of the design to determine preliminary values for effectiveness and efficiency  [16]. This procedure was incorporated into the iterative design of the user interface.

Smart Watch: A comprehensive study was done to formulate an objective assessment during the development of the interface [17]. The watch was evaluated based on numerous different metrics and eye-tracking data enabled researchers to draw many conclusions about different aspects of the user interface. For example, from a task-based analysis, eye tracking showed how users are searching for icons on the interface using a hot spot plot.

Interactive Whiteboard: Eye tracking was used to provide a quantitative evaluation in combination with qualitative evaluation based on the user’s behaviours [15]. The results from this preliminary assessment were able to show that the display is easy to interact with for the children, and that it stimulates their curiosity and draws their attention as desired.

What’s Next for Eye-Tracking?

Several studies were identified that have made contributions towards analyzing and developing enhancements to the eye-tracking methodology itself, including improving the analysis techniques, as well as expanding towards adding additional analysis techniques to detect the user’s mental state.

Recent advances include using better quality cameras and smaller cameras to enable assessing different types of interfaces such as smart watches, as well as progress on the algorithms side. One study emphasized that to get a meaningful assessment, it is necessary to use many different metrics from the eye tracking data [4]. They recommend incorporating metrics such as heat maps to verify whether points were visited and scan path to determine the amount of time spent on information gathering. Another study has been working on improving the heat map technique primarily [2]. They have been attempting to improve the heat map to provide a better assessment of the cognitive load associated with an interface. The study was able to successfully develop a method to incorporate cognitive states of the user into the heat mapping to provide a useful tool to contextualize results.

A recent study has proposed a new methodology as full biofeedback system for evaluating interfaces, by incorporating electrical signals in the brain (EEG, ECG) at the same time as the eye tracking. This way it is possible to quantify the user’s mental state while looking at a certain location on the screen [11], which can relate back to cognitive load, or the user’s level of satisfaction or frustration. As this technology continues to develop, this could enable researchers to obtain a full picture of the user’s mental state during usability evaluations.

Summary

Eye tracking appears to be a promising new development of technology in software usability assessment. Some predominant areas where eye tracking is effectively applied includes enhancing the assessment of user interfaces for children, providing quantitative data for comparing layouts, enhancing more subjective analysis methods with quantitative measures, and for use in preliminary assessments of new types of user interfaces, such as smart watches. The continuous advancements in technology and software will enable increasingly accurate results, and increasingly effective interpretations of the data.

References

[1]           Chen, L. and Pu, P. 2012. Experiments on user experiences with recommender interfaces. Behaviour & Information Technology. 33, January (2012), 1–23.

[2]           Gebhardt, C., Hepp, B., Nägeli, T., Stevšić, S. and Hilliges, O. 2016. Airways: Optimization-Based Planning of Quadrotor Trajectories According to High-Level User Goals. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. (2016), 2508–2519.

[3]           Goh, K.N., Chen, Y.Y., Lai, F.W., Daud, S.C., Sivaji, A. and Soo, S.T. 2013. A comparison of usability testing methods for an e-commerce website: A case study on a Malaysia online gift shop. Proceedings of the 2013 10th International Conference on Information Technology: New Generations, ITNG 2013. 1, (2013), 143–150.

[4]           Goldberg, J.H. and Wichansky, A.M. 2003. Eye Tracking in Usability Evaluation. A Practitioner’s Guide. The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research. August (2003), 493–516.

[5]           Jones, C.E. and Weber, P. 2012. Towards Usability Engineering for Online Editors of Volunteered Geographic Information: A Perspective on Learnability. Transactions in GIS. 16, 4 (2012), 523–544.

[6]           Kiili, K., Koivisto, A., Finn, E. and Ketamo, H. 2012. Measuring User Experience in Tablet Based Educational Game. Proceedings of the European Conference on Games Based Learning. (2012), 242.

[7]           Majaranta, P. and Bulling, A. 2014. Advances in Physiological Computing. Advances in Physiological Computing. (2014), 17–39.

[8]           Manhartsberger, M. and Zellhofer, N. 2005. Eye tracking in usability research: What users really see. Usability Symposium. 198, (2005), 141–152.

[9]           Manson, S.M., Kne, L., Dyke, K.R., Shannon, J. and Eria, S. 2012. Using Eye-tracking and Mouse Metrics to Test Usability of Web Mapping Navigation. Cartography and Geographic Information Science. 39, April 2015 (2012), 48–60.

[10]        Masood, M. and Thigambaram, M. 2015. The Usability of Mobile Applications for Pre-schoolers. Procedia - Social and Behavioral Sciences. 197, February (2015), 1818–1826.

[11]        Medgyesi, Z., Pomazi, K., Szegletes, L. and Forstner, B. 2016. Evaluating application usability with portable biofeedback system for mobile and desktop. 6th IEEE Conference on Cognitive Infocommunications, CogInfoCom 2015 - Proceedings. (2016), 279–283.

[12]        Poole, A. and Ball, L.J. 2005. Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects. Encyclopedia of Human-Computer Interaction. (2005), 211–219.

[13]        Radvile, E., Cenys, A. and Ramanauskaite, S. 2015. Electroencephalography and eye gaze movement signals’usage for estimation of user interface usability. Elektronika ir Elektrotechnika. 21, 5 (2015), 75–80.

[14]        Sharma, C. and Dubey, S.K. 2014. Analysis of eye tracking techniques in usability and HCI perspective. 2014 International Conference on Computing for Sustainable Global Development, INDIACom 2014. (2014), 607–612.

[15]        Standoli, C.E., Costa, F., Mazzola, A., Muschiato, S. and Zambarbieri, D. 2015. User experience of digital dioramas for interactive whiteboard. Proceedings of the International Conferences on Interfaces and Human Computer Interaction 2015, IHCI 2015, Game and Entertainment Technologies 2015, GET 2015 and Computer Graphics, Visualization, Computer Vision and Image Processing 2015, CGVCVIP 2015 - P. January (2015).

[16]        Wang, S. 2015. Integrating Service Design and Eye Tracking Insight for Designing Smart TV User Interfaces. International Journal of Advanced Computer Science and Applications. 6, 7 (2015), 163–171.

[17]        Wu, Y., Cheng, J. and Kang, X. 2015. Design, User Experience, and Usability: Design Discourse. 9186, 130 (2015), 98–109.

[18]        Xiangzhen, H., Fucheng, W. and Zhangyuan, X. 2016. Study on Usability of Agricultural Product Web Page Layout Based on Eye Tracker. 2015 (2016), 78–82.