Usability Assessment through Rethought Eye-Tracking and Spontaneous Verbal Feedback
In a groundbreaking study, researchers have employed an eye-tracking retrospective think-aloud (RTA) method to evaluate the usability of a HIV prevention mobile app among young men aged 15-18 years in New York City, NY; Birmingham, AL; and Chicago, IL.
The innovative approach of eye-tracking RTA enhances user recall and provides detailed insights into users' cognitive processes and interaction patterns. By showing participants a replay of their eye-tracking data during the retrospective think-aloud session, researchers can stimulate more accurate and richer verbalizations about what the user was focusing on and thinking while using the app.
This method combines objective eye-tracking data, which records where and for how long users look at different parts of the app interface, with subjective verbal reports elicited retrospectively while watching their gaze replay. This multimodal data fusion allows better identification of usability issues, such as confusing navigation paths, attention bottlenecks, or missed information, by linking gaze behavior to users’ thoughts and feelings during interactions. It also reduces recall bias because the visual prompt of one's own gaze path helps bring back the context of cognitive processes at specific times, leading to higher validity in usability assessment.
The study demonstrated several benefits of the eye-tracking RTA method in evaluating the HIV prevention app. These include improved recall accuracy of usability problems, enhanced understanding of user attention distribution, the ability to link eye movement patterns with users’ subjective experiences, and providing quantitative data on gaze behavior to complement qualitative think-aloud data.
Two activities had a task completion rate of less than 78%. There were significant differences in time spent, revisits, and total numbers of fixations by participants with/without critical errors. The study identified 19 critical errors on four activities and assessed task performance metrics, critical errors, task completion rate per participant, and task completion rate per task.
The overall mean score of perceived usability rated by the Health IT Usability Evaluation Scale was 4.64 (SD = 0.33). In areas of interest created for the activity with critical usability problems, there were significant differences in time spent, revisits, and total numbers of fixations.
The study demonstrates the utility of an eye-tracking retrospective think-aloud method in consumer health usability evaluation research, offering a valuable tool for understanding critical usability issues in the context of interactions between end-users and the app.
The eye-tracking RTA method, applied in the study, not only enhances the understanding of user attention distribution in health-and-wellness apps like the HIV prevention mobile app, but also aids in linking eye movement patterns with users' subjective experiences, providing valuable insights from the realm of science and technology.
By utilizing this method, researchers can identify crucial usability issues in health-and-wellness apps, contributing significantly to the improvements in these apps' health outcomes and overall user experience.
