Medical AI Decision Support Systems' Usability Assessment via Expert's Observation: An Early-stage Investigation
A recent study has highlighted the potential benefits of incorporating gaze behavior analysis in the design of medical AI support systems for image inspection. By tracking how radiologists visually explore medical images, AI models can be guided to focus on clinically relevant image regions, enhancing both the accuracy of diagnoses and the usability of the systems.
The study, which focused on assessing experts' interaction with an AI software for caries detection in bitewing x-ray images, found that when the AI was used, experts spent approximately 25% of the total inspection task with the AI support enabled. However, it was noted that increased time was needed for the overall assessment due to interruptions in attention caused by the AI's usability.
The study, which compares standard viewing of bitewing images without AI support to viewing where AI support can be toggled on and off, aims to gain insight into designing medical AI support systems with the user in mind. Eye tracking metrics were used to evaluate the usability and user interaction with these systems.
One of the key findings of the study was the improvement in classification and explanation of medical image findings. A two-stage multimodal learning framework integrating radiologist eye-tracking signals with image features improved classification metrics (F1 score increased by 5.7%, AUC by 3.4%) and also enhanced the quality and interpretability of generated radiology reports by aligning generated content with anatomically relevant gaze patterns.
This improvement in user interaction and usability occurs because gaze data reflects expert decision-making processes, enabling AI systems to align more closely with human visual strategies. As a result, AI systems behave in a way that is more transparent and explainable to users, supporting trust and better human–AI collaboration.
Furthermore, gaze analysis can reveal cognitive and attentional patterns, such as fixation duration and scanning sequences, helping identify how users interact with images and where they focus their attention. This information can be used to optimize user interfaces and workflows, making AI support systems more intuitive and reducing diagnostic errors.
The study did not discuss any potential long-term effects of using AI support in clinical workflows. It also did not specify the exact number of experts participating in the assessment, nor did it mention any significant improvements in caries detection accuracy when using the AI support.
Despite these limitations, the study underscores the potential of gaze behavior analysis in improving medical AI support systems for image inspection. As medical AI increasingly becomes a part of clinical workflows, with collaborative systems being the preferred choice, the need for user-friendly and transparent AI systems is paramount. The findings of this study provide a valuable step towards achieving this goal.
[1] Reference for the two-stage multimodal learning framework [2] Reference for the cognitive and attentional patterns [3] Reference for the potential long-term effects (if any are available) [4] Reference for the reduction in diagnostic errors (if any are available)
- The integration of artificial intelligence and eye tracking technology in medical-conditions diagnosis can lead to improved health-and-wellness by enhancing the accuracy of diagnoses and the usability of systems.
- The study of gaze behavior analysis in the design of AI support systems for image inspection reveals cognitive and attentional patterns, such as fixation duration and scanning sequences, which can optimize user interfaces and workflows, reducing diagnostic errors.
- scientific research on medical-conditions diagnosis has shown that the use of gaze behavior analysis in AI support systems can improve classification metrics, making them more transparent and explainable, ultimately supporting trust and better human–AI collaboration.
[1] [Reference for the two-stage multimodal learning framework][2] [Reference for the cognitive and attentional patterns][3] [Reference for the potential long-term effects (if any are available)][4] [Reference for the reduction in diagnostic errors (if any are available)]