Medical-Expert Eye Movement Augmented Vision Transformers for Glaucoma Diagnosis

Shubham Kaushal, Roshan Kenia, Saanvi Aima, Kaveri A. Thakoor

Keywords: attention, computer-aided diagnosis, eyetracking, glaucoma, interpretability, optical coherence tomography, vision transformer

TL;DR: We infuse medical-expert eye tracking into vision transformers to make them more accurate, efficient, and interpretable for expediting eye disease detection.

Abstract: Glaucoma lacks a definitive gold standard for clinical diagnosis, motivating the role of artificial intelligence (AI) in expediting glaucoma detection and enabling consensus. The Vision Transformer (ViT) model is a promising solution for this problem as it uses the self-attention mechanism to improve performance and interpretability. Furthermore, eye-tracking data provides valuable information about a clinician’s decision-making process during the diagnosis of glaucoma using Optical Coherence Tomography (OCT) reports. In this study, two approaches were originated for incorporating eye-tracking data into the ViT’s training process, using solely eye movement fixation order and attention-alignment loss. Fixation-order-informed (FOI) ViT models were found to perform better than the original ViT model, with fewer parameters and faster training. The use of attention-alignment in the ViT loss function resulted in improved performance when the effect of clinician-generated spatial attention was increased. The attention maps generated by these modified ViTs enabled interpretability and made the reasons for missed predictions more transparent especially for our FOI ViT model. Overall, these findings demonstrate the potential of using expert eye-tracking data to improve the performance of ViT models in glaucoma diagnosis.

Track: 4. AI-based clinical decision support systems

Registration Id: JVNNK9NQ6FJ

Submission Number: 193