Influence of artificial intelligence on ophthalmologists’ judgments in glaucoma

Kenji Kashiwagi, Nader Hussien Lotfy Bayoumi, Masahiro Toyoura, Xiaoyang Mao, Kazuhide Kawase, Masaki Tanito, Toru Nakazawa, Atsuya Miki, Kazuhiko Mori, Takeshi Yoshitomi

Abstract: Purpose To examine the influence of artificial intelligence (AI) on physicians’ judgments regarding the presence and severity of glaucoma on fundus photographs in an online simulation system. Methods Forty-five trainee and expert ophthalmologists independently evaluated 120 fundus photographs, including 30 photographs each from patients with no glaucoma, mild glaucoma, moderate glaucoma, and severe glaucoma. A second trial was conducted at least one week after the initial trial in which photograph presentation order was randomized. During the second trial, 30% of the glaucoma judgments made by the AI system were intentionally incorrect. The evaluators were asked about their thoughts on AI in ophthalmology via a 3-item questionnaire. Results The percentage of correct responses for all images significantly improved (P < 0.001) from 48.4 ± 24.8% in the initial trial to 59.6 ± 20.3% in the second trial. The improvement in the correct response rate was significantly greater for trainees (14.2 ± 19.0%) than for experts (8.6 ± 11.4%) (P = 0.04). The correct response rate was 63.9 ± 20.6% when the AI response was correct, significantly greater than the 47.9 ± 26.6% when the AI response was incorrect (P < 0.0001). For trainees, the correct response rate was significantly greater when the AI’s response was correct than when it was incorrect. However, for experts, the effect was less pronounced. The decision time was significantly longer when the AI response was incorrect than when it was correct (P = 0.003). Conclusion In fundus photography-based glaucoma detection, the results of AI systems can influence physicians’ judgments, particularly those of physicians with less experience.

External IDs:doi:10.1371/journal.pone.0321368