On the Importance of Expert Knowledge to Improve Foundation Models for Retinal Fundus Images

Julio Silva-Rodríguez, Hadi Chakor, Riadh kobbi, Jose Dolz, Ismail Ben Ayed

Keywords: Foundation models, Fundus image, Vision-language pre-training

Abstract: Foundation models are currently revolutionizing the medical image analysis community. Pre-trained on large data sources, such networks provide efficient transferability to downstream tasks. In this context, a myriad of foundation models leveraging large amounts of general medical data and increasing network sizes are appearing in the literature. In this short paper, we study the importance of incorporating domain-specific expert knowledge during pre-training of specialized foundation models in the context of fundus retina images. In particular, we focus on introducing the expert knowledge-driven vision-language model FLAIR (Silva-Rodriguez et al., 2023), comparing its benefits to larger-scale generalists and domain-specific self-supervised models.

Submission Number: 144