Skin Disease Classification with LVLMs: An Empirical Study

Xinyi Zeng, Zimu Wang, Haiyang Zhang, Yiming Luo, Wei Wang

Abstract: Skin diseases pose significant challenges to accurate and efficient diagnosis, often due to their diverse and complex representations. This study investigates the capabilities and limitations of Large Vision-Language Models (LVLMs) in addressing these challenges through skin disease classification tasks. We evaluated LVLMs in zero-shot, few-shot, and finetuning scenarios, exploring their performance, bias, and potential for improvement. Results show that LVLMs lack perceptual granularity in skin disease, though positive signals are also observed. Our findings underscore the necessity for domain- specific optimisation and highlight opportunities for advancing LVLMs in medical diagnostics through innovative strategies and collaborative efforts.