Evaluating the ChatGPT family of models for biomedical reasoning and classification

Shan Chen, Yingya Li, Sheng Lu, Hoang Van, Hugo J. W. L. Aerts, Guergana K. Savova, Danielle S. Bitterman

Abstract: Large language models (LLMs) have shown impressive ability in biomedical question-answering, but have not been adequately investigated for more specific biomedical applications. This study investigates ChatGPT family of models (GPT-3.5, GPT-4) in biomedical tasks beyond question-answering.