IvyGPT: InteractiVe Chinese Pathway Language Model in Medical Domain

Rongsheng Wang, Yaofei Duan, Chan-Tong Lam, Jiexi Chen, Jiangsheng Xu, Haoming Chen, Xiaohong Liu, Patrick Cheong-Iao Pang, Tao Tan

Abstract: General large language models (LLMs) such as ChatGPT have shown remarkable success. However, such LLMs have not been widely adopted for medical purposes, due to poor accuracy and inability to provide medical advice. We propose IvyGPT, an LLM based on LLaMA that is trained and fine-tuned with high-quality medical question-answer (QA) instances and Reinforcement Learning from Human Feedback (RLHF). In the training, we used QLoRA to handle 33 billion parameters on a small number of NVIDIA A100 (80 GB) GPUs. Experimental results show that IvyGPT has outperformed other medical GPT models. The online demo is available at http://81.71.71.157:52022. Our demo video can be found at https://youtu.be/O4D74pQh8Is.