Memorize and Rank: Enabling Large Language Models for Medical Event Prediction

Mingyu Derek Ma, Yijia Xiao, Anthony Cuturrufo, Xiaoxuan Wang, Wei Wang

Track: Traditional track

Keywords: Generative LM, LLM, Diagnosis Prediction

Abstract: Medical event prediction produces patient's potential diseases given their visit history. It is personalized yet requires an in-depth understanding of domain knowledge. Existing works integrate clinical knowledge into the prediction with techniques like concept embedding, patient records as knowledge graphs, and external knowledge bases, leaving the knowledge obtained through the pretraining of modern Large Language Models untouched. We introduce Mera, a clinical event prediction model that bridges pertaining natural language knowledge with medical code. We apply contrastive learning on a predicted ranking list for task-specialized optimization. With concept memorization through fine-tuning, we equip the LLM with an in-depth understanding to recall the natural language definitions for medical code during inference. Experimental results on MIMIC datasets show that Mera outperforms state-of-the-art models.

Presentation And Attendance Policy: I have read and agree with the symposium's policy on behalf of myself and my co-authors.

Ethics Board Approval: No, our research does not involve datasets that need IRB approval or its equivalent.

Data And Code Availability: Yes, we will make data and code available upon acceptance.

Primary Area: Clinical foundation models

Student First Author: Yes, the primary author of the manuscript is a student.

Submission Number: 43