Enhancing Clinical Note Summarization: Iterative Reflexions with Small-model Supervision and Error2Correct Demonstrations

Jingping Liu, Boyang Zhong, Weiyan Zhang, Haiyun Jiang, Weichao Ding, Tong Ruan, Jing Sun, Lifeng Zhu

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Clinical Note Summarization, Large Language Model, Error2Correct demonstration

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We are the first to propose a novel iterative reflexion framework for large and small model collaboration with Error2Correct demonstrations in clinical note summarization, and achieve SoTA performance on Chinese and English datasets.

Abstract: Generating clinical notes from doctor-patient dialogues is an important task in medical artificial intelligence. Mainstream methods currently employ large language models with few-shot demonstrations to tackle this challenge. However, the absence of domain knowledge supervision in these models often results in issues like missing key information, irregular writing standards, and non-compliant language styles. To this end, in this paper, we propose a novel iterative reflexion framework with small-model supervision and Error2Correct demonstrations for clinical note summarization. In this framework, we leverage a large model to produce clinical notes and design a small model trained on domain-specific data to evaluate the generated content. To enhance the quality of the generated clinical notes, we further propose Error2Correct demonstrations, which consist of error examples, error analysis, and corresponding correct examples, to help the large model identify and rectify errors effectively. To evaluate the effectiveness of our proposed method, we conduct extensive experiments on both Chinese and English datasets. The results demonstrate that our method achieves state-of-the-art performance on both datasets for the clinical note summarization task.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6694