BiCAL: Bi-directional Contrastive Active Learning for Clinical Report Generation

Anonymous

Abstract: State-of-the-art performance by large pre-trained models in computer vision (CV) and natural language processing (NLP) suggests their potential for domain-specific tasks, such as in the medical sector. However, training these models requires vast amounts of labelled data, a challenge in medicine due to the cost and expertise required for data labelling. Active Learning (AL) can mitigate this by selecting minimal yet informative data for model training. While AL has been mainly applied to single-modal tasks in the fields of NLP and CV, its application in multi-modal tasks remains underexplored, such as generating clinical reports from images. In this work, we proposed a novel AL strategy, Bidirectional Contrastive Active Learning strategy (BiCAL), that uses both image and text latent spaces to identify contrastive samples to select batch to query for labels. BiCAL is robust to cold-start learning problem in AL and class imbalance data by its design. Our experiments show that BiCAL outperforms standard methods in clinical efficacy metrics, improving recall by 2.4\% and F1 score by 9.5\%, showcasing its effectiveness in actively training clinical multi-modal models.

Paper Type: long

Research Area: Efficient/Low-Resource Methods for NLP

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: English

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.