Training Free Adaptive Text Classification through Aggregated Large Language Models

Iman Zare Estakhraji, Allison Li, Andreas Markus Loening, Akshay S Chaudhari, R. Marc Lebel

Keywords: Classification, Medical Foundation Models, Large Language Models, Continual Learning

TL;DR: In this paper, we propose a method for Class-incremental classification in an online environment using LLMs, without the need for re-training.

Abstract:

Class-incremental classification problems typically requires continual learning of the underlying algorithm to adapt to new classes. While current-generation large language models (LLMs) can have excellent few shot performance on several tasks, many tasks still require retraining to account for distribution shifts at either the inputs or task level. Continual learning techniques could be applied to LLMs, yet this requires retraining multiple task- and distribution-specific LLMs versions.
Additionally, for specific applications like medical applications, maintaining compliance with regularity standards becomes challenging as models evolves, requiring transparency and accountability. We overcome these challenges by introducing a semantic search-based method that simultaneously uses multiple LLM vectorizers/encoders and prompts without requiring any fine tuning. We depict that our proposed method has performance comparable to that of LLM fine tuning for clinical (MR and CT protocoling) datasets. In this approach, instead of being restricted to fine-tuning a single LLM, multiple foundation models(LLMs)/vectorizers will be leveraged simultaneously, maximizing their capability without incurring extra expenses for retraining or fine tuning, meaning that their individual potentials will be aggregated to determine the final outcomes. Our method could be utilized for continual learning environments, eliminating the need for retraining and adapts dynamically to incoming data, ensuring continuous updating. This approach uses the diverse perspectives and strengths provided by different LLMs and prompts, enhancing the robustness and comprehensiveness of the responses. By aggregation of different foundation models without the need for fine-tuning, this method demonstrates encouraging accuracy and reliability for medical and non-medical datasets, as multiple LLMs/prompts can highlight various aspects of the same issue, mitigating the biases and limitations that may arise from using a single prompt or model.

Submission Number: 15