Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation

Bohan Lyu, Yadi Cao, Duncan Watson-Parris, Leon Bergen, Taylor Berg-Kirkpatrick, Rose Yu

TL;DR: This paper proposes AWL, a method where LLMs first learn scientific knowledge via tool interaction and then adapt by solving easy problems directly and hard ones with tools.

Abstract:

Large Language Models (LLMs) demonstrate promising capabilities in solving scientific problems but often suffer from the issue of hallucination. While integrating LLMs with tools can mitigate this issue, models fine-tuned on tool usage become overreliant on them and incur unnecessary costs.
Inspired by how human experts assess problem complexity before selecting solutions, we propose a novel two-component fine-tuning method, *Adapting while Learning* (AWL). In the first component *World Knowledge Learning* (WKL), LLMs internalize scientific knowledge by learning from tool-generated solutions. In the second component *Tool Usage Adaptation* (TUA), we categorize problems as easy or hard based on the model's accuracy, and train it to maintain direct reasoning for easy problems while switching to tools for hard ones.
We validate our method on 6 scientific benchmark datasets across climate science, epidemiology, physics, and other domains. Compared to the original instruct model (8B), models post-trained with AWL achieve 29.11\% higher answer accuracy and 12.72\% better tool usage accuracy, even surpassing state-of-the-art models including GPT-4o and Claude-3.5 on 4 custom-created datasets. Our code is open-source at \url{https://github.com/Rose-STL-Lab/Adapting-While-Learning}.

Lay Summary:

Large language models (LLMs) are starting to help scientists tackle complex questions in fields like climate science and epidemiology. However, these models sometimes make up answers to hard questions or use expensive scientific tools even when a simple answer would do. This inefficiency can make them unreliable or costly to use for real scientific work.

Inspired by how human experts decide when to use a calculator and when to solve something in their head, we developed a new training method called Adapting While Learning (AWL). Our approach teaches LLMs to first try to answer scientific questions on their own, and only use specialized tools when the question is truly difficult. We do this by splitting training into two parts: first, the model learns scientific knowledge from tool-generated solutions; second, it is trained to recognize which questions are easy (and can be answered directly) and which are hard (and need a tool).

We tested our method on six scientific datasets, ranging from math and physics to climate and disease modeling. Our trained models not only became more accurate, but also learned to use tools more wisely, saving time and resources. In fact, our approach even outperformed some of the largest, most advanced AI models on new, challenging datasets. We hope this work leads to more trustworthy, efficient AI assistants for science.

Link To Code: https://github.com/Rose-STL-Lab/Adapting-While-Learning

Primary Area: Deep Learning->Large Language Models

Keywords: LLM Alignment, AI for Science

Submission Number: 4000