Decompose, Adapt and Evolve: Towards Efficient Scientific Equation Discovery with Large Language Models

Pouya Behzadifar, Parshin Shojaee, Sanchit Kabra, Kazem Meidani, Chandan K. Reddy

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: Scientific Discovery, Symbolic Regression, Evolutionary Search, Large Language Models

TL;DR: We propose DecAEvolve (Decomponse, Adapt and Evolve), an approach that leverages structural decomposition, and adaptation through RL fine-tuning to enhance efficiency of LLM-based evolutionary discovery frameworks.

Abstract: Finding mathematical relations underlying natural phenomena and scientific systems has been one of the fundamental tasks in the history of scientific discovery. Recent advancements in evolutionary search with Large Language Models (LLMs), with their embedded scientific knowledge, have shown great promise for this task. However, discovering such mathematical models governing scientific observations still remains significantly challenging, as it requires navigating vast combinatorial hypothesis spaces with an explosion of possible relations. Existing LLM-based approaches overlook the impact of data on the structure of mathematical relations, and treat LLMs as a static hypothesis generator unaware of the observed scientific system. This leads to inefficient exploration of the hypothesis space with over-reliance on LLMs' internal priors. To bridge this gap, we introduce DecAEvolve (Decompose, Adapt, and Evolve), a framework that leverages granular feedback from symbolic term decomposition and LLM refinement through reinforcement learning (RL) fine-tuning to enhance both robustness and efficiency of evolutionary discovery frameworks. Our experiments across diverse datasets demonstrate that DecAEvolve significantly improves the accuracy of discovered equations and the efficiency of the discovery process compared to state-of-the-art baselines.

Submission Number: 419