Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning
Keywords: Large Language Models, Reinforcement Learning, Evolutionary Search, Algorithm Discovery, Self-Improvement, AI for Math
TL;DR: We propose an approach that integrates LLM-based evolutionary search with RL fine-tuning for accelerated discovery of algorithms, as demonstrated on combinatorial optimization tasks.
Abstract: Discovering efficient algorithms for solving complex problems has been an outstanding challenge in mathematics and computer science, requiring substantial human expertise over the years.
Recent advancements in evolutionary search with large language models (LLMs) have shown promise in accelerating the discovery of algorithms across various domains, particularly in mathematics and optimization.
However, existing approaches treat the LLM as a static generator, missing the opportunity to update the model with the signal obtained from evolutionary exploration.
In this work, we propose to augment LLM-based evolutionary search by continuously refining the search operator $-$ the LLM $-$ through reinforcement learning (RL) fine-tuning.
Our method leverages evolutionary search as an exploration strategy to discover improved algorithms, while RL optimizes the LLM policy based on these discoveries.
Our experiments on three combinatorial optimization tasks $-$ bin packing, traveling salesman, and the flatpack problem $-$ show that combining RL and evolutionary search improves discovery efficiency of improved algorithms, showcasing the potential of RL-enhanced evolutionary strategies for more efficient algorithm design.
Submission Number: 34
Loading