Fusing LLMs with Scientific Literature for Heuristic Discovery

ICLR 2026 Conference Submission20278 Authors

Keywords: Large Language Models, Algorithm Design

Abstract: The automated design of high-performance algorithms, particularly for NP-hard optimization problems, remains a significant challenge. While Large Language Models (LLMs) demonstrate remarkable code generation capabilities, their reliance on internalized, general-purpose knowledge often limits their efficacy in crafting sophisticated, domain-specific heuristics. This paper introduces Knowledge-Augmented Evolutionary Algorithm Design (KA-EAD), a novel framework that synergistically integrates LLMs with evolutionary computation and dynamic knowledge retrieval from scientific literature. KA-EAD orchestrates a co-evolutionary process where LLMs act as intelligent generative and mutative operators. Crucially, at strategic junctures, the system formulates queries based on intermediate evolutionary artifacts (e.g., LLM-generated reflections) to retrieve pertinent `knowledge chunks' from a curated, domain-specific corpus. This retrieved, verifiable knowledge is then injected into the LLM's context, guiding it to generate more informed and effective algorithmic solutions. By explicitly grounding the LLM's creative process in external scientific insights, KA-EAD transcends the limitations of relying solely on pre-trained knowledge, enabling a more targeted and robust exploration of the heuristic design space. It showcases a step towards AI systems that can actively learn from and build upon human scientific progress.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 20278