Scaling LLM Agents with Self-Evolving Structured Memory

ACL ARR 2026 March Submission1170 Authors

Keywords: Large Language Models, Deep Search Agent, Question Answering

Abstract:

With the growing adoption of large language model (LLM) agents in persistent real-world roles, they increasingly face continuous streams of complex tasks requiring iterative reasoning and evidence integration.
A common strategy for improving agent performance is inference-time scaling, which allocates additional computation to exploration and reasoning.
However, existing self-evolving agents typically scale by expanding interaction trajectories, leaving the knowledge accumulated largely unstructured and making it difficult to consolidate discoveries or guide further reasoning.
We address this limitation by reframing inference-time scaling as structured knowledge accumulation rather than trajectory expansion.
We propose StructMem, a framework that represents the agent's evolving knowledge state as a dynamically constructed knowledge graph.
During exploration, newly discovered facts are incrementally integrated into the graph, where structural constraints enable conflict detection, knowledge gap identification, and consistency-aware reasoning.
Through this iterative build–verify–expand process, StructMem progressively constructs a coherent knowledge structure that supports grounded and verifiable answer generation.
Experiments on challenging knowledge-intensive benchmarks demonstrate substantial improvements over existing agents in reasoning accuracy and robustness.
The code is available at https://anonymous.4open.science/r/StructMem-code-0104.

Paper Type: Long

Research Area: LLM agents

Research Area Keywords: Autonomous agents,LLM agents,agent memory

Contribution Types: NLP engineering experiment

Languages Studied: English,Chinese

Submission Number: 1170