Learning Where to Remember: Dynamic Memory Routing for Reliable LLM Agents

Hyeonjeong Ha, Heng Ji

Keywords: memory-based llm agents, llm hallucination mitigation

TL;DR: We introduce MemRouter, a dynamic memory routing framework with process-level verification to mitigate hallucinations in memory-based LLM agents.

Abstract: Memory-based large language model (LLM) agents extend reasoning beyond the context window by storing and retrieving information from external memory. Inspired by human cognition, recent work attempts to organize memory into different types such as semantic, episodic, and procedural memory. However, existing approaches often store heterogeneous knowledge in a shared memory space or rely on static prompting and heuristic rules to determine memory organization. These designs introduce cross-type interference and inconsistent memory updates, which can propagate errors during retrieval and contribute to hallucinated responses. In this work, we introduce MemRouter, a dynamic memory routing framework with process-level verification for memory-based LLM agents. MemRouter maintains separate memory stores for semantic, episodic, and procedural knowledge and learns a routing policy that determines where extracted knowledge should be written and which memory stores should be queried during retrieval. To ensure reliable memory construction, we further design process-level QA verification, where memory-type-specific question–answer pairs are generated from the evolving context to provide dense feedback for routing decisions. This verification signal encourages memory updates that support correct reasoning while discouraging redundant or inconsistent storage. By combining structured memory separation with learned routing and verification, we expect MemRouter to improve the reliability of stored knowledge and reduces memory contamination in long-horizon reasoning tasks.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 123