Memory-Augmented Large Language Model-Based Agent with Cross-Task Experience Learning

Sijia Wang, Gengzhi Zhang, Xuefeng Chen, Liang Feng

Keywords: Large Language Models, LLM-based Agents, Experience Transfer, Long-Term Memory Mechanisms

TL;DR: We propose a memory-augmented LLM agent with cross-task learning and dynamic memory retrieval to improve adaptability and efficiency in multi-turn instruction-following tasks.

Abstract: Large Language Model (LLM)-based agents have demonstrated impressive capabilities in complex decision-making and multi-turn instruction-following tasks. To enhance knowledge retention and contextual adaptability, recent work has equipped these agents with memory modules that store and reuse historical interaction experiences. However, existing memory-augmented approaches face two key limitations: they often require large amounts of interaction data during early training to reach competitive performance, resulting in low data efficiency; and they rely on static, self-derived experience reuse strategies, limiting their ability to adapt when prior learning is insufficient and preventing the use of transferable knowledge from related tasks. Building on these observations, in this paper, we propose a memory-augmented LLM agent with cross-task experience learning, designed to improve data efficiency and adaptability. Our method augments the conventional task-specific memory with an additional source experience memory that retains transferable knowledge from related but distinct tasks. We further introduce a dynamic memory retrieval mechanism that adaptively draws from both task and source memories, allowing the agent to balance prior task-specific experiences with cross-task knowledge according to the current context and progression. We validate the proposed method on the WebShop benchmark, which comprises diverse, multi-turn instruction-following tasks across product domains with varying semantic complexity. Experimental results show that our approach consistently outperforms state-of-the-art memory-augmented LLM agents in task success rate and generalization, demonstrating the effectiveness of the proposed memory architecture and retrieval mechanism.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 23410