Exploring the Pre-conditions for Memory-Learning Agents
Keywords: LLM Agent, Web Navigation, Memory Learning, Self-Improvement
TL;DR: Memory-learning agents require strong models for effective induction, but weaker models can improve through memory transfer from stronger ones. Additionally, our devised learning curriculum reduces memory costs while slightly enhancing performance.
Abstract: Digital agents supported by large language models (LLMs) have demonstrated potential in real-world tasks such as web navigation, especially with growing memory by learning from past experiences and applying them in later tasks (Wang et al., 2024). Nonetheless, it is unclear if arbitrary agents can benefit from this memory adaption procedure, or in other words, what are the pre-conditions for such adaptive-memory approaches to show effect. In this work, we first ask: Does memory-learning agents necessitate certain model capabilities? We apply AWM to top open-weight LLaMa and DeepSeek-R1 models and observe minimal gains; nonetheless, adopting or transferring memory induced by strong GPT increases success by 64–87%, suggesting that the memory induction module is critical and has stricter capability requirements. We further ask: Can we optimize memory design to loosen the model capability constraint? We propose to induce increasingly granular workflows and schedule their integration into agent memory via curriculum learning. While it shows little correctness improvements, it reduces the compute by 58.2%. Overall, we reveal the pre-conditions of effective memory-learning agents on memory design and induction quality.
Submission Number: 67
Loading