Which Memory Operation Drives Recovery? A Factorial Study of Retrieve, Write, and Manage Adaptation under Domain Shift

Zhaoxiang Feng, Mingyang Yao, Charlie Sun, David Scott Lewis

Keywords: lifelong agents, memory adaptation, domain shift, Thompson sampling, factorial ablation, retrieval-augmented generation

TL;DR: A factorial study isolating retrieve, write, and manage adaptation in LLM agent memory under domain shift finds that store management yields the largest in-distribution gains, while post-shift recovery is dominated by base model competence.

Abstract: Lifelong LLM agents rely on external memory to accumulate and reuse experience, but memory is not a single retrieval knob. A deployed agent's memory involves three coupled operations: retrieving (PROVIDE), writing (TAKE-IN), and managing (MANAGE). When the task domain shifts, it is unclear which operation's adaptation matters most for recovery. Prior adaptive-retrieval work adapts only the retrieval pathway over static corpora, while agent memory systems update content online but with fixed write and manage heuristics. We introduce a factorial study that independently enables or disables online adaptation for each operation under a streaming code-to-math domain shift. Using OMAC, an online memory architecture controller based on block-level Thompson sampling, we compare six conditions: no memory, frozen configuration, full joint adaptation, and three single-operation ablations. Across 60 runs (6 conditions, 10 seeds, 979 episodes each), we find that (1) memory provides substantial benefit during in-distribution deployment, raising code accuracy from 39% to 54-61%; (2) online adaptation outperforms frozen configurations for in-distribution learning, with MANAGE adaptation yielding the largest gains; and (3) post-shift recovery is dominated by the base model's domain competence, with all conditions converging rapidly when the target domain is easier. Our results highlight that the value of memory adaptation depends on domain difficulty and that store management deserves more attention alongside retrieval adaptation.

Submission Number: 226