LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning

Yi Wu, Zhimin Hu

Keywords: LLM Agent, Strategic Reasoning, StarCraft II, Episodic Memory, Cognitive Architecture

TL;DR: We proposed an LLM-based agent with episodic and working memory to enhance strategic reasoning by balancing coherent strategy and adaptation. We tested it on StarCraft II showing significant improvements over the baseline with fewer token budget.

Abstract: Strategic reasoning in dynamic environments, such as games, requires a balance between long-term strategy and short-term adaptations. Although specially trained agents can achieve superhuman performance, they often lack explainability and are highly dependent on extensive data for training. In contrast, approaches that leverage large language models (LLMs) benefit from few-shot learning but struggle to maintain strategic consistency. Drawing inspiration from existing cognitive models of human decision-making, which utilize various forms of memory, we introduce EpicStar, an LLM-based agent with cognitively inspired episodic and working memory modules. Episodic memory enables agents to draw on past experiences to formulate coherent long-term strategies, while working memory modulates active observation and decision variables essential for adaptation. We evaluated EpicStar in the strategy game StarCraft II, where it competes effectively against built-in agents at Level 6 difficulty, surpassing its predecessor at Level 5 with a smaller token budget. Our approach not only enhances adaptability, but also ensures strategic consistency, demonstrating the pivotal role that cognitive memory can play in strategic reasoning.

Submission Number: 117