Learning Game-Playing Agents with Generative Code Optimization

Zhiyi Kuang, Ryan Rong, YuCheng Yuan, Allen Nie

Keywords: Agent Learning, Large Language Model Optimizer

TL;DR: We train Atari game-playing agents by optimizing Python programs with LLMs, enabling efficient, self-improving policies that match deep RL performances with far less training time and fewer environment interactions.

Track: Short Paper (up to 4 pages)

Abstract: We present a generative optimization approach for learning game-playing agents, where policies are represented as Python programs and refined using large language models (LLMs). Our method treats decision-making policies as self-evolving code, with current observation as input and an in-game action as output, enabling agents to self-improve through execution traces and natural language feedback with minimal human intervention. Applied to Atari games, our game-playing Python program achieves performance competitive with deep reinforcement learning (RL) baselines while using significantly less training time and much fewer environment interactions. This work highlights the promise of programmatic policy representations for building efficient, adaptable agents capable of complex, long-horizon reasoning.

Format: We have read the camera-ready instructions, and our paper is formatted with the provided template.

De-Anonymization: This submission has been de-anonymized.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 12