Simulating Environments with Large Language Models for Generic Agent Training

Yuetai Li, Huseyin A Inan, Xiang Yue, Wei-Ning Chen, Lukas Wutschitz, Janardhan Kulkarni, Radha Poovendran, Robert Sim, Saravan Rajmohan

Keywords: Large Language Models, Agent Training, Environment Simulation, Synthetic Data, Tool Use, SIMAGENT

TL;DR: This paper proposes SIMAGENT, a framework using LLM simulated environment for scalable synthetic trajectory generation that improves agent training across benchmarks.

Abstract: LLM agents excel in compact environments requiring deep reasoning but remain brittle when operating in broader, more complex contexts that demand robustness across diverse tools and schemas. Building bespoke environments for training is costly and brittle, limiting progress. We propose \textsc{SimAgent}, a framework that simulates diverse environments with reasoning models to generate complete trajectories from compact seed sets without executing real systems. Our approach combines diversity expansion with schema-verified generation, producing synthetic data that is both broad and training-ready. Fine-tuning open models on these trajectories yields consistent improvements across multiple benchmarks, in some cases surpassing GPT-4 and approaching GPT-4o, demonstrating that environment simulation provides a generic path for advancing agentic LLMs.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 20502