State Design Matters: How Representations Shape Dynamic Reasoning in Large Language Models

Annie Wong, Aske Plaat, Thomas Bäck, Niki van Stein, Anna V Kononova

Abstract: As large language models (LLMs) move from static reasoning tasks toward dynamic environments, their success depends on the ability to navigate and respond to an environment that changes as they interact at inference time. An underexplored factor in these settings is the representation of the state. Holding model parameters fixed, we systematically vary three key aspects: (1) state granularity (long form versus summary), (2) structure (natural language versus symbolic), and (3) spatial grounding (text-only versus images or textual map encodings) across sequential decision-making benchmarks. We find that trajectory summarisation improves performance by reducing noise and stabilising long-horizon reasoning. Second, natural language representations are the most robust across models, whereas structured encodings help mainly for models with strong code or structured output priors, such as JSON schemas. Third, while image-inputs show some benefit, text-based spatial encodings prove most effective. This advantage stems not from the spatial information itself, but from the act of construction, which compels the model to perform the spatial reasoning that static input does not elicit. Overall, we demonstrate that design choices for representing state are a decisive factor in performance, distinct from the availability of information itself. We note, however, that even with improved representations, current LLMs and VLMs remain brittle over long horizons, particularly when they must synthesise information to manage multiple subtasks to reach a goal.

Submission Type: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

• Updated the manuscript to use the official TMLR camera-ready format by enabling the accepted option in the template (\usepackage[accepted]{tmlr}).
• Added the OpenReview link to the manuscript.
• Made the author names visible in the manuscript.
• Replaced the anonymous repository link with the public GitHub repository link.
• Removed all blue text that previously marked additions made during the revision process.
• Performed an additional proofreading pass to correct grammatical errors and resolve minor inconsistencies throughout the manuscript.

Code: https://github.com/ann-w/state-representations-llms-dynamic-tasks

Assigned Action Editor: ~Chen_Sun1

Submission Number: 7096