Can LLM Coding Agents Reason About Time Series?

ACL ARR 2026 May Submission14041 Authors

Keywords: time series analysis, LLM agents, code generation, benchmarking, LLM-as-a-judge, reasoning evaluation, multiple-choice QA

Abstract: Large language models (LLMs) are increasingly being used for automated decision-making systems in finance, healthcare, or environmental monitoring. Time series data are ubiquitous in these fields, yet hard to process automatically. Can time series be analyzed by LLM agents? We examine three approaches: providing the agent with raw numerical data, using the LLM as a coding agent, or a combination of both. In the coding agent setup, the model iteratively queries the data using Python code. Using two time series understanding benchmarks, we show that agents with code access can outperform models processing raw data by up to 10%. However, even the best performing agent still answers about 22-34% of the questions incorrectly. To get insights into models' strategies and reasoning gaps, we analyze the model outputs with a strong LLM judge. Our analysis reveals that coding agents can select appropriate statistical tests, but often miss important nuances. Meanwhile, models with access to raw data can reach the right conclusions using back-of-the-envelope calculations.

Paper Type: Long

Research Area: LLM agents

Research Area Keywords: tool use; function calling; agent evaluation

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: English

EMNLP 2026 AI Reviewing Experiment: yes

Submission Number: 14041