DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for Autonomous Travel Planning Agents

ICLR 2026 Conference Submission5522 Authors

Keywords: agents, travel planning, LLMs, agentic reinforcement learning

TL;DR: In this paper, we present DeepTravel, an end-to-end agentic reinforcement learning framework to build autonomous travel planning agents.

Abstract:

Travel planning (TP) agent has recently worked as an emerging building block to interact with external tools/resources for travel itinerary generation, ensuring enjoyable user experience.
Despite its benefits, existing studies rely on hand-craft prompt and fixed agent workflow, hindering more flexible and autonomous TP agent.
This paper proposes DeepTravel, an end-to-end agentic reinforcement learning framework for building autonomous travel planning agent, capable of autonomously planning, executing tools, and reflecting on tool responses to explore, verify, and refine intermediate actions in multi-step reasoning.
To achieve this, we first construct a robust sandbox environment by caching transportation, accommodation and POI data, facilitating TP agent training without being constrained by real‑world APIs limitations (e.g., inconsistent outputs).
Moreover, we develop a hierarchical reward modeling system, where a trajectory‑level verifier first checks spatiotemporal feasibility and filters unsatisfied travel itinerary, and then the turn‑level verifier further validate itinerary's detail consistency with tool responses, enabling efficient and precise reward service.
Finally, we propose the reply-augmented reinforcement learning method that enables TP agent to periodically replay from a failures experience buffer, emerging notable agentic capacity.
We deploy trained TP agent on DiDi Enterprise Solutions App and conduct comprehensive online and offline evaluations, demonstrating that DeepTravel enables small-size LLMs (e.g., Qwen3-32B) to significantly outperform existing frontier LLMs such as OpenAI-o1/o3 and DeepSeek-R1 in travel planning tasks.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 5522