From Context to Action: Analysis of the Impact of State Representation and Context on the Generalization of Multi-Turn Web Navigation Agents

Nalin Tiwary, Vardhan Dongre, Sanil Arun Chawla, Ashwin Lamani, Dilek Hakkani Tur

Keywords: Speech & Natural Language Processing (SNLP) -> SNLP: Conversational AI/Dialog Systems, Data Mining & Knowledge Management (DMKM) -> DMKM: Web-Based QA, Data Mining & Knowledge Management (DMKM) -> DMKM: Web Search & Information Retrieval

TL;DR: This study explores the optimization of context management in LLM-based multi-turn web navigation agents, highlighting the importance of interaction history and web page representation for improved performance in diverse scenarios.

Abstract: Recent advancements in Large Language Model (LLM)-based frameworks have extended their capabilities to complex real-world applications, such as interactive web navigation. These systems, driven by user commands, navigate web browsers to complete tasks through multi-turn dialogues, offering both innovative opportunities and significant challenges. Despite the introduction of benchmarks for conversational web navigation, a detailed understanding of the key contextual components that influence the performance of these agents remains elusive. This study aims to fill this gap by analyzing the various contextual elements crucial to the functioning of web navigation agents. We investigate the optimization of context management, focusing on the influence of interaction history and web page representation. Our work highlights improved agent performance across out-of-distribution scenarios, including unseen websites, categories, and geographic locations through effective context management. These findings provide insights into the design and optimization of LLM-based agents, enabling more accurate and effective web navigation in real-world applications.

Submission Number: 105