Hierarchical Prompting Assists Large Language Model on Web Navigation

Abishek Sridhar, Robert Lo, Frank F. Xu, Hao Zhu, Shuyan Zhou

Submission Type: Regular Short Paper

Submission Track: Dialogue and Interactive Systems

Submission Track 2: Theme Track: Large Language Models and the Future of NLP

Keywords: LLM Prompting, Web Navigation

TL;DR: We proposed a simple hierarchical prompting approach that improve Large Language Models (LLMs) performance on web-navigation task.

Abstract: Large language models (LLMs) struggle on processing complicated observations in interactive decision making. To alleviate this issue, we propose a simple hierarchical prompting approach. Diverging from previous prompting approaches that always put the full observation (a web page) to the prompt, we propose to first construct an action-aware observation which is more condensed and relevant with a dedicated Summarizer prompt. The Actor prompt then predicts the next action based on the summarized history. While our method has broad applicability, we particularly demonstrate its efficacy in the complex domain of web navigation where a full observation often contains redundant and irrelevant information. Our approach outperforms the previous state-of-the-art prompting mechanism with the same LLM by 6.2% on task success rate, demonstrating its potential on interactive decision making tasks with long observation traces.

Submission Number: 5328