Adaptive Environmental Modeling for Task-Oriented Language Agents
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Large Language Model, Environmental Adaptation, Agents, Interactive Decision Making
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Recent advancements in the realm of intelligent agents, particularly those employing large language models, have been notably significant. Notwithstanding these advancements, intelligent agents encounter substantial challenges, predominantly in interactive and dynamic scenarios such as online shopping, attributed to an absence of integrated environmental modeling. In this paper, we propose a task-oriented environmental adaptation approach, allowing language agents to autonomously model new environments. This approach comprises two pivotal phases: Pre-Task Environment Exploration and In-Task Environment Update. The Pre-Task Environment Exploration phase incorporates a greedy exploration strategy, leveraging an agent in the role of an Evaluator to optimally explore environmental information based on present observations and feasible actions. This strategy is implemented through a recursive algorithm, enabling agents to choose and execute the top-k scored actions, thereby efficiently forming an Action-Observation Tree as the initial environmental modeling. During the In-Task Environment Update phase, agents employ environmental information to enhance task performance. The information generated from task execution and interaction trajectories is used to refine environmental modeling. These processes are iteratively executed, achieving mutual enhancement. We conduct a systematic evaluation of the environmental modeling, assessing both its effectiveness and comprehensiveness. The results demonstrate that under our approach, agents can indeed construct accurate environmental modeling. Simultaneously, we observe a significant enhancement in agent performance on both the ALFWorld-Eco and the WebShop benchmark datasets due to the application of environmental modeling.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 9182
Loading