Accumulated Expectations Improve Intelligent Agent Adaptation in Dynamic Environments

Paola Rizzo, David W. Aha, Hector Munoz-Avila

Keywords: expected action effects, plan execution, plan monitoring, plan repair, goal reasoning, dynamic environments

TL;DR: This paper introduces and empirically evaluates accumulated expectations, which are a new type of expectations that maintain a history of the assumed effects of an agent’s executed actions to enhance its adaptation abilities in dynamic environments.

Abstract: Intelligent agents that adapt to dynamic environments need to (1) verify that the actual effects of their actions are consistent with their expected effects (generated during planning) and (2) change their execution schedules, actions, plans or goals if discrepancies arise among the expected and actual effects. In this paper we introduce accumulated expectations, which are a new type of expectations that maintain a history of the assumed effects of an agent’s executed actions to enhance its adaptation abilities. We formally define accumulated expectations, contrast them with other forms of expectations that have previously been studied, and test them in dynamic simulation environments using both a single agent task domain (maritime search and rescue) and a multiagent task domain (aerial survey). In our evaluations, we found that agents that use accumulated expectations significantly outperform agents that do not use them. More specifically, for the maritime search and rescue domain, using accumulated expectations reduces the negative impact of a dynamic element on performance, and for the aerial survey domain, it improves performance through positive interactions among the agents' behaviors.

Paper Track: Technical paper

Submission Number: 41