Recognizing Plans by Learning Embeddings from Observed Action Distributions

Yantian Zha, Yikang Li, Sriram Gopalakrishnan, Hankz Hankui Zhuo, Baoxin Li, Subbarao Kambhampati

TL;DR: Handling Uncertainty in Visual Perception for Plan Recognition

Abstract:

Plan recognition aims to look for target plans to best explain the observed actions based on plan libraries and/or domain models. Despite the success of previous approaches on plan recognition, they mostly rely on correct action observations.
Recent advances in visual activity recognition have the potential of enabling applications such as automated video surveillance. Effective approaches for such problems would require the ability to recognize the plans of agents from video information. Traditional plan recognition algorithms rely on access to detailed planning domain models. One recent promising direction involves learning approximate (or shallow) domain models directly from the observed activity sequences. Such plan recognition approaches expect observed action sequences as inputs. However, visual inference results are often noisy and uncertain, typically represented as a distribution over possible actions. In this work, we develop a visual plan recognition framework that recognizes plans with an approximate domain model learned from uncertain visual data.

Code: https://drive.google.com/open?id=1tEr6zrXvmm0VvqqO0ecf-dUV7Ave0Rw0

Keywords: action representation learning, plan recognition, shallow model planning