Adaptive Simulation for Grounding Language Instructions that Refer to the Future State of Objects

Tabib Wasit Rahman, Katelyn Shakir, Thomas Howard

Keywords: Symbol Grounding, Spatiotemporal Relationships, Natural Language Understanding

Abstract: Accurate and efficient communication is essential for human-robot interaction. Spatiotemporal relationships are commonly used in language to resolve ambiguity about referred objects when they cannot be uniquely identified based on visual features. Grounding of instructions that involve spatiotemporal relationships remains a difficult problem in human-robot interaction because of the lack of annotated data and the difficulty of representing or encoding such information in an environment model. This paper outlines an approach that builds from previous methods that explored minimal but sufficient environment models for symbol grounding that address spatiotemporal relationships that project into the future. It specifically explores the application of an adaptive timestep that eliminates unnecessary cycles when we can predict that the outcome of inference will not change due to a small step forward in time. An evaluation based on a small corpus and a detailed example are presented.

Submission Number: 13