Frame Semantics for Human-Robot Interaction

Mitchell Abrams, Chris Bao, Matthias Scheutz

Keywords: semantic parsing human-robot interaction, frame semantics, framenet, large language models, cognitive architecture

TL;DR: We offer an approach to FrameNet-based semantic parsing in a cognitive architecture for embodied agents to represent and reason about language in a richer, context-sensitive way.

Abstract: We propose a novel approach to semantic frame parsing for human-robot interaction (HRI), grounded in the FrameNet framework and integrated into the DIARC cognitive architecture. Our system improves natural language understanding by supporting dynamic interpretation of frame-evoking elements and their associated roles (frame elements) in real time. To expand lexical coverage and improve robustness in open-ended dialogue, we incorporate a large language model (LLM) to suggest additional lexical units for frame evocation and to assist in frame element filling. Our hybrid approach moves beyond existing verb-centric, dependency-based parsers by capturing the full range of frame semantics, including peripheral roles, multiple frame triggers, and frame–frame relations, enabling richer and more flexible interpretation in situated interaction. We demonstrate that our system facilitates downstream reasoning, planning, and reference resolution in situated interaction scenarios.

Paper Track: Technical paper

Submission Number: 50