LLM-Augmented Incremental Language Parsing for Robotic Agents

Mitchell Abrams, Thies Oelerich, Matthias Scheutz

Keywords: Incremental Language Parsing, Large-Language Models, Online Motion Planning, Cognitive Architecture

TL;DR: A cognitive architecture that integrates a large language model into an incremental parsing and online planning loop, enabling robots to recover from parsing failures in real time by selectively updating grammar.

Abstract:

Robots that operate with humans must understand language robustly in dynamic, ambiguous, and sometimes noisy environments. Incremental parsing integrated with online motion planning allows robots to adapt their actions in real time as language unfolds, but current systems rely on static grammars and dictionaries, making them brittle when encountering novel or unexpected utterances.
We propose a hybrid framework that integrates a large language model (LLM) as a dynamic repair and grammar-adaptation module within a cognitive architecture. When parsing or planning failures arise, the LLM is used to suggest targeted updates to the parser’s grammar or lexicon, guided by feedback from downstream components. We implement the incremental parsing and planning modules and describe how the repair system operates using illustrative examples that highlight its potential to expand linguistic competence over time.

Paper Track: Technical paper

Submission Number: 47