Learning Automata from Demonstrations, Examples, and Natural Language

Marcell Vazquez-Chanlatte, Karim Elmaaroufi, Stefan Witwicki, Matei Zaharia, Sanjit A. Seshia

Keywords: Neurosymbolic Artificial Intelligence, Multimodal Learning, Learning Automata, Learning from Demonstrations, Deterministic Finite Automata (DFA), Large Language Models (LLMs), Formal Task Specification

TL;DR: We present $L^*LM$, a novel neurosymbolic algorithm for sample-efficient, multimodal learning of DFAs from expert demonstrations and natural language using large language models.

Abstract: Expert demonstrations have proven an easy way to indirectly specify complex tasks. Recent algorithms even support extracting unambiguous formal specifications, e.g. deterministic finite automata (DFA), from demonstrations. Unfortunately, these techniques are generally not sample-efficient. In this work, we introduce $L^\star LM$, an algorithm for learning DFAs from both demonstrations \emph{and} natural language. Due to the expressivity of natural language, we observe a significant improvement in the data efficiency of learning DFAs from expert demonstrations. Technically, $L^\star LM$ leverages large language models to answer membership queries about the underlying task. This is then combined with recent techniques for transforming learning from demonstrations into a sequence of labeled example learning problems. In our experiments, we observe the two modalities complement each other, yielding a powerful few-shot learner.

Submission Number: 17