Automata Guided Skill Composition

Xiao Li, Yao Ma, Calin Belta

Abstract:

Skills learned through (deep) reinforcement learning often generalizes poorly
across tasks and re-training is necessary when presented with a new task. We
present a framework that combines techniques in formal methods with reinforcement
learning (RL) that allows for the convenient specification of complex temporal
dependent tasks with logical expressions and construction of new skills from existing
ones with no additional exploration. We provide theoretical results for our
composition technique and evaluate on a simple grid world simulation as well as
a robotic manipulation task.

Keywords: Skill composition, temporal logic, finite state automata

TL;DR: A formal method's approach to skill composition in reinforcement learning tasks