Generating Plannable Lifted Action Models for Visually Generated Logical Predicates

Masataro Asai

Keywords: Object-Centric Representation, Planning, Discrete VAE

Abstract: We propose FOSAE++, an unsupervised end-to-end neural system that generates a compact discrete state transition model (dynamics / action model) from raw visual observations. Our representation can be exported to Planning Domain Description Language (PDDL), allowing symbolic state-of-the-art classical planners to perform high-level task planning on raw observations. FOSAE++ expresses states and actions in First Order Logic (FOL), a superset of so-called object-centric representation. It is the first unsupervised neural system that fully supports FOL in PDDL action modeling, while existing systems are limited to continuous, propositional, or property-based representations, and/or require manually labeled input for actions/predicates/propositions.

One-sentence Summary: From pixel data of objects, FOSAE++ generates a symbolic planning domain description generalized over objects. The system is fully neural and requires no manual tagging.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=d1A6zmIss