Inverse Decision Making via Inverse Generative Modeling
Keywords: inverse decision-making, concept learning, inverse RL, generative model for decision-making, learning from demonstrations
Abstract: It is often extremely challenging to infer novel concepts in decision making, such as new actions, goals, or plans, from just a few examples. In this work, we formulate the problem of inferring unfamiliar concepts in decision making as Inverse Decision Making via Inverse Generative Modeling (IDM-IGM). We then introduce a novel concept inference method for this new formulation, which can swiftly adapt to new decision making concepts by leveraging invertible neural generative models. The core idea is to pretrain a generative model on a set of basic concepts and their demonstrations. During test time, given a few demonstrations of a new decision making concept (such as a new goal or a new action), our method can conveniently infer the underlying concept through backpropagation thanks to the invertibility of the generative model. This critically avoids any fine-tuning, greatly accelerating the speed of concept learning. We evaluate our method in three domains -- object rearrangement, goal-reaching, and motion caption of human actions. Our experimental results demonstrate that the pretrained generative model can successfully (1) infer learned concepts and generate agent motion or plans of inferred concepts in unseen environments and (2) infer new compositions of learned concepts or even novel concepts to interpret unseen agent behaviors.
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7880
Loading