Representing, learning, and controlling complex object interactions

Yilun Zhou, Benjamin Clark Malloy Burchfiel, George Konidaris

Abstract:

We present a framework for representing scenarios with complex object interactions, where a robot cannot directly interact
with the object it wishes to control and must instead influence it via intermediate objects. For instance, a robot learning to drive
a car can only change the car’s pose indirectly via the steering wheel, and must represent and reason about the relationship
between its own grippers and the steering wheel, and the relationship between the steering wheel and the car. We formalize
these interactions as chains and graphs of Markov decision processes (MDPs) and show how such models can be learned from
data. We also consider how they can be controlled given known or learned dynamics. We show that our complex model can be
collapsed into a single MDP and solved to find an optimal policy for the combined system. Since the resulting MDP may be
very large, we also introduce a planning algorithm that efficiently produces a potentially suboptimal policy. We apply these
models to two systems in which a robot uses learning from demonstration to achieve indirect control: playing a computer
game using a joystick, and using a hot water dispenser to heat a cup of water.