Coping with sample inefficiency of deep-reinforcement learning (DRL) for embodied AI

Vidhi Jain, Simin Liu, Ganesh Subramanian Iyer

Abstract:

Conducted session at WiML Un-Workshop, ICML 2020.
Why learning algorithms haven’t been deployed on consumer robotics/AV platforms? For example, robot
vacuum cleaners use currently rely on a few simple algorithms, such as spiral cleaning (spiraling), room
crossing, wall-following and random walk, angle-changing after bumping into an object or wall. There are
many reasons why robotic learning is challenging in real world - lack of safety assurances, reward specification,
lack of progress on the continual learning front. One of the major focus is mainly on lack of sample efficiency.
While we need to have good simulators to train, we also need better ways of acquiring experience in embodied
AI and robots for real platforms. We want to be efficient and reduce monotonous burden with AI tools like
autonomous vehicle and home assistants.
In our breakout session, we discussed about some ways (either proven or promising) to make DRL feasible
for embodied AI. We look into real embodied AI learning in order to fundamentally enhance the notion of
intelligence by incorporating multi-modal interaction. We talk about two divergent approaches: algorithmic
approaches to improve sample efficiency and alternatively, circumventing the sample efficiency problem by
scaling up data collection for current state-of-the-art algorithms.