Learning State Switching for Improved Exploration in Multi-sensor Environments

Homagni Saha, Sin Yong Tan, Zhanhong Jiang, Soumik Sarkar

Keywords: Multi-agent reinforcement learning, Robotic Planning and control, Manipulator object tracking

Abstract:

Reinforcement learning has been used to achieve state of the art results in several robotics applications. Despite massive success, issues remain regarding transfer to real world domains and being able to optimize over multi-dimensional control
inputs as well as across several agents with different perspective views for the same environment, but sharing the same overall goal (multi-agent, multi-sensor robotics). This paper takes a fresh approach towards multi-sensor multi-agent integration for achieving improved performance using a control theoretic approach of “state-switching”. A formulation based on state switching is adapted as a multi-agent reinforcement learning task in the form of a value iteration algorithm maximizing expected payoffs over time. A reinforcement learning task involving tracking an unknown object with unknown motion dynamics using manipulators is formulated for the well known sawyer one handed manipulator. Thereafter we formulate our state switching algorithm and show superior performance compared to using individual sensors. Our trained agent is then transferred from simulation to a real setup and is shown to perform nicely in the real domain as well.