Learning to Look by Self-Prediction

Matthew Koichi Grimes, Joseph Varughese Modayil, Piotr W Mirowski, Dushyant Rao, Raia Hadsell

Abstract: We present a method for learning active vision skills, to move the camera to observe a robot's sensors from informative points of view, without external rewards or labels. We do this by jointly training a visual predictor network, which predicts future returns of the sensors using pixels, and a camera control agent, which we reward using the negative error of the predictor. The agent thus moves the camera to points of view that are most predictive for a chosen sensor, which we select using a conditioning input to the agent. We observe that despite this noisy learned reward function, the learned policies a exhibit competence by reliably framing the sensor in a specific location in the view, an emergent location which we call a behavioral fovea. We find that replacing the conventional camera with a foveal camera further increases the policies' precision.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: Camera-ready submission. Text previously highlighted in red for the reviewers has now been reverted to black.

Video: https://drive.google.com/file/d/1iOFC1IWk0Jx-poBZW0lh5FcTmT55BdDw/view?usp=sharing

Assigned Action Editor: ~Stefan_Lee1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 653