Measuring Visual Generalization in Continuous Control from Pixels

Jake Grigsby, Yanjun Jane Qi

Keywords: Continuous Control, Reinforcement Learning

Abstract: Self-supervised learning and data augmentation have significantly reduced the performance gap between state and image-based reinforcement learning agents in continuous control tasks. However, it is still unclear whether current techniques can face the variety of visual conditions required by real-world environments. We propose a challenging benchmark that tests agents' visual generalization by adding graphical variety to existing continuous control domains. Our empirical analysis shows that current methods struggle to generalize across a diverse set of visual changes, and we examine the specific factors of variation that make these tasks difficult. We find that data augmentation techniques outperform self-supervised learning approaches and that more significant image transformations provide better visual generalization.

One-sentence Summary: We propose a challenging benchmark that tests agents' visual generalization by adding graphical variety to existing continuous control domains.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2010.06740/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=oDJcf7Fufs