No-Regret Shannon Entropy Regularized Neural Contextual Bandit Online Learning for Robotic Grasping

Kyungjae Lee, Jaegu Choy, Yunho Choi, Hogun Kee, Songhwai Oh

Abstract: In this paper, we propose a novel contextual bandit algorithm that employs a neural network as a reward estimator and utilizes Shannon entropy regularization to encourage exploration, which is called Shannon entropy regularized neural contextual bandits (SERN). In many learning-based algorithms for robotic grasping, the lack of the real-world data hampers the generalization performance of a model and makes it difficult to apply a trained model to real-world problems. To handle this issue, the proposed method utilizes the benefit of an online learning. The proposed method trains a neural network to predict the success probability of a given grasp pose based on a depth image, which is called a grasp quality. We theoretically show that the SERN has a no regret property. We empirically demonstrate that the SERN outperforms ε-greedy in terms of sample efficiency.