Deep Learning with Logged Bandit FeedbackDownload PDF

15 Feb 2018, 21:29 (edited 10 Feb 2022)ICLR 2018 Conference Blind SubmissionReaders: Everyone
  • Keywords: Batch Learning from Bandit Feedback, Counterfactual Learning
  • TL;DR: The paper proposes a new output layer for deep networks that permits the use of logged contextual bandit feedback for training.
  • Abstract: We propose a new output layer for deep neural networks that permits the use of logged contextual bandit feedback for training. Such contextual bandit feedback can be available in huge quantities (e.g., logs of search engines, recommender systems) at little cost, opening up a path for training deep networks on orders of magnitude more data. To this effect, we propose a Counterfactual Risk Minimization (CRM) approach for training deep networks using an equivariant empirical risk estimator with variance regularization, BanditNet, and show how the resulting objective can be decomposed in a way that allows Stochastic Gradient Descent (SGD) training. We empirically demonstrate the effectiveness of the method by showing how deep networks -- ResNets in particular -- can be trained for object recognition without conventionally labeled images.
  • Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10)
10 Replies

Loading