Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial NetworksDownload PDF

Feb 12, 2018 (edited Jun 04, 2018)ICLR 2018 Workshop SubmissionReaders: Everyone
  • TL;DR: Adversarial training for transferring knowledge from teacher network to student network
  • Abstract: There is an increasing interest on accelerating neural networks for real-time applications. We study the student-teacher strategy, in which a small and fast student network is trained with the auxiliary information learned from a large and accurate teacher network. We propose to use conditional adversarial networks to learn the loss function to transfer knowledge from teacher to student. The experiments on three different image datasets show the student network gain a performance boost with proposed training strategy.
5 Replies

Loading