Generous teacher: Good at distilling knowledge for student learning

Yifeng Ding, Gaoming Yang, Shuting Yin, Ji Zhang, Xianjin Fang, Wencheng Yang

Abstract: Highlights•Study knowledge distillation from the teacher's perspective and introduce a novel teacher enhancement method (Generous Teacher).•The knowledge learned by the Generous Teacher is more conducive to student distillation learning.•Replace the standard teacher with the Generous Teacher incurs almost no additional computational or storage costs.•Generous Teacher can be seamlessly integrated with other distillation methods, consistently achieving better performance.