DrKD: Decoupling response-based distillation for object detection

Yilong Lv, Yancheng Cai, Yujie He, Min Li

Abstract: Highlights•Intrinsic sequential training pipelines in object detection would hamper response-based distillation. Therefore, we innovatively optimize the conventional sequential distillation framework into a parallel distillation framework. We propose to decouple the response-based distillation into parallel encoder and decoder distillation. In encoder distillation, we propose a gap-free adapter to bridge the semantic gap. Furthermore, we introduce autocorrelation imitation to further promote student’s performance. In decoder distillation, we set the same inputs for both decoders and then make the outputs close to each other for perfect decoder imitation.