Abstract: Highlights•We design a distillation embedded absorbable pruning method to readily transfer knowledge a teacher network from to a student network.•We propose an asymmetric relation knowledge distillation method to transfer both feature representation knowledge and pairwise similarity knowledge.•Our method achieves the state-of-the-art performance in terms of both accuracy and speed.
Loading