Distilling efficient Vision Transformers from CNNs for semantic segmentation

Xu Zheng, Yunhao Luo, Pengyuan Zhou, Lin Wang

Abstract: Highlights•Novel C2VKD framework for learning compact ViT student from CNN teacher.•We present VLFD and PDD modules to distill visual and linguistic features.•C2VKD achieves state-of-the-art performance on three segmentation datasets.