Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation

Yuanxin Liu, Fandong Meng, Zheng Lin, Weiping Wang, Jie Zhou

Abstract: Yuanxin Liu, Fandong Meng, Zheng Lin, Weiping Wang, Jie Zhou. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.