A diversity-enhanced knowledge distillation model for practical math word problem solving

Yi Zhang, Guangyou Zhou, Zhiwen Xie, Jinjin Ma, Jimmy Xiangji Huang

Abstract: Highlights•We propose a novel diversity-enhanced knowledge distillation model, which adaptively selects high-quality teacher labels as diversity labels for student learning, and estimates the quality of the intermediate soft labels for training student discernment.•We introduce a diversity prior-enhanced student model by incorporating CVAE into existing models, enabling them to capture the latent diversity equations.•We conduct extensive experiments on all the four benchmarks. The results show that the proposed methods effectively improve the performance of existing models without sacrificing model efficiency.