Multi-distribution noise quantisation: an extreme compression scheme for transformer according to parameter distribution

Zai Yang Yu, Shuang Li, Linjun Sun, Liang Liu, Haining Wang

Abstract: With the development of deep learning, neural networks are widely used in various fields, and the improved model performance also introduces a considerable number of parameters and computations. Mo...