Mixhead: Breaking the low-rank bottleneck in multi-head attention language models

Zhong Zhang, Nian Shao, Chongming Gao, Rui Miao, Qinli Yang, Junming Shao