Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel

Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov