Self-attention Networks Localize When QK-eigenspectrum Concentrates

Open Webpage

Han Bao, Ryuichiro Hataya, Ryo Karakida

Published: 01 Jan 2024, Last Modified: 07 Oct 2024ICML 2024EveryoneRevisionsBibTeXCC BY-SA 4.0