How Do Vision Transformers Work?

Namuk Park, Songkuk Kim

Abstract: The success of multi-head self-attentions (MSAs) for computer vision is now indisputable. However, little is known about how MSAs work. We present fundamental explanations to help better understand...