Area Attention

Yang Li, Lukasz Kaiser, Samy Bengio, Si Si

Abstract: Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We propose area atten...