Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models

Tyler A. Chang, Yifan Xu, Weijian Xu, Zhuowen Tu

Abstract: Tyler Chang, Yifan Xu, Weijian Xu, Zhuowen Tu. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.