Thinking Like Transformers

Gail Weiss, Yoav Goldberg, Eran Yahav

Abstract: What is the computational model behind a Transformer? Where recurrent neural networks have direct parallels in finite state machines, allowing clear discussion and thought around architecture varia...