Mechanistic Interpretability of Binary and Ternary Transformer Networks
Keywords: mechanistic interpretability, binary neural networks, quantization
TL;DR: Reverse engineering the BitNet binary transformer after it has grokked on modular addition
Abstract: Recent research (Wang et al., 2023; Ma et al.,2024) has proposed binary and ternary transformer networks as a way to significantly reduce memory and improve inference speed in Large Language Models (LLMs) while maintaining accuracy. In this work, we apply techniques from mechanistic interpretability to investigate whether such networks learn distinctly different or similar algorithms when compared to full-precision transformer networks. In particular, we reverse engineer the algorithms learned for the toy problem of modular addition where we find that binary and ternary networks learn similar algorithms as full precision networks. This provides evidence against the possibility of using binary and ternary networks as a more interpretable alternative in the LLM setting.
Submission Number: 15
Loading