Tokenized SAEs: Disentangling SAE Reconstructions

Thomas Dooms, Daniel Wilhelm

Keywords: Sparse Auto-Encoders, N-grams, Class Imbalance

TL;DR: We propose using a per-token bias in SAEs to separate token reconstructions from semantic features, yielding more interesting features.

Abstract: Sparse auto-encoders (SAEs) have become a prevalent tool for interpreting language models' inner workings. However, it is unknown how strongly SAE features correspond to computationally important directions in the model. We empirically show that many RES-JB SAE features predominantly correspond to simple input statistics. We hypothesize this is caused by a large class imbalance in training data combined with a lack of complex error signals. We propose a method to reduce this behavior by disentangling token reconstruction from feature reconstruction. We achieve this by introducing a per-token bias, which provides an improved baseline for interesting reconstruction. This change yields significantly more interesting features and improved reconstruction in sparse regimes.

Submission Number: 111