Measuring Sparse Autoencoder Feature Sensitivity

Claire Tian, Katherine Tian, Nathan Zixia Hu

Open Source Links: https://github.com/nathanhu0/sae-sensitivity

Keywords: Sparse Autoencoders, Automated interpretability, Benchmarking interpretability

TL;DR: Automated evaluation for SAE feature sensitivity.

Abstract:

Sparse Autoencoder (SAE) features have become essential tools for mechanistic interpretability research. SAE features are typically characterized by examining their activating examples, which are often ``monosemantic" and align with human interpretable concepts. However, these examples don't reveal *feature sensitivity*: how reliably a feature activates on texts similar to its activating examples. In this work, we develop a scalable method to evaluate feature sensitivity.
Our approach avoids the need to generate natural language descriptions for features; instead we use language models to generate text with the same semantic properties as a
feature’s activating examples. We then test whether the feature activates on these generated texts.
We demonstrate that sensitivity measures a new facet of feature quality and find that many interpretable features have poor sensitivity. Human evaluation confirms that when features fail to activate on our generated text, that text genuinely resembles the original activating examples. Lastly, we study feature sensitivity at the SAE level and observe that average feature sensitivity declines with increasing SAE width across 7 SAE variants. Our work establishes feature sensitivity as a new dimension for evaluating both individual features and SAE architectures.

Submission Number: 227