nnterp: A Standardized Interface for Mechanistic Interpretability of Transformers

Clément Dumas

Open Source Links: https://github.com/Butanium/nnterp/, https://butanium.github.io/nnterp/

Keywords: Interpretability tooling and software

TL;DR: Unified transformer interface using NNsight relying on huggingface implementation rather than custom one like transformerLens

Abstract: Mechanistic interpretability research requires reliable tools for analyzing transformer internals across diverse architectures. Current approaches face a fundamental tradeoff: custom implementations like TransformerLens ensure consistent interfaces but require manual adaptation for each architecture and may diverge from original models, while direct HuggingFace access through NNsight preserves exact behavior but lacks standardization across models. We present nnterp, a lightweight wrapper around NNsight that provides a unified interface for transformer analysis while preserving original HuggingFace implementations. Through automatic module renaming and comprehensive validation testing, nnterp enables researchers to write intervention code once and deploy it across 50+ model variants spanning 16 architecture families. The library includes built-in implementations of common interpretability methods (logit lens, patchscope, activation steering) and provides direct access to attention probabilities for models that support it. By packaging validation tests with the library, researchers can verify compatibility with custom models locally. nnterp bridges the gap between correctness and usability in mechanistic interpretability tooling.

Submission Number: 323