Make it SING: Analyzing Semantic Invariants in Classifiers

Harel Yadid, Meir Yossef Levi, Roy Betser, Guy Gilboa

Keywords: XAI, Interpretability, Neural-networks, vision-classifiers, mechanistic-interpretability, CLIP

TL;DR: A new method to quantify and visualize invariants hidden inside vision classifier

Abstract: All classifiers, including state-of-the-art vision models, possess invariants, partially rooted in the geometry of their linear mappings. These invariants, which reside in the null-space of the classifier, induce equivalent sets of inputs that map to identical outputs. The semantic content of these invariants remains vague, as existing approaches struggle to provide human-interpretable information. To address this gap, we present Semantic Interpretation of the Null-space Geometry (SING), a method that constructs equivalent images, with respect to the network, and assigns semantic interpretations to the available variations. We use a mapping from network features to multi-modal vision language models. This allows us to obtain natural language descriptions and visual examples of the induced semantic shifts. SING can be applied to a single image, uncovering local invariants, or to sets of images, allowing a breadth of statistical analysis at the class and model levels. For example, our method reveals that ResNet50 leaks relevant semantic attributes to the null space, whereas DINO-ViT, a ViT pretrained with self-supervised DINO, is superior in maintaining class semantics across the invariant space.

Primary Area: interpretability and explainable AI

Submission Number: 6988