Comparing Apples and Oranges: is Stitching Similarity a Load of Spheres?

Damian Smith, Antonia Marcu

TL;DR: We construct a series of experiments challenging the idea that model stitching can identify similar (or better) representations of information.

Abstract: Model stitching is used in the literature to assess the extent to which models capture similar information. The intuition is that if two models classify samples in the same way, they must be capturing the same information. We construct a series of experiments to show that two models can make the same predictions but represent very different information. We therefore argue that unlike previously claimed, stitching cannot reflect the extent to which models represent or capture similar information. This paper draws the community’s attention to the need to correctly interpret the results of such functional similarity measures and highlights the need for similarity measures that capture informational similarity.

Style Files: I have used the style files.

Submission Number: 24