Modeling Human Concepts with Subspaces in Deep Vision Models

Anna Bavaresco, Nhut Truong, Uri Hasson

Abstract:

Improving the modeling of human representations of everyday semantic categories, such as animals or
food, can lead to better alignment between AI systems and humans. Humans are thought to represent such
categories using dimensions that capture relevant variance, in this way defining the relationship between
category members. In AI systems, the representational space for a category is defined by the distances between
its members. Importantly, in this context, the same features are used for distance computations across all
categories. In two experiments, we show that pruning a model’s feature space to better align with human
representations of a category selects for different model features and different subspaces fordifferentcategories.
In addition, we provide a proof of concept demonstrating the relevance of these findings for evaluating the
quality of images generated by AI systems.