HAPPI: Hyperbolic Hierarchical Part Prototypes for Image Recognition

Hooman Vaseli, Victoria Wu, Nima Kondori, Nguyen Nhat Minh To, Andrea Fung, Ang Nan Gu, Purang Abolmaesumi

Keywords: Explainable AI, Prototype Part Networks, Hyperbolic Embedding Space

TL;DR: HAPPI organizes prototypes hierarchically in hyperbolic space, where localized generic features near the origin and broader specific features farther away enhance part-based classification while matching Euclidean approaches.

Abstract: Prototypical neural networks have gained prominence in computer vision due to their inherent interpretability, enabling decisions based on representative examples without the need for post-hoc explainability methods. However, many prototype-based models overlook the hierarchical relationships within image features, treating them independently and often resulting in suboptimal performance for tasks requiring complex structural understanding. To address this limitation, we propose HAPPI (Hierarchical And Part Prototypical Image recognition), a framework that leverages hyperbolic geometry to organize prototypes hierarchically within a Lorentzian manifold. By arranging generic features near the origin of the hyperboloid and specific features farther away, HAPPI enables the learning of generic prototypes for consistent and defining patterns and specific prototypes for fine-grained and variable details, effectively capturing hierarchical relationships in image data. Our approach is model-agnostic and can be applied to various prototypical neural networks and backbones. We evaluate HAPPI across several prototypical model baselines and datasets, demonstrating its versatility and showing that hyperbolic prototypes consistently match or outperform their Euclidean counterparts in quantitative accuracy while providing additional interpretability. Qualitative visualizations reveal that generic prototypes capture consistent, semantically important distinctions between classes. In contrast, specific prototypes capture fine-grained variations within each class, providing essential nuances for detailed classification and enhancing the model’s ability to differentiate across multiple levels of abstraction. Our code can be found at github.com/*********************.

Track: Full paper (8 pages excluding references, same as main conference requirements)

Submission Number: 11