Compositional Zero-Shot Learning via Fine-Grained Dense Feature Composition

Dat Huynh, Ehsan Elhamifar

Abstract:

We develop a novel generative model for zero-shot learning to recognize finegrained unseen classes without training samples. Our observation is that generating
holistic features of unseen classes fails to capture every attribute needed to distinguish small differences among classes. We propose a feature composition
framework that learns to extract attribute-based features from training samples
and combines them to construct fine-grained features for unseen classes. Feature
composition allows us to not only selectively compose features of unseen classes
from only relevant training samples, but also obtain diversity among composed
features via changing samples used for composition. In addition, instead of building a global feature of an unseen class, we use all attribute-based features to form
a dense representation consisting of fine-grained attribute details. To recognize
unseen classes, we propose a novel training scheme that uses a discriminative
model to construct features that are subsequently used to train itself. Therefore,
we directly train the discriminative model on composed features without learning
separate generative models. We conduct experiments on four popular datasets
of DeepFashion, AWA2, CUB, and SUN, showing that our method significantly
improves the state of the art.