Multi-teacher Invariance Distillation for Domain-Generalized Action Recognition

Jinwoo Choi, Jongmin Shin, Abhishek Maiti, Yuliang Zou

Abstract:

n this work, we tackle the problem of domain-generalized action
recognition, i.e. we train a model on a source domain and then test the model
on other unseen target domains with different data distributions. Generalizing
across different domains often requires distinct representational invariances and
variances, which makes domain generalization even more challenging. However,
existing methods overlook the nuanced requirements of representational invari-
ances/variances across different domains. To this end, we propose Multi-teacher
Invariance Distillation for domain-generalized Action Recognition (MIDAR), a
method to learn multiple representational invariances/variances tailored to the
unique characteristics of diverse domains. MIDAR comprises two key learning
stages. First, we learn multiple teacher models to specialize in distinct represen-
tational invariances/variances. Then, we distill the knowledge of teachers to a
student model through the adaptive reweighting (ARW) layer, which determines
the ratio of supervision from different teachers. We validate the proposed method
on public benchmarks. The proposed method shows favorable performance com-
pared to the existing methods across multiple domains on public benchmarks.