Finding agreement in disagreement: Simultaneous label alignment and multi-dataset training with SLAMDUNKS

Petra Bevandić, Riza Velioglu, Barbara Hammer

Keywords: computer vision, multi-dataset training, image classification, aligning taxonomies

TL;DR: This paper considers automated relation discovery between classes in different datasets as a standalone task and proposes an architecture that simultaneously discovers those relations and trains the model using this information.

Abstract: Multi-dataset training is a key strategy for improving the versatility and robustness of deep models, but its effectiveness is often hindered by unaligned and contradictory dataset taxonomies. These inconsistencies introduce training noise and prevent effective knowledge sharing. To address this, we propose SLAMDUNKS, a framework for simultaneous multi-dataset training and label alignment. Its core is a shared feature extractor trained with two competing heads: a gating head that determines which dataset-specific classes should be shared, and a classification head that maps samples to the emerging shared taxonomy. To rigorously evaluate alignment quality, we introduce a synthetic benchmark where ground-truth relations are modeled as bipartite graphs. Our method demonstrates remarkable precision, perfectly recovering the true taxonomy (a Graph Edit Distance of 0) for same-domain datasets. Across more challenging cross-domain pairs, SLAMDUNKS achieves an Average Precision of 0.8, outperforming the state-of-the-art by 0.1 to 0.2 and validating its superior alignment capabilities.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 24664