DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks

ICLR 2026 Conference Submission21341 Authors

Keywords: meta learning, few-shot learning, self-supervised learning, disentangled representation, fast adaptation

TL;DR: By using disentangled representations to create a diverse set of self-supervised meta-learning tasks, we can train a model to fast adapt to unseen few-shot learning tasks of distinct natures.

Abstract: Meta-learning represents a strong class of approaches for solving few-shot learning tasks. Nonetheless, recent research suggests that simply pre-training a generic encoder can potentially surpass meta-learning algorithms. In this paper, we hypothesize that the reason meta-learning fails to stand out in popular few-shot learning benchmarks is the lack of diversity among the few-shot learning tasks. We propose DRESS, a task-agnostic Disentangled REpresentation-based Self-Supervised meta-learning approach that enables fast model adaptation on highly diversified few-shot learning tasks. Specifically, DRESS utilizes disentangled representation learning to create self-supervised tasks that can fuel the meta-training process. We validate the effectiveness of DRESS through experiments on datasets with multiple factors of variation and varying complexity. The results suggest that DRESS is able to outperform competing methods on the majority of the datasets and task setups. Through this paper, we advocate for a re-examination of how task adaptation studies are conducted, and aim to reignite interest in the potential of meta-learning for solving few-shot learning tasks via disentangled representations.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 21341