Self-Supervised Generalisation with Meta Auxiliary Learning

Shikun Liu, Edward Johns, Andrew Davison

Abstract: Auxiliary learning has been shown to improve the generalisation performance of a principal task. But typically, this requires manually-defined auxiliary tasks based on domain knowledge. In this paper, we consider that it may be possible to automatically learn these auxiliary tasks to best suit the principal task, towards optimum auxiliary tasks without any human knowledge. We propose a novel method, Meta Auxiliary Learning (MAXL), which we design for the task of image classification, where the auxiliary task is hierarchical sub-class image classification. The role of the meta learner is to determine sub-class target labels to train a multi-task evaluator, such that these labels improve the generalisation performance on the principal task. Experiments on three different CIFAR datasets show that MAXL outperforms baseline auxiliary learning methods, and is competitive even with a method which uses human-defined sub-class hierarchies. MAXL is self-supervised and general, and therefore offers a promising new direction towards automated generalisation.

Keywords: meta learning, auxiliary learning, multi-task learning, self-supervised learning

TL;DR: We propose Meta AuXiliary Learning (MAXL), a learning framework which can automatically generate auxiliary tasks to improve generalisation of the principal task in a self-supervised manner.

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:1901.08933/code)