Searching for Robustness: Loss Learning for Noisy Classification Tasks

Boyan Gao, Henry Gouk, Timothy Hospedales

Keywords: meta-learning, loss function learning

Abstract: We present a ``learning to learn'' approach for automatically constructing white-box classification loss functions that are robust to label noise in the training data. We paramaterize a flexible family of loss functions using Taylor polynomials, and apply evolutionary strategies to search for noise-robust losses in this space. To learn re-usable loss functions that can apply to new tasks, our fitness function scores their performance in aggregate across a range of training datasets and architecture combinations. The resulting white-box loss provides a simple and fast ``plug-and-play'' module that enables effective noise-robust learning in diverse downstream tasks, without requiring a special training procedure or network architecture.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=j0WIzA8hsU