Learning Pareto manifolds in high dimensions: How can regularization help?

Tobias Wegel, Filip Kovačević, Alexandru Tifrea, Fanny Yang

TL;DR: This work presents a multi-objective learning estimator that utilizes low-dimensional structure and unlabeled data.

Abstract:

Simultaneously addressing multiple objectives is becoming increasingly important in modern machine learning. At the same time, data is often high-dimensional and costly to label.
For a single objective such as prediction risk, conventional regularization techniques are known to improve generalization when the data exhibits low-dimensional structure like sparsity. However, it is largely unexplored how to leverage this structure in the context of multi-objective learning (MOL) with multiple competing objectives.
In this work, we discuss how the application of vanilla regularization approaches can fail, and propose a two-stage MOL framework that can successfully leverage low-dimensional structure.
We demonstrate its effectiveness experimentally for multi-distribution learning and fairness-risk trade-offs.

Submission Number: 1746