Learning Pareto fronts in high dimensions: How can regularization help?

Tobias Wegel, Filip Kovačević, Alexandru Tifrea, Fanny Yang

TL;DR: This work presents a multi-objective learning estimator that utilizes low-dimensional structure and unlabeled data.

Abstract: Modern machine learning methods often have to rely on high-dimensional data that is expensive to label, while unlabeled data is abundant. When the data exhibits low-dimensional structure such as sparsity, conventional regularization techniques are known to improve generalization for a single objective (e.g., prediction risk). However, it is largely unexplored how to leverage this structure in the context of multi-objective learning (MOL) with multiple competing objectives. In this work, we discuss how the application of vanilla regularization approaches can fail, and propose the first MOL estimator that provably yields improved performance in the presence of sparsity and unlabeled data. We demonstrate its effectiveness experimentally for multi-distribution learning and fairness-risk trade-offs.

Submission Number: 1746