Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness ObjectiveDownload PDF

Published: 16 May 2023, Last Modified: 05 Sept 2023AutoML 2023 WorkshopReaders: Everyone
Keywords: fairness, personalization, imbalance, bilevel optimization
Abstract: Modern classification problems exhibit heterogeneities across individual classes: Each class may have unique attributes, such as sample size, label quality, or predictability (easy vs difficult), and variable importance at test-time. Without care, these heterogeneities impede the learning process, most notably, when optimizing fairness objectives. We propose an effective and general method to personalize the optimization strategy of individual classes so that optimization better adapts to heterogeneities. Concretely, class-attribute priors (CAP) is a meta-strategy which proposes a class-specific strategy based on its attributes. This meta approach leads to substantial improvements over naive approach of assigning separate hyperparameters for each class. We instantiate CAP for loss function design and posthoc logit adjustment, with an emphasis on label-imbalanced problems. We show that CAP is competitive with prior art and its flexibility unlocks noticeable improvements for fairness objectives beyond balanced accuracy. Finally, we evaluate CAP on problems with label noise as well as weighted test objectives to showcase how CAP can synergistically leverage different class attributes.
Submission Checklist: Yes
Broader Impact Statement: Yes
Paper Availability And License: Yes
Code Of Conduct: Yes
Reviewers: Yes
CPU Hours: 0
GPU Hours: 0
TPU Hours: 0
Code And Dataset Supplement: zip
16 Replies