LACER: Loss-Aware Clustering for Effective Reweighting
Track: regular paper (up to 6 pages)
Keywords: spurious correlations, robustness
Abstract: Deep neural networks trained with Empirical Risk Minimization (ERM) are prone to rely on simple spurious features—features that are correlated with the target but are not causally related to it. To mitigate this over-reliance, Deep Feature Reweighting (DFR) has emerged as an efficient approach, which works by retraining the last layer of an ERM model on a small reweighting dataset. While effective, DFR requires group annotations to create the reweighting dataset, which may be challenging and costly to obtain. Though subsequent works have proposed ways to alleviate this constraint, existing methods still largely rely on group annotations for hyperparameter tuning to achieve robust performance. In this paper, we present LACER, a method that improves group robustness without requiring explicit group annotations for either training or model selection. LACER operates in two stages: first estimating group labels through a loss-weighted clustering formulation that effectively identifies clusters corresponding to underrepresented groups in the validation set, then leveraging these estimated labels for last-layer retraining. Our results provide the empirical evidence that combining semantic feature information with loss values enables effective group label estimation. We validate LACER across multiple vision spurious correlations benchmarks, demonstrating performance comparable to oracle last-layer retraining methods that utilize ground-truth group annotations.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Format: Maybe: the presenting author will attend in person, contingent on other factors that still need to be determined (e.g., visa, funding).
Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.
Presenter: ~Saksham_Rastogi2
Submission Number: 62
Loading