RISE : Regression Imbalance Handling using Switching Experts

Archit Sharma, PRASENJIT DEY, Charul Paliwal, Sivaramakrishnan R Kaveri

Keywords: Imbalanced Regression, Minority labels, Fairness, Mixture of experts

Abstract:

Deep Imbalanced Regression (DIR) is challenging due to skewed label distributions and the need to preserve target continuity. Existing DIR methods rely on a single,monolithic model, yet empirical analysis shows that standard benchmarks exhibit strong distributional heterogeneity, exposing a core limitation of such approaches. We theoretically prove that this property creates an irreducible bias for any single model, leading to poor performance in data-scarce regions. This creates a core challenge for algorithmic fairness, as these regions often correspond to marginalized demographic groups. To address this, we propose RISE—Regression Imbalance handling via Switching Experts—a modular Mixture-of-Experts–inspired framework, theoretically motivated by our analysis. RISE employs a novel imbalance-aware algorithm to identify underperforming regions via validation loss and trains dedicated experts with targeted upsampling. As a complementary framework, RISE achieves new state-of-the-art performance while improving fairness, highlighting a
principled new direction for imbalanced regression

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 20156