Minimax-Optimal Aggregation for Density Ratio Estimation

ICLR 2026 Conference Submission18087 Authors

Keywords: Density Ratio Estimation, Self-Concordance, Kernel Methods, Domain Adaptation

TL;DR: An aggregation algorithm for minimax-optimal hyperparameter optimization in density ratio estimation.

Abstract: Density ratio estimation (DRE) is fundamental in machine learning and statistics, with applications in domain adaptation and two-sample testing. However, DRE methods are highly sensitive to hyperparameter selection, with suboptimal choices often resulting in poor convergence rates and empirical performance. To address this issue, we propose a novel model aggregation algorithm for DRE that trains multiple models with different hyperparameter settings and aggregates them. Our aggregation provably achieves minimax-optimal error convergence without requiring prior knowledge of the smoothness of the unknown density ratio. Our method surpasses cross-validation-based model selection and model averaging baselines for DRE on standard benchmarks for DRE and large-scale domain adaptation tasks, setting a new state of the art on image and text data.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 18087