Adaptive Stratified Active Statistical Inference

Pinaki Mohanty, Rajiv Khanna

Keywords: Stratified Sampling, Statistical Inference, ML-guided Annotation

Abstract: Active Statistical Inference (ASI) leverages machine learning predictions to guide label acquisition and improve statistical inference under limited labeling budgets, but lacks the nuanced understanding of model reliability varying across the data manifold, leading to inefficient budget allocation. To address this, we propose Adaptive Stratified Active Statistical Inference (AdaStrat-ASI), a framework that replaces ASI's global sampling rule with stratum-specific local policies, designed via a model-guided ‘scouting’ phase. We provide theoretical results showing that AdaStrat-ASI achieves strictly lower asymptotic variance compared to ASI while preserving the inferential guarantees of ASI. We verify our theoretical findings through empirical results on real-world datasets demonstrate that AdaStrat-ASI yields tighter confidence intervals than existing baselines under the same labeling budget.

Submission Number: 67