GeoBIS: Budget-Optimal Block Importance Sampling for Stochastic Riemannian Optimization

ICLR 2026 Conference Submission18995 Authors

Keywords: Geometry, RiemannianManifold, StochasticOptimization, Sampling

Abstract: This paper studies budgeted block subsampling for stochastic Riemannian optimization. Starting from the Horvitz–Thompson estimator, we derive an independent Bernoulli design with a water-filling probability rule that minimizes the second moment under a fixed expected number of active blocks. The resulting estimator, GeoBIS, is unbiased and achieves the canonical inverse-in-budget behavior of its second moment. We also analyze exact-$K$ negatively dependent designs, including projection determinantal point processes and sampling without replacement with unequal probabilities. Under a mild alignment condition on block directions, exact-$K$ strictly reduces the cross term in the variance. A simple wall-clock model provides a closed-form rule for selecting the active-block budget and clarifies when exact-$K$ is worthwhile. Experiments on orthogonality-constrained sequence models and thin-Stiefel adapters follow the predicted trends and validate GeoBIS as a practical default.

Primary Area: optimization

Submission Number: 18995