Reusing Samples in Variance Reduction

Yujia Jin, Ishani Karmarkar, Aaron Sidford, Jiayi Wang

Keywords: optimization theory, oracle complexity trade-offs, variance reduction, reusing randomness, finite-sum minimization, MDPs, matrix-vector games, top-eigenvector computation

TL;DR: We develop a sample reuse framework to improve the theoretical query complexity trade-off between full batch and sample queries for several structured optimization problems.

Abstract: We provide a general framework to improve trade-offs between the number of _full batch_ and _sample_ queries used to solve structured optimization problems. Our results apply to a broad class of randomized optimization algorithms that iteratively solve sub-problems to high accuracy. We show that such algorithms can be modified to _reuse the randomness_ used to query the input across sub-problems. Consequently, we improve the trade-off between the number of gradient (full batch) and individual function (sample) queries for finite sum minimization, the number of matrix-vector multiplies (full batch) and random row (sample) queries for top-eigenvector computation, and the number of matrix-vector multiplies with the transition matrix (full batch) and generative model (sample) queries for optimizing Markov Decision Processes. To facilitate our analysis we introduce the notion of _pseudo-independent algorithms_, a generalization of pseudo-deterministic algorithms (Gat and Goldwasser 2011) that quantifies how independent the output of a randomized algorithm is from a randomness source.

Submission Number: 76