Sequential Least-Squares Estimators with Fast Randomized Sketching for Linear Statistical Models

Guanyu Chen, Xi Yang

Keywords: Iteration Method, Large Scale, Least-Squares Optimization, Linear Statistical Model, Randomized Sketching

Abstract:

We propose a novel randomized framework for the estimation problem of large-scale linear statistical
models, namely Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS),
which integrates Sketch-and-Solve and Iterative-Sketching methods for the first time. By iteratively
constructing and solving sketched least-squares (LS) subproblems with increasing sketch sizes to
achieve better precisions, SLSE-FRS gradually refines the estimators of the true parameter vector,
ultimately producing high-precision estimators. We analyze the convergence properties of SLSE-FRS,
and provide its efficient implementation. Numerical experiments show that SLSE-FRS outperforms
the state-of-the-art methods, namely the Preconditioned Conjugate Gradient (PCG) method, and the
Iterative Double Sketching (IDS) method.

Primary Area: optimization

Submission Number: 8713