On Constructing Confidence Region for Model Parameters in Stochastic Gradient Descent Via Batch Means

Yi Zhu, Jing Dong

Abstract: We study an easy-to-implement algorithm to construct asymptotically valid confidence regions for model parameters in stochastic gradient descent. The main idea is to cancel out the covariance matrix which is hard/costly to estimate using the batch means method with a fixed number of batches. In developing the algorithm, we establish a process-level functional central limit theorem for Polyak-Ruppert averaging iterates. We also extend the batch means method to accommodate more general batch size specifications.