Shuffling Heuristic in Variational Inequalities: Establishing New Convergence Guarantees

Daniil Medyakov, Gleb Molodtsov, Evseev Grigoriy, Egor Petrov, Aleksandr Beznosikov

Keywords: stochastic optimization, variational inequalities, shuffling, variance reduction

Abstract: Variational inequalities have gained significant attention in machine learning and optimization research. While stochastic methods for solving these problems typically assume independent data sampling, we investigate an alternative approach - the shuffling heuristic which involves permuting the dataset before sequential processing and ensures equal consideration of all data points. Despite its practical utility, theoretical guarantees for shuffling in variational inequalities remain unexplored. We address this gap by providing the first theoretical convergence estimates for shuffling methods in this context. Our analysis establishes rigorous bounds and convergence rates, extending the theoretical framework for this important class of algorithms. We validate our findings through extensive experiments on diverse benchmark variational inequality problems, demonstrating faster convergence of shuffling methods compared to independent sampling approaches.

Submission Number: 23