A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers

Ye He, Alireza Mousavi-Hosseini, Krishna Balasubramanian, Murat A Erdogdu

Keywords: Proximal samplers, Complexity of heavy-tailed sampling, Restricted Gaussian oracle, Restricted Stable oracle

TL;DR: Gaussian-based proximal samplers face accuracy limits sampling heavy-tailed targets. Stable-based samplers offer high-accuracy guarantees, surpassing this constraint.

Abstract: We study the complexity of heavy-tailed sampling and present a separation result in terms of obtaining high-accuracy versus low-accuracy guarantees i.e., samplers that require only $\mathcal{O}(\log(1/\varepsilon))$ versus $\Omega(\text{poly}(1/\varepsilon))$ iterations to output a sample which is $\varepsilon$-close to the target in $\chi^2$-divergence. Our results are presented for proximal samplers that are based on Gaussian versus stable oracles. We show that proximal samplers based on the Gaussian oracle have a fundamental barrier in that they necessarily achieve only low-accuracy guarantees when sampling from a class of heavy-tailed targets. In contrast, proximal samplers based on the stable oracle exhibit high-accuracy guarantees, thereby overcoming the aforementioned limitation. We also prove lower bounds for samplers under the stable oracle and show that our upper bounds cannot be fundamentally improved.

Primary Area: Learning theory

Submission Number: 11058