On the Sample Complexity of Lipschitz Constant Estimation
Event Certifications: iclr.cc/ICLR/2024/Journal_Track
Abstract: Estimating the Lipschitz constant of a function, also known as Lipschitz learning, is a fundamental problem with broad applications in fields such as control and global optimization. In this paper, we study the Lipschitz learning problem with minimal parametric assumptions on the target function. As a first theoretical contribution, we derive novel lower bounds on the sample complexity of this problem for both noise-free and noisy settings under mild assumptions. Moreover, we propose a simple Lipschitz learning algorithm called $\textit{Lipschitz Constant Estimation by Least Squares Regression}$ (referred to as LCLS). We show that LCLS is asymptotically consistent for general noise assumptions and offers finite sample guarantees that can be translated to new upper bounds on the sample complexity of the Lipschitz learning problem. Our analysis shows that the sample complexity rates derived in this paper are optimal in both the noise-free setting and in the noisy setting when the noise is assumed to follow a Gaussian distribution and that LCLS is a sample-optimal algorithm in both cases. Finally, we show that by design, the LCLS algorithm is computationally faster than existing theoretically consistent methods, and can be readily adapted to various noise assumptions with little to no prior knowledge of the target function properties or noise distribution.
Certifications: Featured Certification
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: Some small edits for the camera-ready version:
- changed part of the title ("Lipschitz learning" to "Lipschitz constant estimation") to make it more widely understandable.
- minor edits for typos and sentence structure.
- added some explanations in the appendix and improved the formatting of equations.
Assigned Action Editor: ~Jonathan_Scarlett1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 1224
Loading