The Internal Growth Function: A More General PAC Framework for Scenario Decision Making

Guillaume O Berger, Raphael Jungers

Abstract:

This paper introduces a new PAC framework for scenario decision-making problems.
Scenario decision making consists in making a decision that satisfies a probabilistic constraint (also called a chance constraint) from finitely many sampled realizations (called scenarios) of the constraint.
PAC bounds are sufficient conditions on the number of samples to guarantee with high confidence that the sample-based decision satisfies the true constraint with a prescribed probability.
Existing PAC bounds rely on intrinsic properties of the problem, such as convexity (Calafiore and Campi, 2005), finite VC dimension (Alamo et al., 2009) or existence of a compression scheme (Margellos et al., 2014).
While powerful in some applications, these PAC bounds can be vacuous (or infinite) when the properties are not satisfied.
In this paper, we propose a new PAC framework, leading to PAC bounds that are not vacuous for a strictly larger class of scenario decision-making problems.
This bound is based on the novel notion of ``internal growth'', which adapts the notion of ``growth function'' from classical machine learning (Vapnik and Chervonenkis, 1968) to scenario decision making.
We also relate this notion to other novel properties of the system, such as the $k$-VC dimension.
Furthermore, we show a partial converse result: namely, that for the family of stable monotone scenario decision algorithms, the algorithm is PAC if \emph{and only if} it satisfies our criterion.
Finally, we demonstrate the usefulness of our framework, and compare with existing approaches, on practical problems.

Certifications: J2C Certification

Submission Type: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

- Added footnotes 1, 2, and 3;
- Added details on Examples 3, 4, and 7;
- Minor correction in Theorem 1: additional assumption "(i) $\epsilon N\geq8$";
- Added details on the use of Chebyshev inequality in the proof of Lemma 2;
- Added concrete bounds on $\tau(m)$ in Propositions 2 and 3;
- Added Remark 3 to highlight the polynomiality of $\tau(m)$ with existing criteria;
- Added Remark 6 to provide additional intuition on Corollary 3;
- Added Section 7.3 to add another example of application;
- Added section "Acknowledgments".

Assigned Action Editor: ~Jie_Shen6

Submission Number: 5994