Bounding Generalization Error Through Bias and Capacity

Ramya Ramalingam, Nicolas A. Espinosa Dice, Megan L. Kaye, George D. Montañez

Abstract: We derive generalization bounds on learning algorithms through algorithm capacity and a vector representation of inductive bias. Leveraging the algorithmic search framework, a formalism for casting machine learning as a type of search, we present a unified interpretation of the upper bounds of generalization error in terms of a vector representation of bias and the mutual information between the hypothesis and the dataset.