A PAC-Bayesian Generalization Bound for Equivariant NetworksDownload PDF

Published: 31 Oct 2022, Last Modified: 24 Oct 2022NeurIPS 2022 AcceptReaders: Everyone
Keywords: generalization error, equivariant networks, group representation, PAC Bayesian
TL;DR: We provide generalization bounds for equivariant networks in terms of group representations chosen for each layer.
Abstract: Equivariant networks capture the inductive bias about the symmetry of the learning task by building those symmetries into the model. In this paper, we study how equivariance relates to generalization error utilizing PAC Bayesian analysis for equivariant networks, where the transformation laws of feature spaces are deter- mined by group representations. By using perturbation analysis of equivariant networks in Fourier domain for each layer, we derive norm-based PAC-Bayesian generalization bounds. The bound characterizes the impact of group size, and multiplicity and degree of irreducible representations on the generalization error and thereby provide a guideline for selecting them. In general, the bound indicates that using larger group size in the model improves the generalization error substantiated by extensive numerical experiments.
Supplementary Material: pdf
17 Replies