PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners

Anthony Sicilia, Katherine Atwell, Malihe Alikhani, Seong Jae Hwang

Keywords: PAC-Bayes, Domain Adaptation, Multiclass

TL;DR: We propose multiclass domain adaptation bounds, which (importantly) use non-uniform sample complexity, and apply these to analyze a common DA algorithm based on neural networks.

Abstract: Multiclass neural networks are a common tool in modern unsupervised domain adaptation, yet an appropriate theoretical description for their non-uniform sample complexity is lacking in the adaptation literature. To fill this gap, we propose the first PAC-Bayesian adaptation bounds for multiclass learners. We facilitate practical use of our bounds by also proposing the first approximation techniques for the multiclass distribution divergences we consider. For divergences dependent on a Gibbs predictor, we propose additional PAC-Bayesian adaptation bounds which remove the need for inefficient Monte-Carlo estimation. Empirically, we test the efficacy of our proposed approximation techniques as well as some novel design-concepts which we include in our bounds. Finally, we apply our bounds to analyze a common adaptation algorithm that uses neural networks.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2207.05685/code)