Correcting Model misspecification via Generative Adversarial Networks

TMLR Paper1581 Authors

Abstract:

Machine learning models are often misspecified in the likelihood, which leads to a lack of robustness in the predictions. In this paper, we introduce a framework for correcting likelihood misspecifications in several paradigm agnostic noisy prior models and test the model’s
ability to remove the misspecification. The ”ABC-GAN” framework introduced is a novel generative modeling paradigm, which combines Generative Adversarial Networks (GANs) and Approximate Bayesian Computation (ABC). This new paradigm assists the existing
GANs by incorporating any subjective knowledge available about the modeling process via ABC, as a regularizer, resulting in a partially interpretable model that operates well under low data regimes. At the same time, unlike any Bayesian analysis, the explicit knowledge
need not be perfect, since the generator in the GAN can be made arbitrarily complex. ABCGAN eliminates the need for summary statistics and distance metrics as the discriminator implicitly learns them, and enables simultaneous specification of multiple generative models.
The model misspecification is simulated in our experiments by introducing noise of various biases and variances. The correction term is learnt via the ABC-GAN, with skip connections, referred to as skipGAN. The strength of the skip connection indicates the amount of
correction needed or how misspecified the prior model is. Based on a simple experimental setup, we show that the ABC-GAN models not only correct the misspecification of the prior, but also perform as well as or better than the respective priors under noisier conditions. In
this proposal, we show that ABC-GANs get the best of both worlds.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=ZnCqCA34ta

Changes Since Last Submission: The code link was not anonymized and hence the manuscript was desk-rejected. We made the code link anonymous now. This was an oversight earlier.

Assigned Action Editor: ~Stanislaw_Kamil_Jastrzebski1

Submission Number: 1581