Causal Covariate Shift Correction using Fisher information penalty

Behraj Khan, Behroz Mirza, Tahir Syed

Keywords: covariate shift, distribution shift, hyper-parameter selection, non-stationary data

TL;DR: Our proposed method $C^{3}$ mitigates induced covariate shift caused by dataset fragmentation through natural covariate shift correction.

Abstract:

Evolving feature densities across batches of training data bias cross-validation, making model selection and assessment unreliable (Sugiyama & Kawanabe (2012)). This work takes a distributed density estimation angle to the training setting where data are temporally distributed. Causal Covariate Shift Correction (C3), accumulates knowledge about the data density of a training batch using
Fisher Information, and use to penalize the loss in all subsequent batches. The penalty improves accuracy by 12.9% over the full-dataset baseline, by 20.3% accuracy at maximum in batchwise and 5.9% at minimum in foldwise benchmarks.

Submission Number: 104