Near-Optimal Linear Regression under Distribution Shift

Qi Lei, Wei Hu, Jason D. Lee

Keywords: minimax estimator, covariate shift, model shift

Abstract: Transfer learning is an essential technique when sufficient data comes from the source domain, while no or scarce labeled data is from the target domain. We develop estimators that achieve minimax linear risk for linear regression problems under the distribution shift. Our algorithms cover different kinds of settings with covariate shift or model shift. We also consider when data are generating from either linear or general nonlinear models. We show that affine minimax rules are within an absolute constant of the minimax risk even among nonlinear rules, for a variety of source/target distributions.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: We derive near optimal estimators for linear regression under distribution shift.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=RVC3VzuUaK