Unified Transfer Learning in High-Dimensional Linear Regression

Shuo Shuo Liu

Abstract:

Transfer learning plays a key role in mod-
ern data analysis when: (1) the target data
are scarce but the source data are sufficient;
(2) the distributions of the source and target
data are heterogeneous. This paper devel-
ops an interpretable unified transfer learning
model, termed as UTrans, which can detect
both transferable variables and source data.
More specifically, we establish the estimation
error bounds and prove that our bounds are
lower than those with target data only. Be-
sides, we propose a source detection algo-
rithm based on hypothesis testing to exclude
the nontransferable data. We evaluate and
compare UTrans to the existing algorithms
in multiple experiments. It is shown that
UTrans attains much lower estimation and
prediction errors than the existing methods,
while preserving interpretability. We finally
apply it to the US intergenerational mobility
data and compare our proposed algorithms
to the classical machine learning algorithms.