Data Enrichment: Multi-task Learning in High Dimension with Theoretical Guarantees

Anonymous

Keywords: multi-task learning, high-dimensional estimation, sample complexity, non-asymptotic error bound

Abstract: Given samples from a group of related regression tasks, a data-enriched model describes observations by a common and per-group individual parameters. In high-dimensional regime, each parameter has its own structure such as sparsity or group sparsity. In this paper, we consider the general form of data enrichment where data comes in a fixed but arbitrary number of tasks $G$ and any convex function, e.g., norm, can characterize the structure of both common and individual parameters. We propose an estimator for the high-dimensional data enriched model and investigate its statistical properties. We delineate the sample complexity of our estimator and provide high probability non-asymptotic bound for estimation error of all parameters under a condition weaker than the state-of-the-art. We propose an iterative estimation algorithm with a geometric convergence rate. Overall, we present a first through statistical and computational analysis of inference in the data enriched model.

TL;DR: We provide an estimator and an estimation algorithm for a class of multi-task regression problem and provide statistical and computational analysis..