Data Enrichment: Multi-task Learning in High Dimension with Theoretical GuaranteesDownload PDF


16 May 2019 (modified: 13 Jun 2019)ICML 2019 Workshop AMTL Blind SubmissionReaders: Everyone
  • Keywords: multi-task learning, high-dimensional estimation, sample complexity, non-asymptotic error bound
  • Abstract: Given samples from a group of related regression tasks, a data-enriched model describes observations by a common and per-group individual parameters. In high-dimensional regime, each parameter has its own structure such as sparsity or group sparsity. In this paper, we consider the general form of data enrichment where data comes in a fixed but arbitrary number of tasks $G$ and any convex function, e.g., norm, can characterize the structure of both common and individual parameters. We propose an estimator for the high-dimensional data enriched model and investigate its statistical properties. We delineate the sample complexity of our estimator and provide high probability non-asymptotic bound for estimation error of all parameters under a condition weaker than the state-of-the-art. We propose an iterative estimation algorithm with a geometric convergence rate. Overall, we present a first through statistical and computational analysis of inference in the data enriched model.
  • TL;DR: We provide an estimator and an estimation algorithm for a class of multi-task regression problem and provide statistical and computational analysis..
0 Replies