Multi-Task Learning for Classification with Dirichlet Process Priors

Xuejun Liao

Abstract:

Consider the problem of learning logistic-regression models for multiple classification tasks, where
the training data set for each task is not drawn from the same statistical distribution. In such a
multi-task learning (MTL) scenario, it is necessary to identify groups of similar tasks that should
be learned jointly. Relying on a Dirichlet process (DP) based statistical model to learn the extent
of similarity between classification tasks, we develop computationally efficient algorithms for two
different forms of the MTL problem. First, we consider a symmetric multi-task learning (SMTL)
situation in which classifiers for multiple tasks are learned jointly using a variational Bayesian
(VB) algorithm. Second, we consider an asymmetric multi-task learning (AMTL) formulation in
which the posterior density function from the SMTL model parameters (from previous tasks) is
used as a prior for a new task: this approach has the significant advantage of not requiring storage
and use of all previous data from prior tasks. The AMTL formulation is solved with a simple
Markov Chain Monte Carlo (MCMC) construction. Experimental results on two real life MTL
problems indicate that the proposed algorithms: (a) automatically identify subgroups of related
tasks whose training data appear to be drawn from similar distributions; and (b) are more accurate
than simpler approaches such as single-task learning, pooling of data across all tasks, and simplified
approximations to DP.