Temporal Probabilistic Asymmetric Multi-task Learning

Nguyen Anh Tuan, Hyewon Jeong, Eunho Yang, Sungju Hwang

TL;DR: We proposed a novel probabilistic asymmetric multi-task learning framework that allows asymmetric knowledge transfer between tasks and across time-steps, based on the uncertainty

Abstract: When performing multi-task predictions with time-series data, knowledge learned for one task at a specific time step may be useful in learning for another task at a later time step (e.g. prediction of sepsis may be useful for prediction of mortality for risk prediction at intensive care units). To capture such dynamically changing asymmetric relationships between tasks and long-range temporal dependencies in time-series data, we propose a novel temporal asymmetric multi-task learning model, which learns to combine features from other tasks at diverse timesteps for the prediction of each task. One crucial challenge here is deciding on the direction and the amount of knowledge transfer, since loss-based knowledge transfer Lee et al. (2016; 2017) does not apply in our case where we do not have loss at each timestep. We propose to tackle this challenge by proposing a novel uncertainty- based probabilistic knowledge transfer mechanism, such that we perform knowledge transfer from more certain tasks with lower variance to uncertain ones with higher variance. We validate our Temporal Probabilistic Asymmetric Multi-task Learning (TP-AMTL) model on two clinical risk prediction tasks against recent deep learning models for time-series analysis, which our model significantly outperforms by successfully preventing negative transfer. Further qualitative analysis of our model by clinicians suggests that the learned knowledge transfer graphs are helpful in analyzing the model’s predictions.

Code: https://www.dropbox.com/sh/vmfv7kvd5h0rwbu/AAAye8ybP9PCPb-3RozMPEjQa?dl=0

Keywords: Multi-task learning, Time-series analysis, Variational Inference