Concatenated Tensor Networks for Deep Multi-Task Learning

Maolin Wang, Zeyong Su, Xu Luo, Yu Pan, Shenggen Zheng, Zenglin Xu

Abstract: Deep Multi-Task Learning has achieved great success in a number of domains. However, the enormous number of parameters results in extremely large storage costs for current deep Multi-Task models. Several methods based on tensor networks were proposed to address this problem. However, the tensor train format based methods only share the information of one mode. The huge central core tensor of the tucker format is hard to be stored and optimized. To tackle these problems, we introduce a novel Concatenated Tensor Network structure, in particular, Projected Entangled Pair States (PEPS) like, into multi-task models. We name the resulted multi-task models as Concatenated Tensor Multi-Task Learning (CT-MTL).