Multi-Task Learning by a Top-Down Control Network

Hila Levi, Shimon Ullman

Keywords: multi task learning, computer vision

Abstract:

As the range of tasks performed by a general vision system expands, executing multiple tasks accurately and efficiently in a single network has become an important and still open problem. Recent computer vision approaches address this problem by branching networks, or by a channel-wise modulation of the network feature-maps with task specific vectors. We present a novel architecture that uses a dedicated top-down control network to modify the activation of all the units in the main recognition network in a manner that depends on the selected task, image content, and spatial location. We show the effectiveness of our scheme by achieving significantly better results than alternative state-of-the-art approaches on four datasets. We further demonstrate our advantages in terms of task selectivity, scaling the number of tasks and interpretability.
Code is supplied in the supplementary materials and will be publicly available.

One-sentence Summary: We present a multi-task learning scheme, which uses a dedicated top-down control network to modify the main recognition network in a manner that makes it highly selective to the selected task, obtaining high accuracy demonstrated on several datasets.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=3J21zNPX4n