Q-Learning Scheduler for Multi-Task Learning through the use of Histogram of Task Uncertainty

Kourosh Meshgi, Maryam Sadat Mirzaei, Satoshi Sekine

Keywords: Q-learning, Multi-Task Learning, MTL Scheduling, Histogram of Task Uncertainty

Abstract: Simultaneous training of a multi-task learning network on different domains or tasks is not always straightforward. It could lead to inferior performance or generalization compared to the corresponding single-task networks. To maximally taking advantage of the benefits of multi-task learning, an effective training scheduling method is deemed necessary. Traditional schedulers follow a heuristic or prefixed strategy, ignoring the relation of the tasks, their sample complexities, and the state of the emergent shared features. We proposed a deep Q-Learning Scheduler (QLS) that monitors the state of the tasks and the shared features using a novel histogram of task uncertainty, and through trial-and-error, learns an optimal policy for task scheduling. Extensive experiments on multi-domain and multi-task settings with various task difficulty profiles have been conducted, the proposed method is benchmarked against other schedulers, its superior performance has been demonstrated, and results are discussed.

One-sentence Summary: A deep Q-learning-based task scheduling method to improve multi-tasking learning based on a novel histogram of task uncertainty.