AuxiLight: Fast, Lightweight Auxiliary Loss Balancing Algorithm with Application in 6D Pose Estimation
Abstract: Jointly learning multiple tasks has proven to be beneficial. Auxiliary task learning builds on this by jointly training a set of predefined auxiliary tasks to improve the performance of a desired main task. Unfortunately, auxiliary task learning is often not practical because 1) jointly training multiple tasks requires an exponential hyperparameter search space of task loss weights; 2) the auxiliary tasks often lead to expensive annotation costs to obtain ground truth. In this work, we propose \textit{AuxiLight}, a generic auxiliary learning algorithm, and consider the concrete real-world use case of 6D pose estimation. AuxiLight addresses the first issue via an algorithm that adaptively balances the auxiliary task losses based on an analysis of the training dynamics. Experiments on standard multi-task datasets show that our method consistently outperforms single-task models and state-of-the-art auxiliary task learning methods, being the fastest and the most lightweight among the known task-weighting algorithms. To demonstrate the practicality of auxiliary learning on real-world tasks, we further apply our method to 6D object pose estimation. We highlight that, for this task, multiple ground-truth auxiliary annotations can in fact be generated for free. This lets us showcase a concrete use of auxiliary learning for real-world problems that does induce annotation costs.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Kui_Jia1
Submission Number: 3248
Loading