Auto-Lambda: Disentangling Dynamic Task Relationships
Abstract: Understanding the structure of multiple related tasks allows for multi-task learning to improve the generalisation ability of one or all of them. However, it usually requires training each pairwise combination of tasks together in order to capture task relationships, at an extremely high computational cost. In this work, we learn task relationships via an automated weighting framework, named Auto-Lambda. Unlike previous methods where task relationships are assumed to be fixed, i.e., task should either be trained together or not trained together, Auto-Lambda explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training. We apply the proposed framework to both multi-task and auxiliary learning problems in computer vision and robotics, and show that AutoLambda achieves state-of-the-art performance, even when compared to optimisation strategies designed specifically for each problem and data domain. Finally, we observe that Auto-Lambda can discover interesting learning behaviors, leading to new insights in multi-task learning. Code is available at https://github.com/lorenmt/auto-lambda.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Updated for camera-ready version.
Code: https://github.com/lorenmt/auto-lambda
Assigned Action Editor: ~Tongliang_Liu1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 27
Loading