Feature disentanglement, selection, and reaggregation method for multi-task learning

Renyuan Liu, Xuejie Zhang, Jin Wang, Xiaobing Zhou

Abstract: Multi-task learning utilizes the dependencies among tasks to make the tasks boost each other and achieve better results. However, there are not only dependencies but also conflicts among tasks. Task conflict is a critical issue that needs to be addressed to avoid mutual interference among tasks. Previous studies focus on weighting each task to intervene between two or more tasks to reach a compromise. The issue of dependency and conflict among tasks is more complex, where there may be both dependencies and conflicts between two tasks. Simply adjusting weights cannot effectively handle such complex relationships and achieve better results. This paper analyzes the reasons for the task conflicts and discovers that different requirements exist for features in different tasks. For a single task, the features required for other tasks may interfere with the decoder. A new method named feature disentanglement, selection, and reaggregation method is proposed based on the reason for task conflict, which is to disentangle encoder output to obtain high-level features and then select and aggregate high-level features according to the requirements of the task. Experiments show that our method achieves state-of-the-art results on the Multi-Domain Sentiment dataset and 20 Newsgroups dataset. The results prove that our method effectively alleviates conflicts among tasks.

External IDs:dblp:journals/kais/LiuZWZ25