MTL-UE: Learning to Learn Nothing for Multi-Task Learning

Yi Yu, Song Xia, SIYUAN YANG, Chenqi Kong, Wenhan Yang, Shijian Lu, Yap-Peng Tan, Alex Kot

TL;DR: We propose a framework for generating unlearnable examples on both multi-task learning and single-task learning models applied to multi-task learning datasets.

Abstract: Most existing unlearnable strategies focus on preventing unauthorized users from training single-task learning (STL) models with personal data. Nevertheless, the paradigm has recently shifted towards multi-task data and multi-task learning (MTL), targeting generalist and foundation models that can handle multiple tasks simultaneously. Despite their growing importance, MTL data and models have been largely neglected while pursuing unlearnable strategies. This paper presents MTL-UE, the first unified framework for generating unlearnable examples for multi-task data and MTL models. Instead of optimizing perturbations for each sample, we design a generator-based structure that introduces label priors and class-wise feature embeddings which leads to much better attacking performance. In addition, MTL-UE incorporates intra-task and inter-task embedding regularization to increase inter-class separation and suppress intra-class variance which enhances the attack robustness greatly. Furthermore, MTL-UE is versatile with good supports for dense prediction tasks in MTL. It is also plug-and-play allowing integrating existing surrogate-dependent unlearnable methods with little adaptation. Extensive experiments show that MTL-UE achieves superior attacking performance consistently across 4 MTL datasets, 3 base UE methods, 5 model backbones, and 5 MTL task-weighting strategies. Code is available at https://github.com/yuyi-sd/MTL-UE.

Lay Summary:

Modern machine learning models are often trained to handle many tasks at once — like recognizing faces, detecting diseases, or understanding scenes — using a technique called multi-task learning (MTL). However, as more personal or proprietary data is used in training, concerns have grown about data privacy and misuse.

Our research introduces MTL-UE, a method to protect multi-task data from being exploited by machine learning models. The idea is simple but powerful: we add tiny, invisible changes (called unlearnable examples) to each training image so that any model trained on this data performs poorly — effectively "learning nothing" useful. Unlike earlier methods that focused only on single tasks, MTL-UE is the first method specifically designed to work on multi-task setups, where multiple outputs are predicted from the same input.

We designed a new generator that adds these hidden changes more intelligently using shared representations across tasks, which makes the attack more effective. We also built in controls to make these changes harder to undo.
This technique is important for safeguarding sensitive datasets in fields like healthcare, surveillance, and finance, where preventing unauthorized AI training is as crucial as enabling it.

Link To Code: https://github.com/yuyi-sd/MTL-UE

Primary Area: Social Aspects->Safety

Keywords: unlearnable examples, unlearnable datasets, multi-task learning, poisoning attacks

Submission Number: 7487