Multi-Task Learning as Stratified Variational Inequalities

ICLR 2026 Conference Submission12653 Authors

Keywords: Multi-Task Learning, Recommendation Systems, Priority-Aware Optimization

Abstract:

Multi-task learning (MTL) provides a powerful paradigm for jointly optimizing multiple objectives, yet real-world tasks often differ in maturity, difficulty, and importance. Naively training all tasks simultaneously risks premature updates from unstable objectives and interference with high-priority goals. We introduce SCD-VIO---Stratified Constraint Descent via Variational Inequalities and Operators---a new operator-theoretic paradigm for hierarchy-aware MTL. Rather than heuristic reweighting, SCD-VIO formulates training as a stratified variational inequality, where task feasibility is defined relative to its own cumulative performance and enforced through Yosida-regularized soft projections. This self-calibrated gating (SC Gate) requires no extra hyperparameters and ensures that lower-priority tasks are activated only after higher-priority ones have stabilized, aligning optimization flow with natural task dependencies.

SCD-VIO is model-agnostic and integrates seamlessly with standard MTL backbones. Experiments on three large-scale recommendation benchmarks---TikTok, QK-Video, and KuaiRand1k---show that it consistently boosts prioritized objectives while maintaining or improving overall performance. Taken together, these results position SCD-VIO as both a principled theoretical formulation and a practical, plug-and-play solution for hierarchy-aware multi-task learning.

Primary Area: optimization

Submission Number: 12653