Neural Multivariate Regression with Multi-Task Learning and Target Preprocessing

George Andriopoulos, Soyuj Jung Basnet, Juan Guevara, Li Guo, Bimarsha Adhikari, Keith W. Ross

Keywords: neural multivariate regression, multi-task learning, target preprocessing, unconstrained feature model, spectral properties

TL;DR: We derive and test qualitative predictions for multi-task learning and target preprocessing in neural multivariate regression using the Unconstrained Feature Model.

Abstract: The Unconstrained Feature Model (UFM) enables closed-form approximations for training loss in deep neural networks (DNNs). We use the UFM to motivate testable hypotheses about neural multivariate regression—fundamental to imitation learning, robotics, and reinforcement learning. Specifically, we analyze multi-task versus single-task models and the impact of target preprocessing. The UFM predicts advantages for multi-task models under comparable or stronger regularization and identifies regimes where whitening or normalization reduces training error. Experiments across four robotic and autonomous driving datasets consistently support these qualitative trends. This work illustrates how simplified analytical models can structure empirical investigation by generating theoretical predictions that are then validated through practice.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Style Files: I have used the style files.

Submission Number: 89