Adaptive Passive-Aggressive Framework for Online Regression with Side Information
Keywords: passive-aggressive, online learning, adaptive method, online regression problem, side information, financial engineering
TL;DR: We introduce an adaptive Passive-Aggressive (PA) framework that leverages side information for optimal threshold selection, ensuring low tracking error and improved performance in online regression tasks.
Abstract: The Passive-Aggressive (PA) method is widely used in online regression problems for handling large-scale streaming data, typically updating model parameters in a passive-aggressive manner based on whether the error exceeds a predefined threshold. However, this approach struggles with determining optimal thresholds and adapting to complex scenarios with side information, where tracking accuracy is not the sole metric in the regression model. To address these challenges, we introduce a novel adaptive framework that allows finer adjustments to the weight vector in PA using side information. This framework adaptively selects the threshold parameter in PA, theoretically ensuring convergence to the optimal setting. Additionally, we present an efficient implementation of our algorithm that significantly reduces computational complexity. Numerical experiments show that our model achieves outstanding performance associated with the side information while maintaining low tracking error, demonstrating marked improvements over traditional PA methods across various scenarios.
Primary Area: Online learning
Submission Number: 5007
Loading