An Accuracy Guaranteed Online Solver for Learning in Dynamic Feature Space

Diyang Li, Bin Gu

Keywords: Online Learning, Differential Equations, Group Lasso

Abstract: We study the problem of adding or deleting features of data from machine learning models trained using empirical risk minimization. Our focus is on algorithms in an online manner which is capable for a more general regularization term, and present practical guides to two classical regularizers, i.e., the group Lasso and $\ell_p$-norm regularizer. Across a variety of benchmark datasets, our algorithm improves upon the runtime of prior methods while maintaining the *same* generalization accuracy.