Online Learning from Trapezoidal Data Streams.

Qin Zhang, Peng Zhang, Guodong Long, Wei Ding, Chengqi Zhang, Xindong Wu

Abstract: In this paper, we study a new problem of continuous learning from doubly-streaming data where both data volume and feature space increase over time. We refer to the doubly-streaming data as trapezoidal data streams and the corresponding learning problem as online learning from trapezoidal data streams. The problem is challenging because both data volume and data dimension increase over time, and existing online learning <xref ref-type="bibr" rid="ref1"> [1]</xref> , <xref ref-type="bibr" rid="ref2">[2]</xref> , online feature selection <xref ref-type="bibr" rid="ref3">[3]</xref> , and streaming feature selection algorithms <xref ref-type="bibr" rid="ref4">[4]</xref> , <xref ref-type="bibr" rid="ref5">[5]</xref> are inapplicable. We propose a new Online Learning with Streaming Features algorithm (OL <inline-formula> <tex-math notation="LaTeX">$_{SF}$</tex-math></inline-formula> for short) and its two variants, which combine online learning <xref ref-type="bibr" rid="ref1">[1]</xref> , <xref ref-type="bibr" rid="ref2">[2]</xref> and streaming feature selection <xref ref-type="bibr" rid="ref4">[4]</xref> , <xref ref-type="bibr" rid="ref5">[5]</xref> to enable learning from trapezoidal data streams with infinite training instances and features. When a new training instance carrying new features arrives, a classifier updates the existing features by following the passive-aggressive update rule <xref ref-type="bibr" rid="ref2">[2]</xref> and updates the new features by following the structural risk minimization principle. Feature sparsity is then introduced by using the projected truncation technique. We derive performance bounds of the OL <inline-formula> <tex-math notation="LaTeX">$_{SF}$</tex-math></inline-formula> algorithm and its variants. We also conduct experiments on real-world data sets to show the performance of the proposed algorithms.