Online Functional Tensor Decomposition via Continual Learning for Streaming Data Completion

Xi Zhang, Yanyi Li, Yisi Luo, Qi Xie, Deyu Meng

Keywords: data stream; tensor decomposition

TL;DR: We propose a novel online functional tensor decomposition (OFTD) framework for streaming data completion.

Abstract: Online tensor decompositions are powerful and proven techniques that address the challenges in processing high-velocity streaming tensor data, such as traffic flow and weather system. The main aim of this work is to propose a novel online functional tensor decomposition (OFTD) framework, which represents a spatial-temporal continuous function using the CP tensor decomposition parameterized by coordinate-based implicit neural representations (INRs). The INRs allow for natural characterization of continually expanded streaming data by simply adding new coordinates into the network. Particularly, our method transforms the classical online tensor decomposition algorithm into a more dynamic continual learning paradigm of updating the INR weights to fit the new data without forgetting the previous tensor knowledge. To this end, we introduce a long-tail memory replay method that adapts to the local continuity property of INR. Extensive experiments for streaming tensor completion using traffic, weather, user-item, and video data verify the effectiveness of the OFTD approach for streaming data analysis. This endeavor serves as a pivotal inspiration for future research to connect classical online tensor tools with continual learning paradigms to better explore knowledge underlying streaming tensor data.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 10565