Functional Virtual Adversarial Training for Semi-Supervised Time Series Classification

Qingyi Pan, Yicheng Li

Keywords: Semi-Supervised classification; Virtual Adversarial Training; Function Space; Sobolev Space

TL;DR: We propose functional Virtual Adversarial Training (f-VAT) that incorporates the functional structure of data into perturbations.

Abstract: Real-world time series analysis, such as healthcare, autonomous driving, and solar energy, faces unique challenges arising from the scarcity of labeled data, highlighting the need for effective semi-supervised learning methods. While the Virtual Adversarial Training (VAT) method has shown promising performance in leveraging unlabeled data for smoother predictive distributions, straightforward extensions of VAT often fall short on time series tasks as they neglect the temporal structure of the data in the adversarial perturbation. In this paper, we propose the framework of functional Virtual Adversarial Training (f-VAT) that can incorporate the functional structure of the data into perturbations. By theoretically establishing a duality between the perturbation norm and the functional model sensitivity, we propose to use an appropriate Sobolev ($H^{-s}$) norm to generate structured functional adversarial perturbations for semi-supervised time series classification. Our proposed f-VAT method outperforms recent methods and achieves superior performance in extensive semi-supervised time series classification tasks (e.g., up to $ \approx 9$ % performance improvement). We also provide additional visualization studies to offer further insights into the superiority of f-VAT.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 8630