FreqMAE: Frequency-Aware Masked Autoencoder for Multi-Modal IoT Sensing
Keywords: IoT Sensing on the Edge, Sensing Stream and Time Series Data Representation Framework, Self-Supervised Learning, Foundation Models
TL;DR: This paper presents FreqMAE, a novel self-supervised learning framework that combines masked autoencoding with physics-informed insights, offering versatile representations for diverse sensing streams and time series applications.
Abstract: This paper introduces FreqMAE, a novel self-supervised learning framework that synergizes masked autoencoding (MAE) with
physics-informed signal insights to capture feature patterns from multi-modal IoT sensing signals. By enhancing the representation of sensor data semantics in a latent feature space, FreqMAE diminishes the dependence on data labeling and boosts the accuracy of
downstream AI tasks. Unlike paradigms relying on data augmentations, such as contrastive learning, FreqMAE’s automated approach
avoids handcrafted label-invariant transformations. Adapting MAE for IoT sensing signals, we present three contributions from frequency domain insights: First, a Temporal-Shifting Transformer (TS-T) encoder that enables temporal interactions while distinguishing different frequency regions; Second, a factorized multimodal fusion mechanism that leverages cross-modal correlations while allowing for modality-private features; Third, a hierarchically weighted loss function that prioritizes the reconstruction of important frequency components and high Signal-to-Noise Ratio (SNR) samples. Comprehensive evaluations of two sensing applications validate FreqMAE’s proficiency in reducing labeling needs and enhancing resilience against domain shifts.
Track: Systems and Infrastructure for Web, Mobile, and WoT
Submission Guidelines Scope: Yes
Submission Guidelines Blind: Yes
Submission Guidelines Format: Yes
Submission Guidelines Limit: Yes
Submission Guidelines Authorship: Yes
Student Author: Yes
Submission Number: 190
Loading