FedQAPer: Query Attention Pooling for Dimension Alignment in Federated Non-IID Time-series Forecasting with Personalized Heads

Seul Lee, Kim Jonghun, Gilsoo JANG, Gwangwoo Han

Keywords: Federated learning, Time series forecasting, Dimension alignment, Personalized head

TL;DR: FedQAPer is a federated learning framework that addresses feature heterogeneity in time-series forecasting by using Query Attention Pooling to align different client feature dimensions and personalized heads to preserve local specialization.

Abstract: Federated learning (FL) has shown great promise for time-series forecasting, yet a key challenge in real-world applications is feature heterogeneity. Unlike prior work that assumes uniform feature spaces, we construct a more realistic feature-level non-independent and identically distributed (non-IID) scenario by allocating subsets of features to each client. The number of features varies from 1 up to a defined maximum. We introduce FedQAPer, a novel FL framework that combines Query Attention Pooling (QAP) with FedPer algorithm that uses personalized heads for each client to capture local patterns. QAP projects heterogeneous client feature dimensions into a unified representational space, enabling collaborative backbone training across diverse feature configurations. FedPer transforms these aligned representations back to each client’s original feature dimension through personalized heads, achieving both global knowledge integration and local specialization. FedQAPer works for various backbone architectures, including both artificial neural network (ANN) models and spiking neural network (SNN) models. Experiments on multivariate time-series benchmarks demonstrate that FedQAPer effectively handles feature heterogeneity and consistently improves forecasting performance across different backbone models.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 13061