A Set-Sequence Model for Time Series

Elliot L Epstein, Apaar Sadhwani, Kay Giesecke

Keywords: Set modeling, Sequence modeling, High-dimensional time series, Cross-sectional dependencies, Interpretable AI, Synthetic data, Mortgage risk prediction, Exchangeability, Time series prediction, Financial AI, Deep learning

TL;DR: When numerous time series are exchangeable, the Set-Sequence model efficiently captures cross-sectional dependencies

Abstract: Many prediction problems across science and engineering, especially in finance and economics, involve large cross-sections of individual time series, where each unit (e.g., a loan, stock, or customer) is driven by unit-level features and latent cross-sectional dynamics. While sequence models have advanced per-unit temporal prediction, capturing cross-sectional effects often still relies on hand-crafted summary features. We propose Set-Sequence, a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering. At each time step, a permutation-invariant Set module summarizes the unit set; a Sequence module then models each unit’s dynamics conditioned on both its features and the learned summary. The architecture accommodates unaligned series, supports varying numbers of units at inference, integrates with standard sequence backbones (e.g., Transformers), and scales linearly in cross-sectional size. Across a synthetic contagion task and two large-scale real-world applications—equity portfolio optimization and loan risk prediction—Set-Sequence significantly outperforms strong baselines, delivering higher Sharpe ratios, improved AUCs, and interpretable cross-sectional summaries.

Primary Area: learning on time series and dynamical systems

Submission Number: 11754