From Tabular to Time Series: Can TabPFN Handle Mixed Data? A Study on PhysioNet

Zichao Li, Bingyang Wang, Zong Ke

Keywords: foundation models, tabular data, time-series analysis, cross-modal learning, temporal attention

TL;DR: This paper adapts TabPFN for mixed tabular/time-series data via cross-modal embeddings and temporal attention, achieving SOTA accuracy (92.3% on PhysioNet) with 10.9× faster inference than hybrids.

Abstract:

This paper introduces a novel adaptation of TabPFN for mixed tabular and time-series data, addressing a critical gap in foundation models for structured data. We propose cross-modal embedding layers and temporal attention mechanisms to enable seamless integration of static and sequential features while preserving TabPFN's fast inference capabilities. Evaluated on clinical (PhysioNet), retail (M5), and trajectory (UEA) benchmarks, our method achieves 4.7-13.4\% higher accuracy than modality-specific baselines (Chronos, TabForestPFN) and hybrids (TSMixer), with 10.9× faster inference. The model demonstrates particular strength in low-data regimes (82.4\% accuracy with just 100 samples) and robustness to distribution shifts (6.2\% accuracy drop vs. 14.7\% for TSMixer %under COVID-19 disruptions
). Theoretical contributions include a synthetic pretraining protocol for mixed data and ablation studies validating our architectural choices. Results show consistent improvements across healthcare, finance, and motion analysis tasks, establishing a new state-of-the-art for unified structured data modeling.

Submission Number: 23