Stretch Transformation for Tabular Data

Zihao Ye, Juyong Kim, Johnna Sundberg, Burak Varıcı, Pradeep Ravikumar

Keywords: Tabular Machine learning, Preprocessing

TL;DR: We propose a "stretch" transformation framework featuring the first supervised method to make target functions smoother for numerical data, achieving SOTA performance.

Abstract: Tabular data presents unique challenges for deep learning due to its heterogeneous nature, where features exhibit diverse distributions, scales, and statistical properties. Although recent advances have achieved strong performance on tabular benchmarks, feature transformation, a critical preprocessing step, remains largely unsupervised despite the availability of target information during training. We introduce the stretch transformation framework, which formulates feature preprocessing as an optimization problem to make the target function smoother and thus more learnable. Our framework has two variants: (1) unsupervised stretch, which uniformly redistributes feature density via minimax optimization, and (2) supervised stretch, which is the first method to systematically leverage target information for numeric features by minimizing the target function's Dirichlet energy in the transformed space. Our theoretical analysis reveals fundamental connections to existing methods, as unsupervised stretch explains why empirical CDF transformation can improve learning despite being label-agnostic, and supervised stretch generalizes target encoding with principled regularization for numeric features. Comprehensive experiments on 38 datasets from the TALENT benchmark demonstrate that supervised stretch consistently outperforms all baselines. These results show that explicitly optimizing for target function smoothness is a powerful and underexplored strategy for tabular deep learning.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 14897