ProFit: Unsupervised Fine-Tuning of Tabular Models via Proxy Tasks for Label-Scarce Anomaly Detection

ICLR 2026 Conference Submission407 Authors

Keywords: Anomaly Detection, Unsupervised Tabular Learning

Abstract: Anomaly detection in tabular data is crucial for applications such as fraud prevention and risk control, yet it remains challenging due to heterogeneous features, class imbalance, and limited labeled anomalies. Although pretrained tabular in context learning (TICL) models reduce label dependence, the inductive biases they develop on synthetic tasks are often misaligned with the actual data distributions encountered in downstream scenarios. Effective adaptation to new domains is thus difficult when labels are scarce. We propose **ProFiT**, an unsupervised fine-tuning framework that leverages only unlabeled target-domain data to adjust pretrained tabular models. ProFiT constructs a variety of proxy tasks by sampling different features as targets and using correlated features as inputs, encouraging the model to capture the underlying structure of the new data. To improve training effectiveness, we introduce a consistency regularizer that aligns the predictions from two different proxy views using Jensen–Shannon divergence. Experiments on tabular anomaly detection benchmarks show that ProFiT outperforms weakly-supervised and unsupervised methods, as well as vanilla TICL models. ProFiT offers a practical way to improve tabular anomaly detection under limited labeled data conditions and vast amounts of unlabeled data.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 407