Zero-shot Outlier Detection via Synthetically Pretrained Transformers: Model Selection Bygone!
Keywords: zero-shot outlier detection, prior-data fitted networks
Abstract: Outlier detection (OD) has a vast literature as it finds numerous applications in
environmental monitoring, security, manufacturing, and finance to name a few.
Being an inherently unsupervised task, model selection is a key bottleneck for OD
(both algorithm and hyperparameter selection) without label supervision. There is
a long list of techniques to choose from – both classical algorithms and deep neural
architectures – and while several studies report their hyperparameter sensitivity, the
literature remains quite slim on unsupervised model selection—limiting the effective use of OD in practice. In this paper we present FoMo-0D, for zero/0-shot OD
exploring a transformative new direction that bypasses the hurdle of model selection
altogether (!), thus breaking new ground. The fundamental idea behind FoMo-0D is
the Prior-data Fitted Networks, recently introduced by Müller et al. (2022), which
trains a Transformer model on a large body of synthetically generated data from a
prior data distribution. In essence, FoMo-0D is a pretrained Foundation Model
for zero/0-shot OD on tabular data, which can directly predict the (outlier/inlier)
label of any test data at inference time, by merely a single forward pass—making
obsolete the need for choosing an algorithm/architecture and tuning its associated
hyperparameters, besides requiring no training of model parameters when given a
new OD dataset. Extensive experiments on 57 public benchmark datasets against
26 baseline methods show that FoMo-0D performs statistically no different from the
2nd top baseline, while significantly outperforming the majority of the baselines,
with an average inference time of 7.7 ms per test sample.
Primary Area: foundation or frontier models, including LLMs
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4003
Loading