Pre-Trained Tabular Transformer for Real-Time, Efficient, Stable Radiomics Data Processing: A Comprehensive Study

Zekun Jiang, Ruchun Jia, Le Zhang, Kang Li

Abstract: Radiomics is an important research direction in the field of medical image analysis. Although the number of publications is increasing year by year, it has been difficult to translate into clinical practice due to the small size of clinical data. In most cases, Radiomics data can be considered as small tabular data. Deep learning is often less effective than classical machine learning algorithms in processing tabular data. Recently, table representation learning has started to receive more widespread attention which is often an easily overlooked but very important area of research, helping improve the status of tabular deep learning. Here, we first apply a pre-trained Transformer model named Tabular Prior-Data Fitted Network (TabPFN) to the field of Radiomics analysis. We implement extensive experiments on three real-world clinical datasets: (a) Ultrasound Radiomics dataset for the classificatory diagnosis of Kidney tumor, (b) CT Radiomics dataset for the prediction of EGFR gene mutations in non-small cell lung cancer, (c) MRI Radiomics dataset for the prediction of treatment response of brain metastases to gamma knife radiosurgery. By comprehensive analysis, we demonstrate that the pre-trained tabular Transformer can be used as a realtime, efficient, and stable Radiomics data processor with superior performance over other tabular machine learning methods in different clinical tasks. We also simulate an ideal clinical practice scenario for evaluating the clinical translation potential of pretrained models. Finally, we explore the advantages and limitations of pre-trained tabular models for Radiomics analysis.