TabText: A Flexible and Contextual Approach to Tabular Data Representation

TMLR Paper4534 Authors

Abstract: Tabular data is an essential data format for applying machine learning tasks across various industries. However, traditional data processing methods do not fully utilize all the information available in the tables, ignoring important contextual information such as column header descriptions. In addition, pre-processing data into a tabular format can remain a labor-intensive bottleneck in model development. This work introduces TabText, a processing and feature extraction framework that extracts contextual information from tabular data structures. TabText addresses processing difficulties by converting the content into language and utilizing pre-trained large language models (LLMs). We evaluate our framework on ten healthcare prediction tasks—including patient discharge, ICU admission, and mortality—and validate its generalizability on an additional task from a different domain. We show that 1) applying our TabText framework enables the generation of high-performing and simple machine learning baseline models with minimal data pre-processing, and 2) augmenting pre-processed tabular data with TabText representations improves the average and worst-case AUC performance of standard machine learning models by as much as 5% additive points. All the code to reproduce the results can be found at https://anonymous.4open.science/r/TabText-18F0.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Fixed a couple of mistakes: 19 instead of 21 tasks were performed; included definition of n, p and k in Table 2.

Assigned Action Editor: ~Kenta_Oono1

Submission Number: 4534