D2R2: Diffusion-based Representation with Random Distance Matching for Tabular Few-shot Learning

Ruoxue Liu, Linjiajie Fang, Wenjia Wang, Bingyi Jing

Keywords: Classification, Few-shot learning, Tabular data, Representation learning, Diffusion models, Self-supervised learning

TL;DR: We propose a novel framework for tabular few-shot learning, comprising a diffusion-based model with random distance matching for representation learning, and an instance-wise iterative prototype scheme for few-shot classification.

Abstract: Tabular data is widely utilized in a wide range of real-world applications. The challenge of few-shot learning with tabular data stands as a crucial problem in both industry and academia, due to the high cost or even impossibility of annotating additional samples. However, the inherent heterogeneity of tabular features, combined with the scarcity of labeled data, presents a significant challenge in tabular few-shot classification. In this paper, we propose a novel approach named Diffusion-based Representation with Random Distance matching (D2R2) for tabular few-shot learning. D2R2 leverages the powerful expression ability of diffusion models to extract essential semantic knowledge crucial for denoising process. This semantic knowledge proves beneficial in few-shot downstream tasks. During the training process of our designed diffusion model, we introduce a random distance matching to preserve distance information in the embeddings, thereby improving effectiveness for classification. During the classification stage, we introduce an instance-wise iterative prototype scheme to improve performance by accommodating the multimodality of embeddings and increasing clustering robustness. Our experiments reveal the significant efficacy of D2R2 across various tabular few-shot learning benchmarks, demonstrating its state-of-the-art performance in this field.

Primary Area: Deep learning architectures

Submission Number: 5140