Latent Feature Mining with Large Language Models

Bingxuan Li, Pengyi Shi, Amy R Ward

Keywords: criminal justice; large language models; feature extraction; AI for social good;

TL;DR: A framework to augment observed features with latent features using large language model, enhancing the predictive power of ML models in downstream task.

Abstract: Predictive modeling often faces challenges due to limited data availability and quality, especially in domains where collected features are weakly correlated with outcomes and where additional data collection is constrained by ethical or practical difficulties. Traditional machine learning (ML) models struggle to incorporate unobserved yet critical factors. We propose a framework that leverages large language models (LLMs) to augment observed features with latent features, enhancing the predictive power of ML models in downstream tasks. Our novel approach transforms the latent feature mining task to a text-to-text propositional reasoning task. We validate our framework with a case study in the criminal justice system, a domain characterized by limited and ethically challenging data collection. Our results show that inferred latent features align well with ground truth labels and significantly enhance the downstream classifier. Our framework is generalizable across various domains with minimal domain-specific customization, ensuring easy transfer to other areas facing similar challenges in data availability.

Primary Area: Machine learning for social sciences

Submission Number: 19574