LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems

Or Bachar, Or Levi, Sardhendu Mishra, Adi Levi, Justin Miller, Manpreet Singh Minhas, Omer Ben-Porat, Eilon Sheetrit, Jonathan Morra

Keywords: Large Language Models, Uncertainty Quantification, Human-AI Collaboration, Content Moderation

TL;DR: We propose a meta-model that learns to predict when LLM outputs can be trusted, enabling cost-aware selective human-AI content moderation.

Abstract:

As LLMs are increasingly integrated into human-in-the-loop content moderation systems, a central challenge is deciding when their outputs can be trusted versus when escalation for human review is preferable. We propose a novel framework for supervised LLM uncertainty quantification, learning a dedicated meta-model based on LLM Performance Predictors (LPPs) derived from LLM outputs: log-probabilities, entropy, and novel uncertainty attribution indicators.
We demonstrate that our method enables cost-aware selective classification in real-world human-AI workflows: escalating high-risk cases while automating the rest. Experiments across state-of-the-art LLMs, including both off-the-shelf (Gemini, GPT) and open-source (Llama, Qwen) on multimodal and multilingual moderation tasks show significant improvements over existing uncertainty estimators in accuracy-cost trade-offs. Beyond uncertainty estimation, the LPPs enhance explainability by providing new insights into failure conditions (e.g., ambiguous content vs. under-specified policy). This work establishes a principled framework for uncertainty-aware, scalable, and responsible human-AI moderation workflows.

Area: Learning and Adaptation (LEARN)

Generative A I: I acknowledge that I have read and will follow this policy.

Submission Number: 1241