Embracing Ambiguity: Bayesian Nonparametrics and Stakeholder Participation for Ambiguity‑Aware Safety Evaluation

Yanan Long

Track: Main track

Published Or Accepted: false

Keywords: Bayesian Nonparametrics, Prior Elicitation, Participatory Methods

Abstract: Safety evaluations of generative models often collapse nuanced behaviour into a single number computed for a single decoding configuration. Such \emph{point estimates} obscure tail risks, demographic disparities, and the existence of multiple near‑optimal operating points—phenomena collectively known as the Rashomon or predictive multiplicity effect. We propose a unified framework that embraces multiplicity by modelling the distribution of harmful behaviour across the entire space of decoding knobs and prompts, quantifying risk through tail‑focused metrics, and integrating stakeholder preferences. Our technical contributions are threefold: (i) we formalise \emph{decoding Rashomon sets}—regions of knob space whose risk is near‑optimal under given criteria—and measure their size and disagreement; (ii) we develop a dependent Dirichlet process mixture with stakeholder‑conditioned, prompt‑aware stick‑breaking weights to learn multi‑modal harm surfaces; and (iii) we introduce an active sampling and calibration pipeline that uses Bayesian deep learning surrogates and conformal wrappers to explore knob space efficiently while maintaining finite‑sample coverage guarantees. The framework supports simulated stakeholder participation: synthetic stakeholders draw prompts from a topic mixture anchored to real datasets and rate outputs according to demographic‑specific sensitivities. We demonstrate on synthetic and real LLM evaluations that our method reveals hidden failure modes, quantifies disagreement across stakeholders, and identifies safe operating regions that single‑point evaluations miss. Our approach bridges multiplicity theory, Bayesian nonparametrics, uncertainty quantification, and participatory AI, paving the way for trustworthy deployment of generative models.

Submission Number: 18