Nonparametric Bayesian inference of item-level features in classifier combination

Patrick Stinson, Nikolaus Kriegeskorte

Keywords: Bayesian nonparametrics, classifier combination, crowdsourcing

Abstract: In classification tasks, examples belonging to the same class can often still differ substantially from one another, and being able to capture such heterogeneity and its impact on classification can be important for aggregating estimates across multiple classifiers. Bayesian models developed so far have relied on a fixed set of latent variables to model these causal factors, which not only introduces the need for model selection but also assumes that each item is governed by the same set of causal factors. We develop a Bayesian model that can infer generic item features by modeling item feature membership as distributed according to an Indian Buffet Process. Despite its flexibility, our model is scalable to a large number of classifiers and examples. We compare our method with models from item response theory and Bayesian classifier combination on black-box crowdsourcing tasks and with neural network instance-dependent models in white-box classifier combination tasks.

Readers: auai.org/UAI/2025/Conference, auai.org/UAI/2025/Conference/Area_Chairs, auai.org/UAI/2025/Conference/Reviewers, auai.org/UAI/2025/Conference/Submission361/Authors, auai.org/UAI/2025/Conference/Submission361/Reproducibility_Reviewers

Submission Number: 361