Multi-model evaluation with labeled and unlabeled data
Keywords: model evaluation, unlabeled data
Abstract: It remains difficult to select a machine learning model in the absence of a large, labeled dataset. To address this challenge, we propose a framework to compare multiple models that makes use of both labeled \emph{and} unlabeled data. The key idea is to estimate the joint distribution of model predictions using a mixture model, where each component corresponds to a different class. The framework exploits three aspects of modern machine learning settings: multiple machine learning models, continuous predictions on all examples, and abundant unlabeled data. We present preliminary experiments on a large health dataset and conclude with future directions.
Primary Subject Area: Data governance frameworks for ML
Paper Type: Extended abstracts: up to 2 pages
DMLR For Good Track: Participate in DMLR for Good Track
Participation Mode: In-person
Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.
Submission Number: 18
Loading