Truthful Reporting of Competence with Minimal Verification

Reshef Meir, Jonathan Wagner, Omer Ben-Porat

Keywords: Principal-agent; Scoring rules

TL;DR: We propose truthful mechanisms for eliciting agents' competence, with minimal verification effort.

Abstract:

Suppose you run a home exam, where students should report their own score but can cheat freely. You can, if needed, call a limited amount of students to class and verify their actual performance against their reported score.

We consider the class of mechanisms where truthful reporting is a dominant strategy, and truthful agents are never penalized---even off-equilibrium.

How many students do we need to verify, in expectation, if we want to minimize the bias, i.e. the difference between agents' competence and their expected grade?

When perfect verification is available, we characterize the best possible tradeoff between these requirements, and provide a simple parametrized mechanism that is optimal in the class, for any distribution of agents' types.

When verification is noisy, the task becomes much more challenging. We show how proper scoring rules can be leveraged in different ways to construct truthful mechanisms with good (though not necessarily optimal) tradeoff.

Area: Game Theory and Economic Paradigms (GTEP)

Generative A I: I acknowledge that I have read and will follow this policy.

Submission Number: 366