Incentivized Exploration via Filtered Posterior Sampling

Yonatan Gur, Anand Kalvit, Aleksandrs Slivkins

Abstract: We consider a principal interacting sequentially with a flow of self-interested agents that each consume information, take actions, and generate new information over time. The principal's goal is to maximize the aggregate utility of all agents, which necessitates agents to occasionally acquire new information by exploratory actions that might otherwise be deemed inferior from an empirical standpoint. Such exploratory actions help discerning the best actions over time, but they are also the core of misaligned incentives between the principal and the agents. While a desirable alignment of incentives may be achieved via monetary payments to the agents, such payments are often infeasible, impractical, or unethical. The essence of the Incentivized Exploration (IE) problem is to leverage information asymmetry to incentivize agents to take exploratory actions. Online learning algorithms are a natural vehicle for studying this problem.