Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budget

Baihan Lin, Djallel Bouneffouf

Keywords: Contextual Bandit, Reinforcement Learning, Epidemic Intervention, Public Health, Policy Making

TL;DR: We solve the epidemic control problem as a combinatorial contextual bandit problem to balance the disease spread and the resource stringency.

Abstract: In light of the COVID-19 pandemic, it is an open challenge and critical practical problem to find a optimal way to dynamically prescribe the best policies that balance both the governmental resources and epidemic control in different countries and regions. To solve this multi-dimensional tradeoff of exploitation and exploration, we formulate this technical challenge as a contextual combinatorial bandit problem that jointly optimizes a multi-criteria reward function. Given the historical daily cases in a region and the past intervention plans in place, the agent should generate useful intervention plans that policy makers can implement in real time to minimizing both the number of daily COVID-19 cases and the stringency of the recommended interventions. We prove this concept with simulations of multiple realistic policy making scenarios and demonstrate a clear advantage in providing a pareto optimal solution in the epidemic intervention problem.