Risk-Aware Interventions in Public Health: Planning with Restless Multi-Armed Bandits

Aditya Shrikant Mate, Andrew Perrault, Milind Tambe

Abstract:

Community Health Workers (CHWs) form an important component
of health-care systems globally, especially in low-resource settings.
CHWs are often tasked with monitoring the health of and intervening on their patient cohort. Previous work has developed several
classes of Restless Multi-Armed Bandits (RMABs) that are computationally tractable and indexable, a condition that guarantees
asymptotic optimality, for solving such health monitoring and intervention problems (HMIPs). However, existing solutions to HMIPs
fail to account for risk-sensitivity considerations of CHWs in the
planning stage and may run the danger of ignoring some patients
completely because they are deemed less valuable to intervene on.
Additionally, these also rely on patients reporting their state of adherence accurately when intervened upon. Towards tackling these
issues, our contributions in this paper are as follows: (1) We develop
an RMAB solution to HMIPs that allows for reward functions that
are monotone increasing, rather than linear, in the belief state and
also supports a wider class of observations. (2) We prove theoretical
guarantees on the asymptotic optimality of our algorithm for any
arbitrary reward function. Additionally, we show that for the specific reward function considered in previous work, our theoretical
conditions are stronger than the state-of-the-art guarantees. (3) We
show the applicability of these new results for addressing the three
issues pertaining to: risk-sensitive planning, equitable allocation
and reliance on perfect observations as highlighted above. We evaluate these techniques on both simulated as well as real data from
a prevalent CHW task of monitoring adherence of tuberculosis
patients to their prescribed medication in Mumbai, India and show
improved performance over the state-of-the-art. Full paper and code
is available at: https://github.com/AdityaMate/risk-aware-bandits