Adaptive policies for perimeter surveillance problems

James A. Grant, David S. Leslie, Kevin Glazebrook, Roberto Szechtman, Adam N. Letchford

Abstract: Highlights • Sequential search problem is formulated as a combinatorial multi-armed bandit. • We consider a novel setting with action-dependent filtering of Poisson rewards. • A new upper confidence bound approach based on martingale inequalities is proposed. • We derive order-optimal upper bounds on the regret of the approach. • We verify its strong performance and robustness in extensive numerical experiments. Abstract We consider the problem of sequentially choosing observation regions along a line, with an aim of maximising the detection of events of interest. Such a problem may arise when monitoring the movements of endangered or migratory species, detecting crossings of a border, policing activities at sea, and in many other settings. In each case, the key operational challenge is to learn an allocation of surveillance resources which maximises successful detection of events of interest. We present a combinatorial multi-armed bandit model with Poisson rewards and a novel filtered feedback mechanism - arising from the failure to detect certain intrusions - where reward distributions are dependent on the actions selected. Our solution method is an upper confidence bound approach and we derive upper and lower bounds on its expected performance. We prove that the gap between these bounds is of constant order, and demonstrate empirically that our approach is more reliable in simulated problems than competing algorithms.