Centralized vs Individual Models for Decision Making in Interconnected Infrastructure

Stephanie Allen, John P Dickerson, Steven A. Gabriel

Keywords: markov Decision processes, reinforcement learning, interdependent infrastructure

TL;DR: We model disaster relief for interconnected infrastructure using three models that represent various points on the coordination spectrum.

Abstract: The 2013 National Infrastructure Protection Plan outlines the need for interconnected infrastructure systems to coordinate more and recognize their interdependencies. We model the two extremes of this coordination spectrum using two different multi-agent models: (a) a model called the centralized model in which the agents are fully centralized and act as one unit in making decisions and (b) a model called the individual model in which the agents act completely separately and have either a pessimistic or optimistic assumption regarding the damages of the other infrastructure systems controlled by the other agents. We then use the individual model to establish a point along the coordination spectrum by providing the individual agents with delayed information regarding the other player(s). To test this framework, we use a small but illustrative model from a 2020 paper in which there is a power and a water network, and we assume that there are operators for both networks that would like to maximize flow according to a specific metric. Our results comparing partially repaired networks using the two models find that (i) the centralized model acts as an upper bound upon the individual model in terms of our flow metric and (ii) the delayed information individual model leads to less variability in results compared to the other individual model assumptions which points to the value of at least delayed coordination in decision making.

Track: Original Research Track