Influence Learning in Complex Systems
Abstract: High sample complexity hampers successful applications of reinforcement learning methods especially in real-world scenarios whose complex dynamics are typically computationally demanding to simulate. One idea is to decompose a large factored problem into small local subproblems including only few state variables and model the influence that the external portion of the system exerts on each of them. This principled approach allows to convert the global simulator of the entire environment into local lightweight simulators, thus enabling faster simulations, planning and solutions. The ability to represent accurately the influence experienced by each local component is crucial for the effectiveness of this method. In this work, we examine different aspects of the problem of learning approximations of the influence in realistic domains. We empirically investigate several learning methods to conclude that even for large and complex systems, in practice, the influence problem often turns into a relatively manageable learning task. Finally, we discuss how to leverage effectively the influence models for long horizon tasks for planning or reinforcement learning problems. Our results show that in many cases short horizon trajectories collected from the global simulator can be used to obtain accurate approximations of the influence for much longer horizons.
Submission Length: Long submission (more than 12 pages of main content)
Assigned Action Editor: ~Pascal_Poupart2
Submission Number: 1140
Loading