Crowd Evacuation Simulation Using Hierarchical Deep Reinforcement Learning

Zheng Zhang, Dianjie Lu, Jialiuyuan Li, Pingshan Liu, Guijuan Zhang

Abstract: Data-driven crowd evacuation learning methods are often used to enhance the realism of crowd simulation. However, the learning results of traditional methods cannot adapt to the dynamic changes of the simple scene, and thus have the disadvantage of poor generalization. To solve this problem, we propose a data-driven crowd evacuation framework based on hierarchical deep reinforcement learning. The framework consists of: a macro-control layer with path programming function and a micro control layer with collision avoidance function. In this paper, a path programming method combining data-driven and deep reinforcement learning is proposed in the macro-control layer. The method combines the pedestrian motion attributes in the video with the DDPG algorithm to learn the pedestrian track in the video from a macro perspective. In the micro-control layer, the track sequence learned in the macro-control layer is used as the motion target to learn the collision-free motion velocity of individuals using the multiple agent deep reinforcement learning method. When the scene changes, the micro-control layer adaptively adjusts the motion speed without the need for the macro-control layer to repeat the path programming learning. The experimental results demonstrate that the proposed hierarchical crowd evacuation framework can not only simulate the real crowd movement behavior and improve the simulation fidelity, but also flexibly adapt to the dynamic changes of the simple scene and enhance the generalization.