Intervention-based Recurrent Casual Model for Non-stationary Video Causal Discovery

Yuke Li, Kenneth Li, Pin Wang, Donglai Wei, Hanspeter Pfister, Ching-Yao Chan

Abstract: Non-stationary casual structures are prevalent in real-world physical systems. For example, the stacked blocks interacted with one another until they fall apart, while the billiard balls are moving independently until they collide. However, most video causal discovery methods can not discover such non-stationary casual structures due to the lack of modeling for the instantaneous change and the dynamics of the casual structure. In this work, we propose the Intervention-based Recurrent Casual Model (IRCM) for non-stationary video casual discovery. First, we extend the existing intervention-based casual discovery framework for videos to formulate the instantaneous change of the casual structure in a principled manner. Then, we use a recurrent model to sequentially predict the causal structure model based on previous observations to capture the non-stationary dynamic of the casual structure. We evaluate our method on two popular physical system simulation datasets with various types of multi-body interactions. Experimental results show that the proposed IRCM achieves the state-of-the-art performance on both the counterfactual reasoning and future forecasting tasks.