Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data

Aoran Wang, Tsz Pan Tong, Andrzej Mizera, Jun Pang

Keywords: Structural Inference, Benchmark, AI4Science, Interaction Graphs, Dynamical Systems

TL;DR: We benchmarked 13 structural inference methods on our new dataset containing various simulations and a biological dataset, evaluating accuracy, scalability, robustness, and sensitivity.

Abstract: Understanding complex dynamical systems begins with identifying their topological structures, which expose the organization of the systems. This requires robust structural inference methods that can deduce structure from observed behavior. However, existing methods are often domain-specific and lack a standardized, objective comparison framework. We address this gap by benchmarking 13 structural inference methods from various disciplines on simulations representing two types of dynamics and 11 interaction graph models, supplemented by a biological experimental dataset to mirror real-world application. We evaluated the methods for accuracy, scalability, robustness, and sensitivity to graph properties. Our findings indicate that deep learning methods excel with multi-dimensional data, while classical statistics and information theory based approaches are notably accurate and robust. Additionally, performance correlates positively with the graph's average shortest path length. This benchmark should aid researchers in selecting suitable methods for their specific needs and stimulate further methodological innovation.

Submission Number: 1295