Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data

Aoran Wang, Tsz Pan Tong, Andrzej Mizera, Jun Pang

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Structural Inference, AI4Science, Benchmark

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

In the quest to unravel the complexities of dynamical systems, the initial imperative is to unveil their inherent topological structure, a key determinant of system organization. Achieving this necessitates the deployment of robust structural inference techniques capable of deriving this structure from observed system behaviors.
However, these methods are often tailor-made for specific domains and datasets, lacking a unified and objective framework for comparative assessment.
In response to this pressing challenge, we present a comprehensive benchmarking study encompassing 12 structural inference methodologies sourced from diverse disciplines.
Our evaluation protocol spans dynamical systems generated via two distinct simulation paradigms and encompasses 11 distinct interaction graph typologies.
We gauge the methods' performance in terms of accuracy, scalability, robustness, and sensitivity to graph properties.
Key findings emerge: 1) Deep learning techniques excel in the context of multi-dimensional data, 2) classical statistics and information-theory-based methods exhibit exceptional accuracy and resilience, and 3) method performance correlates positively with the average shortest path length of the graph.
Our benchmark not only aids researchers in method selection for specific problem domains but also serves as a catalyst for inspiring novel methodological advancements in the field.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5073