Alpha-DAG: a reinforcement learning based algorithm to learn Directed Acyclic Graphs

Fan Zhou, Yifeng Pan, Shenghua Zhu, Xin HE

Keywords: Directed acyclic graph, reinforcement learning, Q Learning, Graph Auto-Encoder

Abstract: Directed acyclic graphs (DAGs) are widely used to model the casual relationships among random variables in many disciplines. One major class of algorithms for DAGs is called `search-and-score', which attempts to maximize some goodness-of-fit measure and returns a DAG with the best score. However, most existing methods highly rely on their model assumptions and cannot be applied to the more general real-world problems. This paper proposes a novel Reinforcement-Learning-based searching algorithm, Alpha-DAG, which gradually finds the optimal order to add edges by learning from the historical searching trajectories. At each decision window, the agent adds the edge with the largest scoring improvement to the current graph. The advantage of Alpha-DAG is supported by the numerical comparison against some state-of-the-art competitors in both synthetic and real examples.

One-sentence Summary: This paper proposes a novel Reinforcement-Learning-based searching algorithm Alpha-DAG which gradually finds the optimal direction to add edges by learning from the historical searching trajectories.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=sDEUMkMghD