Learning Identifiable Causal Structures with Pairwise Representation

Jiaru Zhang, Rui Ding, Qiang Fu, Yang Hua, Haibing Guan, Shi Han, Dongmei Zhang

Primary Area: causal reasoning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: causality, causal discovery, deep learning, supervised causal learning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: A DNN-based supervised causal learning approach to learn identifiable causal relations with pairwise representation

Abstract:

Supervised Causal Learning (SCL) aims to obtain causal relations from observational data, leveraging the model learned from prior datasets with ground truth causal relations.
Deep Neural Network (DNN) based SCL, which learns DNNs as causal models, has gained significant attention with its numerous advantages.
A recently proposed transformer-based architecture employs sample-wise and node-wise attention mechanisms to capture representations of individual variables.
In the inference stage, the trained model takes the test data as input and outputs a Directed Acyclic Graph (DAG) represented as a weighted adjacency matrix.

However, this paper identifies two limitations of these approaches.
First, using the adjacency matrix as a learning target can yield inconsistent results, w.r.t. structure identifiability if Bernoulli sampling is further adopted to generate the DAG.
Second, current network architecture does not adequately encode the essential causal information for learning causal structures.
To address these issues, we propose a novel DNN-based SCL approach, PAIRE, which incorporates a unique pairwise encoder module with a unidirectional attention layer.
By taking both node features and pairwise features as layer input, it can model the internal and external relationships of variable pairs.
In addition, we use a skeleton matrix along with a v-tensor, a third-order tensor representing v-structures, as our output, so as to represent the Markov Equivalence Class (MEC), which resolves identifiability inconsistency.
Empirical evidence indicates PAIRE significantly outperforms other DNN-based SCL approaches.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 704