[Re] Learning Fair Graph Representations via Automated Data Augmentations
Abstract: We evaluate the reproducibility of the paper "Learning Fair Graph Representations via Automated Data Augmentations" by Ling et al. (2023). Our objective is to reproduce the three major claims that (1) fair augmentations improve fairness while retaining similar accuracy compared to other fairness methods, (2) augmenting both edges and node features performs better than augmenting only one of the two, and (3) learned augmentations reduce node-wise sensitive homophily and correlation between node features and the sensitive attribute. The authors provide an implementation of their method in PyTorch. We use and extend the given code, implementing an additional multi-run evaluation protocol with different random seeds. We further create additional baselines by disabling fairness in the model and investigating the generalizability of the method to other graph neural network (GNN) architectures and graphs with varying homophily. We partially reproduce claims (1), (2), and (3), attaining similar performance for two out of the three datasets originally used, as well as noisy results for the third dataset. Additionally, in our work, the correlation between node features and the sensitive attribute does not drop as significantly as in the original paper. On the other hand, we find that the method generalizes to other GNN structures yet does not generalize to graphs with varying homophily, failing for unbalanced homophily settings. Overall, the outcomes of the experiments indicate a lack of stability in the Graphair framework.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Alain_Durmus1
Submission Number: 2210
Loading