Crystals with Transformers on Graphs, for predictions of crystal material properties

Hongyi Wang, Ji Sun, Jinzhe Liang

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: AI for science, Graph networks, transformers, materials informatics, crystal materials

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose CrysToGraph, a transformer-based graph network for capturing short-range and long-range dependencies in the crystalline system, achieving state-of-the-art results on MatBench benchmark datasets.

Abstract: Graph neural networks (GNN) has found extensive applications across diverse domains, notably in the modeling molecules. Crystals differ from molecules by the ionic bonding across the lattice and the highly ordered microscopic structure, which provides crystals unique symmetry and determines the macroscopic properties. Therefore, long-range orders are essential in predicting the physical and chemical properties of crystals. GNNs successfully model the local environment of atoms in crystals, however, they struggle to capture long-range interactions due to a limitation of depth. In this paper, we propose CrysToGraph ($\textbf{Crys}$tals with $\textbf{T}$ransformers $\textbf{o}$n $\textbf{Graph}$s), a novel transformer-based geometric graph network designed specifically for crystalline systems. CrysToGraph effectively captures short-range dependencies with transformer-based graph convolution blocks and long-range dependencies with graph-wise transformer blocks. Our model outperforms most existing methods by achieving new state-of-the-art results on the MatBench benchmark datasets.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7122