Wyckoff Transformer: Generation of Symmetric Crystals

Nikita Kazeev, Ruiming Zhu, Ignat Romanov, Wei Nong, Andrey E Ustyuzhanin, Shuya Yamazaki, Kedar Hippalgaonkar

Keywords: material design, machine learning, crystal generation, space group symmetry, Transformer, Wyckoff position, generative model, autoregressive model, permutation invariance

TL;DR: A generative Transformer for crystals conditioned on space group symmetry; based on Wyckoff positions.

Abstract: We propose Wyckoff Transformer, a generative model for materials conditioned on space group symmetry. Most real--world inorganic materials have internal symmetry beyond lattice translation. Symmetry rules that atoms obey play a fundamental role in determining the physical, chemical, and electronic properties of crystals. These symmetries determine stability, and influence key material structural and functional properties such as electrical and thermal conductivity, optical and polarization behavior, and mechanical strength. And yet, despite the recent advancements, state--of--the--art diffusion models struggle to generate highly symmetric crystals. We use Wyckoff positions as the basis for an elegant, compressed, and discrete structure representation. To model the distribution we develop a permutation--invariant autoregressive model based on Transformer and absence of positional encoding. Our experiments demonstrate that Wyckoff Transformer has the best performance in generating novel diverse stable structures conditioned on the symmetry space group, while also having competitive metric values when compared to model not conditioned on symmetry. We also show that it is competitive in predicting formation energy, band gap, mechanical properties, and thermal conductivity.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 13726