AutoMat: Enabling Automated Crystal Structure Reconstruction from Microscopy via Agentic Tool Use

Yaotian Yang, Yiwen Tang, Yizhe Chen, Xiao Chen, Jiangjie Qiu, Hao Xiong, Zhiyao Luo, Haoyu Yin, Yifei Zhang, Sijia Tao, Wentao Li, Qinghua Zhang, Yuqiang Li, Wanli Ouyang, Bin Zhao, Xiaonan Wang, Fei Wei

Keywords: AI for Science, Agent-Based System, Scanning Transmission Electron Microscopy, Crystal Structure Reconstruction and property prediction

Abstract: Machine learning models for interatomic potentials and force fields require high-quality structural data, yet experimental crystal structures remain limited, creating a critical gap between computational simulations and real structures. While atomic-resolution electron microscopy can provide valuable images, converting these into simulation-ready structures is time-consuming and error-prone. We present **AutoMat**, an agent-based framework that directly converts Scanning Transmission Electron Microscopy (STEM) images to atomistic crystal structures for property prediction. The framework adaptively calls tools, including pattern-adaptive denoising, physics-guided template retrieval, symmetry-aware atomic reconstruction, and property prediction, enabling closed-loop reasoning from "image → structure → property." For systematic evaluation, we introduce **STEM2Mat-Bench**, a benchmark dataset containing 450+ annotated samples. Performance is assessed using lattice root-mean-square deviation (RMSD), formation energy mean absolute error (MAE), and structure matching accuracy. Results demonstrate that AutoMat outperforms existing approaches including SOTA models, specialized domain tools, and closed-source multimodal large models. This work establishes a direct pathway from microscopic characterization to atomic-scale modeling, addressing a fundamental challenge in materials science.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 11417