SAM-EM: Real-Time Segmentation for Automated Liquid Phase Transmission Electron Microscopy

Alexander Wang, Max Xu, Risha Goel, Zain Shabeeb, Isabel Panicker, Vida Jamali

Keywords: electron microscopy, vision transformer, Segment Anything Model, SAM2, LPTEM, segmentation

TL;DR: We present SAM-EM, a fine-tuned foundation model that unifies video segmentation, particle tracking, and statistical analysis, enabling quantitative characterization of low-SNR liquid-phase transmission electron microscopy videos.

Abstract: The absence of robust segmentation frameworks for noisy liquid phase transmission electron microscopy (LPTEM) videos prevents reliable extraction of particle trajectories, creating a major barrier to quantitative analysis and to connecting observed dynamics with materials characterization and design. To address this challenge, we present Segment Anything Model for Electron Microscopy (SAM-EM), a domain-adapted foundation model that unifies segmentation, tracking, and statistical analysis for LPTEM data. Built on Segment Anything Model 2 (SAM 2), SAM-EM is derived through full-model fine-tuning on 46,600 curated LPTEM synthetic video frames, substantially improving mask quality and temporal identity stability compared to zero-shot SAM~2 and existing baselines. Beyond segmentation, SAM-EM integrates particle tracking with statistical tools, including mean-squared displacement and particle displacement distribution analysis, providing an end-to-end framework for extracting and interpreting nanoscale dynamics. Crucially, full fine-tuning allows SAM-EM to remain robust under low signal-to-noise conditions, such as those caused by increased liquid sample thickness in LPTEM experiments. By establishing a reliable analysis pipeline, SAM-EM transforms LPTEM into a quantitative single-particle tracking platform and accelerates its integration into data-driven materials discovery and design. Project page: \href{https://github.com/JamaliLab/SAM-EM}{github.com/JamaliLab/SAM-EM}.

Submission Track: Paper Track (Short Paper)

Submission Category: Automated Material Characterization

Institution Location: {Atlanta, USA}, {Evanston, USA}

AI4Mat Journal Track: Yes

AI4Mat RLSF: Yes

Submission Number: 121