Low-Resolution-Only Microscopy Super-Resolution Models Generalizing to Non-Periodicities at Atomic Scale

Björn Möller, Zhengyang Li, Markus Etzkorn, Tim Fingscheidt

Abstract: Super-resolution (SR) methods can accelerate microscopy image capturing and improve quality. Yet, training data is often scarce and low in variability, leading to overfitting models that fail to preserve unseen image structures. Therefore, in this work, we investigate SR model generalization in a low-resource domain, here: material science. First, we propose a training pipeline based on PixMix augmentation for microscopy SR using low-resolution only training data to generate pseudo LR/HR training pairs. The augmentation introduces variability into training images by blending them with high-detail out-of-domain images. Second, using scanning transmission electron microscopy (STEM) images, we show that our proposed training pipeline improves the SR model generalization for non-periodic high-resolution test data of crystalline atomic structures, even if only periodic low-resolution data is used for training. Furthermore, our proposed pipeline enables STEM SR models to generalize to images with noise characteristics from an unseen recording session. Third, we investigate effects of mixing augmentation strength. Finally, we validate the usage of PixMix on a more comprehensive STEM dataset. Our results demonstrate that frequent image mixing utilizing high-detail out-of-domain data improves SR generalization within low-resource domains such as atomic-scale STEM images of non-periodic matter. Data and code is available1.