TAHIR: Transformer-Based Affine Histological Image Registration

Vladislav A. Pyatov, Dmitry V. Sorokin

Abstract: In medical practice it is often necessary to jointly analyze differently stained histological sections. However, when slides are being prepared tissues are subjected to deformations and registration is highly required. Although the transformation between images is generally non-rigid, one of the most challenging subproblems is to calculate the initial affine transformation. Existing learning-based approaches adopt convolutional architectures that are usually limited in receptive field, while global context should be considered. Coupled with small datasets and unsupervised learning paradigm, this results in overfitting and adjusting the structure only locally. We introduce transformer-based affine histological image registration (TAHIR) approach. It successfully aggregates global information, requires no histological data to learn and is based on knowledge transfer from nature domain. The experiments show that TAHIR outperforms existing methods by a large margin on most-commonly used histological image registration benchmark in terms of target registration error, being more robust at the same time. The code is available at https://github.com/VladPyatov/ImgRegWithTransformers.