Reference-based MRI Reconstruction Using Texture Transformer

Pengfei Guo, Vishal M. Patel

Keywords: MRI Reconstruction, Deep Learning, Transformer

TL;DR: We propose a novel Texture Transformer Module for the reference-based MR image reconstruction.

Abstract: Deep Learning (DL) based methods for magnetic resonance (MR) image reconstruction have been shown to produce superior performance. However, previous methods either only leverage under-sampled data or require a paired fully-sampled auxiliary MR sequence to perform guidance-based reconstruction. Consequently, existing approaches neglect to explore attention mechanisms that can transfer texture from reference data to under-sampled data within a single MR sequence, which either limits the performance of these approaches or increases the difficulty of data acquisition. In this paper, we propose a novel $\textbf{T}$exture $\textbf{T}$ransformer $\textbf{M}$odule ($\textbf{TTM}$) for the reference-based MR image reconstruction. The TTM facilitates joint feature learning across under-sampled and reference data, so feature correspondences can be discovered by attention and accurate texture features can be leveraged during reconstruction. Notably, TTM can be stacked on prior MRI reconstruction methods to improve their performance. In addition, a $\textbf{R}$ecurrent $\textbf{T}$ransformer $\textbf{R}$econstruction backbone ($\textbf{RTR}$) is proposed to further improve the performance in a unified framework. Extensive experiments demonstrate the effectiveness of TTM and show that RTR can achieve the state-of-the-art results on multiple datasets. Implementation code and pre-trained weights will be made public after the review process.