Spatio-temporal collaborative multiple-stream transformer network for liver lesion classification on multiple-sequence magnetic resonance imaging

Shuangping Huang, Zinan Hong, Bianzhe Wu, Jinglin Liang, Qinghua Huang

Abstract: Accurate identification of focal liver lesions is essential for determining the appropriate therapeutic approach in clinical practice. Magnetic resonance imaging (MRI) is a valuable technology for precise classification, revealing diverse physical and biological details of lesions. However, due to the wide variety and morphological variability of the lesions, unsystematic mixing analysis of multiple-sequences MRI may cause aliasing of lesion information, obscuring the inter-tissue relationships between various imaging sequences and impeding a comprehensive diagnosis. In this paper, we proposed a Spatio-Temporal Collaborative Multiple-Stream Transformer Network that simultaneously considers spatial contrast and temporal variations to obtain detailed information on anatomical structures and tissue dynamics, effectively organizing and utilizing multiple-sequence MRI for analysis. Specifically, multiple-sequence MRI is first grouped into multiple streams based on MRI diagnostic characteristics. To reduce the interference of redundancy across multiple streams, we design a bottleneck bridge structure for spatial information aggregation. Additionally, we adopt a bidirectional Long Short-Term Memory to simulate radiologists observing the vascular morphology and hemodynamics in lesion sites from contrast-enhanced sequences. Experiments conducted on public MRI dataset, which includes seven categories of focal liver lesions from 498 patients, demonstrate that our framework achieves state-of-the-art performance, with an accuracy of 85.6%, a precision of 87.4%, a recall of 84.2%, an F1-score of 85.3%, and an Area Under the Curve of 97.1%. The experimental results indicate that the network performs well in predicting focal liver lesions, advancing the application of precision medicine.