RRGMambaFormer: A hybrid Transformer-Mamba architecture for radiology report generation

Hongzhao Li, Siwei Liu, Hui Wang, Xiaoheng Jiang, Mingyuan Jiu, Li Chen, Yang Lu, Shupan Li, Mingliang Xu

Abstract: Highlights•Proposed RRGMambaFormer, a radiology report generation model with the fewest known parameters, achieving state-of-the-art performance with reduced computational complexity.•Introduced a preprocessing Mamba Block that leverages long-range dependencies and global semantic information, enhancing adaptability to complex, sequential medical data without relying on computationally expensive positional encoding.•Developed a novel decoder integrating visual and textual information, combining Mamba’s long-range dependency modeling with the efficiency of Transformer architecture to improve report accuracy and contextual richness.•Designed the Multi-Granularity Contextual Memory Block, ensuring semantic coherence between hierarchical visual features and generated text, balancing fine-grained details and global textual context.•Achieved 30% parameter reduction and 41% faster inference time compared to multi-layer Transformer models, demonstrating superior performance on standard radiology datasets.