Multi-modal medical Transformers: A meta-analysis for medical image segmentation in oncology

Gustavo Andrade-Miranda, Vincent Jaouen, Olena Tankyevych, Catherine Cheze Le Rest, Dimitris Visvikis, Pierre-Henri Conze

Abstract: Highlights•A meta-analysis of multi-modal medical image segmentation in oncology is provided.•A unified framework describing various Transformer-based architectures is introduced.•Variations in performance are explored between one-path versus multi-path encoders.•The impact of multiple stream interactions is studied for multi-modal Transformers.•Assessment comprises various ranking schemes, cost indicators and significance tests.