Factorizer: A scalable interpretable approach to context modeling for medical image segmentation

Pooya Ashtari, Diana Maria Sima, Lieven De Lathauwer, Dominique Sappey-Marinier, Frederik Maes, Sabine Van Huffel

Abstract: Highlights•The first end-to-end deep model with matrix factorization layers for medical image segmentation•A differentiable NMF layer constructed using a block coordinate descent solver to efficiently model contextual information•Shifted Window (SW) Matricize operation introduced and combined with NMF to fully exploit local contexts•Scalable interpretable U-shaped segmentation models based on NMF•State-of-the-art results on the BraTS and ISLES’22 datasets