Consistent View Alignment Improves Foundation Models for 3D Medical Image Segmentation

Mei Vaish, Felix Meister, Tobias Heimann, Christoph Brune, Jelmer M. Wolterink

Keywords: Self-supervised learning; Representation learning; Contrastive methods; Consistency regularization; Feature alignment; 3D medical imaging; Robust visual representations

TL;DR: We propose Consistent View Alignment, a self-supervised framework that aligns only shared regions between views and combines local consistency with global contrastive learning to improve 3D MRI representations for segmentation and classification.

Abstract: Many recent approaches in representation learning implicitly assume that uncorrelated views of a data point are sufficient to learn meaningful representations for various downstream tasks. In this work, we challenge this assumption and demonstrate that meaningful structure in the latent space does not emerge naturally. Instead, it must be explicitly induced. We propose a method that aligns representations from different views of the data to align complementary information without inducing false positives. Our experiments show that our proposed self-supervised learning method, \textit{Consistent View Alignment}, improves performance for downstream tasks, highlighting the critical role of structured view alignment in learning effective representations. The code and pretrained model weights are released at https://github.com/Tenbatsu24/LatentCampus.

Submission Number: 39