Rethinking Masked Autoencoders for Multi-Channel Fluorescence Microscopy: Adaptive Inter-Channel Masking

Daeun Kim, Hyesong Choi, Hyejin Park, Dongbo Min

Keywords: self-supervised learning, masked autoencoder, multi-channel image, representation learning, bioimaging

Abstract: Masked Autoencoder (MAE) have demonstrated strong transfer learning performance in self-supervised pretraining but face challenges when directly applied to multi-channel image data, as they do not fully capture both channel-specificity and cross-channel interactions. To address this, we propose Adaptive Inter-Channel Masking (AIM), a novel masking strategy designed for MAE-based pretraining in multi-channel settings. AIM integrates Channel-Specific Saliency Masking (CSSM) to enhance channel-specific representation learning and Channel Exclusive Masking (CEM) to promote cross-channel information exchange, achieving a balanced representation across channels. We further introduce two new metrics—Channel Specificity Index (CSI) and Cross-Channel Interaction Index (CCI)—to quantitatively assess channel disentanglement and inter-channel interaction. Experimental results on the JumpCP dataset show that AIM improves downstream performance by 2.5–5.1% over existing masking strategies and by 19.3% compared to training from scratch, demonstrating its effectiveness for multi-channel image pretraining.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 10727