Supervised Contrastive Block Disentanglement

Taro Makino, Ji Won Park, Natasa Tagasovska, TAKAMASA KUDO, Paula Coelho, Heming Yao, Jan-Christian Huetter, Ana Carolina Leote, Burkhard Hoeckendorf, Stephen Ra, David Richmond, Kyunghyun Cho, Aviv Regev, Romain Lopez

Track: Main track (up to 8 pages)

Abstract: Real-world datasets often combine data collected under different experimental conditions. This yields larger datasets, but also introduces spurious correlations that make it difficult to model the phenomena of interest. We address this by learning two embeddings to independently represent the phenomena of interest and the spurious correlations. The embedding representing the phenomena of interest is correlated with the target variable $y$, and is invariant to the environment variable $e$. In contrast, the embedding representing the spurious correlations is correlated with $e$. The invariance to $e$ is difficult to achieve on real-world datasets. Our primary contribution is an algorithm called Supervised Contrastive Block Disentanglement (SCBD) that effectively enforces this invariance. It is based purely on Supervised Contrastive Learning, and applies to real-world data better than existing approaches. We empirically validate SCBD on the real-world problem of batch correction. Using a dataset of 26 million Optical Pooled Screening images, we learn embeddings for \num{5050} genetic perturbations that are nearly free of technical artifacts that arise from unintended variation across wells.

Submission Number: 15