Emergent SO(3)-Invariant Molecular Representations from Multimodal Alignment

Eduardo Soares, Victor Y. Shirasuna, Emilio Vital Brazil, Dmitry Zubarev, Enzo Reis de Oliveira, Caio Rodrigues Gama, Daniel Djinishian de Briquez

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: Contrastive Learning, 3D electron density grids, SMILES, SO(3) invariance

Abstract: Learning molecular representations robust to 3D rotations typically relies on symmetry-aware architectures or extensive augmentation. Here, we show that contrastive multimodal pretraining alone can induce SO(3) invariance in molecular embeddings. We jointly train a 3D electron density encoder, based on a VQGAN, and a SMILES-based transformer encoder on 855k molecules, using CLIP-style and SigLIP objectives to align volumetric and symbolic modalities. Because SMILES embeddings are rotation-invariant, the contrastive loss implicitly enforces rotation-consistency in the 3D encoder. To assess geometric generalization, we introduce a benchmark of 1,000 molecules with five random SO(3) rotations each. Our model retrieves rotated variants with 77\% Recall@10 (vs. 9.8\% for a unimodal baseline) and organizes latent space by chemical properties, achieving functional group-wise Recall@10 above 98\% and a Davies–Bouldin index of 2.35 (vs. 34.46 baseline). Fine-tuning with rotated data reveals a trade-off between retrieval precision and pose diversity. These results demonstrate that contrastive multimodal pretraining can yield symmetry-aware molecular representations without explicit equivariant design.

Submission Number: 108