Deep Multimodal Representation Learning for Stellar Spectra

Tobias Buck, Christian Schwarz

Abstract:

Recently, contrastive learning (CL), a technique most prominently used in natural
language and computer vision, has been used to train informative representation
spaces for galaxy spectra and images in a self-supervised manner. Following this
idea, we implement CL for stars in the Milky Way, for which recent astronomical
surveys have produced a huge amount of heterogeneous data. Specifically, we
investigate Gaia XP coefficients and RVS spectra. Thus, the methods presented in
this work lay the foundation for aggregating the knowledge implicitly contained
in the multimodal data to enable downstream tasks like cross-modal generation or
fused stellar parameter estimation. We find that CL results in a highly structured
representation space that exhibits explicit physical meaning. Using this representa-
tion space to perform cross-modal generation and stellar label regression results in
excellent performance with high-quality generated samples as well as accurate and
precise label predictions.