Augmenting X-ray Astronomical Representations with Scientific Knowledge through Contrastive Learning

Juan Rafael Martínez-Galarza, Nicolò Oreste Pinciroli Vago, Shivam Raval, Carolina Cuesta-Lazaro, Melanie Weber, David Alvarez-Melis, Alberto Accomazzi, Cecilia Garraffo, Joshua Knutson, Ryan Thill, Christopher B. Green, Imantha Ahangama

Track: tiny / short paper (up to 5 pages)

Domain: machine learning

Abstract: Astronomers have produced large multimodal datasets that include images, spectra, and time series, and that encode physical information about the observed objects. In addition, a large amount of physics-specific knowledge about these objects has been accumulated in the astronomical literature. We introduce a physics-informed representation alignment framework that matches X-ray observations of astrophysical objects and text summaries describing the physical properties of those sources. We perform contrastive learning between data representations learned using a Poisson process autodecoder and text summary representations generated with a Large Language Model. We demonstrate the generalization capabilities of the system and evaluate the performance of the post-alignment shared representations for regression tasks. We also present a use case for the physical interpretation of newly observed astrophysical sources.

Submission Number: 26