Sim$\rightarrow$Exp-MMNMR: A Benchmark for Simulation-to-Experiment Generalization in Multimodal NMR with Chemistry-Aware Metrics

Susanna Di Vita

Keywords: NMR, metrics, distribution shift evaluation, shift marginalized maximum mean discrepancy

Abstract: We introduce Sim$\rightarrow$Exp-MMNMR, the first benchmark designed to systematically evaluate how well machine learning models and similarity metrics generalize from simulated to experimental nuclear magnetic resonance (NMR) spectra. Unlike prior work, which primarily relies on cosine similarity and simulated data alone, our benchmark features a curated dataset of 1,849 molecules with both simulated and experimental ${}^1$H and ${}^{13}$C spectra, standardized under a common solvent and validated for peak alignment. We propose chemistry-aware similarity metrics, including Shift-Marginalized Maximum Mean Discrepancy (SM-MMD), which explicitly account for peak shift uncertainty and calibration noise. Through a suite of four evaluation tasks-including matching, shift robustness, SMILES-to-spectra prediction, and candidate ranking—we show that traditional metrics often misrepresent performance under realistic conditions. Our results demonstrate that SM-MMD offers stronger robustness and structure-awareness, particularly in ${}^{13}$C spectra, suggesting it as a more suitable metric for real-world NMR applications involving domain shift.

Submission Track: Benchmarking in AI for Materials Design - Short Paper

Submission Category: Automated Synthesis

AI4Mat Journal Track: Yes

AI4Mat RLSF: Yes

Submission Number: 154