Go to ICLR 2025 Workshop LMRL homepageReframing LLM Finetuning through Bayesian Optimization Lenses
Chemical reaction optimization remains a critical bottleneck in drug discovery and materials science. While reaction procedures are naturally documented as text in research papers and protocols, converting these descriptions into structured features for machine learning poses significant challenges. We present a novel framework that leverages LLMs to directly process textual reaction descriptions, combined with deep kernel learning to accelerate optimization. Our approach adapts LLM embeddings through joint optimization with Gaussian processes, enabling dynamic reorganization of the latent space to reflect reaction performance. Unlike previous methods using static LLM embeddings, our approach induces a natural metric learning effect through the GP marginal likelihood, clustering successful reaction conditions while separating unsuccessful ones. We demonstrate that this embedding adaptation emerges independently of the initial LLM, suggesting broad applicability across different foundation models. Empirical evaluation on Buchwald-Hartwig reactions shows our method reduces the number of experiments needed to identify optimal conditions by 45% compared to static embeddings, while maintaining well-calibrated uncertainty estimates.