Concept-based Steering of Large Language Models for Conditional Molecular Generation

NeurIPS 2025 Workshop AI4Mat Submission37 Authors

Keywords: activation engineering, concept bottleneck, conditional molecular generation

TL;DR: Activation Engineering is an efficient, flexible and scalable approach for improved conditional molecular generation.

Abstract: Modern LLMs, with their internet-scale pretraining and advanced human-level capabilities across specialized tasks, have demonstrated promising performance in molecular discovery using existing text-based molecular representations, such as SMILES and SELFIES. However, generating valid, unique, and high-fidelity molecules while precisely controlling for multiple properties simultaneously remains challenging. While prior works demonstrated success by fine-tuning language models on a novel corpus of molecules with property-conditioned tags, real-world applications require generating molecules from diverse property distributions, previously unseen in the training data. To this end, we present Concept-based Activation STeering (CAST), the first approach to apply activation steering to directly edit a model's internal representation for conditional molecular generation. CAST offers a lightweight, flexible alternative to fine-tuning by computing property-conditioned steering vectors via a concept network that does not require retraining the LLM. Through extensive experiments on datasets such as Therapeutics Data Commons, we show that CAST consistently outperforms existing methods on both in-distribution and out-of-distribution conditional generation tasks. We also conduct comprehensive ablation studies to highlight the extent of control our concept-guided steering provides on the molecules generated by the LLM.

Submission Track: Paper Track (Full Paper)

Submission Category: AI-Guided Design

AI4Mat RLSF: Yes

Submission Number: 37