Translating Biomedical Observations into Signal Temporal Logic with LLMs using Structured Feedback

Hanna Krasowski, Lauren E. Malek, Sanjit A. Seshia, Murat Arcak

Keywords: Large Language Models, Signal Temporal Logic, Systems Biology, AI Safety

TL;DR: We propose an LLM-based approach that translates natural language statements from biomedical literature into signal temporal logic specifications, guided by semantic and syntactic feedback.

Abstract: Biomedical literature contains valuable knowledge that can be used to validate or monitor machine learning models. To leverage this knowledge for machine learning, we propose an LLM-based approach that translates natural language statements into formal Signal Temporal Logic (STL) specifications, guided by semantic and syntactic feedback. To capture temporal and logical dependencies in biomedical sentences, we design an STL grammar and apply structured syntax checking alongside embedding-based cosine similarity to ensure syntactic validity and semantic alignment. Evaluating sentences from nine biomedical publications on COVID-19, we find that our approach generates semantically correct STL specifications, with GPT-4o achieving the strongest performance. The resulting specifications can be flexibly applied to monitor model outputs or incorporated into training objectives or constraints, enabling interpretable and specification-aware learning.

Submission Number: 12