SAGE: A FRAMEWORK FOR SEMANTIC-ALIGNMENT- GUIDED ENGINEERING OF PROMPTS AND FINE- TUNING IN INDUSTRIAL CONTROL TASKS

JiatongZheng, Dong Li, Chunhe Song

Keywords: Fine-Tuning Strategy, Industrial Control Code Generation, Large Language Model, Semantic Alignment

Abstract: Large language models show great potential for code generation tasks, but automatic code generation for industrial control systems still faces challenges such as inaccurate semantic understanding, a lack of alignment evaluation, and a shortage of domain-specific fine-tuning models. Given the stringent requirements for real-time performance, security, logical rigor, and correct execution of industrial control code, existing general-purpose methods struggle to meet these demands. Therefore, this paper proposes a semantic alignment-guided prompt engineering approach for industrial control tasks. The approach consists of three core components: first, a dataset of function prompt formats covering five structured prompt patterns and a selection of 1,500 prompt examples for industrial control tasks is constructed; second, a semantic alignment analysis metric is designed to evaluate the semantic correctness and task consistency of code generated by different models; and third, an alignment-guided fine-tuning strategy is proposed, leveraging prompt-output-intent triples to enhance the model’s generation capabilities for industrial control tasks. Experiments are conducted on five mainstream 7B models: DeepSeek-7B, Qwen2.5-7B, InternLM2-7B, Mistral-7B, and Gemma-7B. Results show that after fine-tuning, the executable performance of Mistral-7B and DeepSeek-7B increased from 0.719 to 0.886 and from 0.676 to 0.837, respectively, and the BLEU scores increased from 3.79 to 7.45 and from 3.45 to 6.62, respectively. All models maintained intent consistency (Intent = 1.000). Gemma-7B and Qwen2.5-7B showed decreases in executable performance, success rate and BLEU, suggesting possible overfitting or distribution mismatch issues. The method proposed in this paper significantly improves the code executable performance and semantic alignment of some models in industrial control scenarios. It also reveals the sensitivity of model architecture to fine-tuning strategies, providing an important reference for subsequent architecture aware alignment optimization.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 2134