MEAV: Model Editing with Alignment Vectors for inference time LLM alignment in single and multidomain preference spectrum

ACL ARR 2026 January Submission1333 Authors

Keywords: alignment, inference

Abstract:

Aligning LLMs to nuanced preference levels requires adequate flexibility and control, which can be a resource-intensive and time-consuming procedure. Existing
training-time alignment methods require full re-training when a change is needed and inference-time ones typically require access to the reward model at each inference step. We introduce **MEAV**, an inference-time model-editing-based LLM alignment method that learns encoded representations of preference dimensions, called *Alignment Vectors* (AV). These representations enable dynamic adjusting of the model behavior during inference through simple linear operations. Here, we focus on three gradual response levels across three specialized domains: medical, legal, and financial, exemplifying its practical potential. We introduce adjustable preference knobs during inference, allowing users to tailor their LLM outputs while reducing the inference cost by half compared to the prompt engineering approach. Additionally, AVs are transferable across different fine-tuning stages of the same model, demonstrating flexibility. AVs also facilitate multidomain, diverse preference alignment, making the process 12x faster than the retraining approach.

Paper Type: Long

Research Area: Safety and Alignment in LLMs

Research Area Keywords: safety and alignment, efficient models, inference methods

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency, Data resources

Languages Studied: English

Submission Number: 1333