Go to ICLR 2026 Conference homepageAn Efficient and Modular Framework for Targeted Harm Mitigation in LLMs
Keywords: harm alignment, mixture of experts, safety, alignment, large language models, routing
Abstract: Large language models (LLMs) are powerful zero-shot learners but remain prone to misalignment with human preferences, often producing biased, toxic, or otherwise harmful outputs. Existing alignment methods, while effective, are costly and tightly coupled to the base model, limiting flexibility and scalability. To eliminate harms from misaligned model responses, we propose a modular correction framework that augments pretrained LLMs with Activated LoRA (aLoRA) adapters and a context-aware routing mechanism. Our approach enables expert adapters to activate mid-sequence without invalidating the KV cache, allowing low-latency, targeted correction during generation. Each expert is trained to detect and mitigate specific harms, such as bias or toxicity. A learned router dynamically selects appropriate experts based on the model's intermediate outputs. We demonstrate that our system improves alignment on standard safety benchmarks while preserving task performance and enabling composable, scalable corrections. This work offers a lightweight, efficient path toward safer and more controllable LLM deployments.
Primary Area: alignment, fairness, safety, privacy, and societal considerations
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2026/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 21071
Loading