VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models

ICLR 2026 Conference Submission16366 Authors

Keywords: LLM, Human Values, Value-based Alignment, Value Evaluation

TL;DR: VALUEFLOW introduces the first unified framework for value-based alignment in LLMs—spanning extraction, evaluation, and calibrated intensity steering—providing scalable infrastructure for pluralistic and accountable control.

Abstract: Aligning Large Language Models (LLMs) with the diverse spectrum of human values remains a central challenge: preference-based methods often fail to capture deeper motivational principles. Value-based approaches offer a more principled path, yet three gaps persist--extraction often ignores hierarchical structure, evaluation detects presence but not calibrated intensity, and therefore, the steerability of LLMs at controlled intensities remains insufficiently understood. To address these limitations, we introduce VALUEFLOW, the first unified framework that spans extraction, evaluation, and steering with calibrated intensity control. The framework integrates three components: (i) HiVES, a hierarchical value embedding space that captures intra- and cross-theory value structure; (ii) the Value Intensity DataBase (VIDB), a large-scale resource of value-labeled texts with intensity estimates derived from ranking-based aggregation; and (iii) an anchor-based evaluator that produces consistent intensity scores for model outputs by ranking them against VIDB panels. Using VALUEFLOW, we conduct a comprehensive large-scale study across ten models and four value theories, identifying asymmetries in steerability and composition laws for multi-value control. This paper establishes a scalable infrastructure for evaluating and controlling value intensity, advancing pluralistic and accountable alignment of LLMs.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 16366