Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds

Edward Chen, Sang T. Truong, Natalie Dullerud, Sanmi Koyejo, Carlos Guestrin

Keywords: multi-objective, preference learning, interactive learning

Abstract: Complex decision-making tasks across healthcare, drug discovery, engineering design, and deep learning frequently involve optimizing multiple competing objectives while navigating computationally expensive evaluations. Decision-makers (DMs) must select Pareto-optimal solutions that align with their implicit preferences, yet the computational burden of evaluating solutions and the cognitive load of validating tradeoffs make exhaustive Pareto frontier exploration infeasible. While DMs often possess domain knowledge that helps constrain the initial search space -- for instance, clinicians typically have priors for specific regions of the tradeoff surface to explore -- existing methods lack systematic approaches for iteratively refining these regions of interest. Critically, in high-stakes domains like healthcare, DMs must develop confidence that they have not overlooked superior solutions before committing to a final decision. We present Active-MoSH, a novel interactive framework that formalizes this exploration process by integrating soft-hard utility functions with probabilistic preference learning. Our framework maintains distributions over both the DM's latent preference vector and feedback, by way of soft and hard bounds, enabling adaptive refinement of the explored Pareto frontier subset. We develop an active sampling strategy that optimizes the exploration-exploitation tradeoff while minimizing cognitive burden. To address the fundamental need for solution validation, we propose Active-MoSH, which leverages local multi-objective sensitivity analysis to systematically build DM trust by quantifying the robustness of solutions and identifying potentially overlooked regions of the Pareto frontier. Through extensive experiments on synthetic benchmarks and real-world applications in engineering design and cervical cancer brachytherapy treatment planning, we demonstrate that our framework efficiently guides DMs toward optimal tradeoff points while providing rigorous validation of solution quality.

Submission Type: Long Paper (9 Pages)

Archival Option: This is a non-archival submission

Presentation Venue Preference: ICLR 2025

Submission Number: 87