For Questions of Ought, AI Could Use Some SAGE Advice

Smitha Milli, Ratip Emin Berker, Sonja Kraiczy, Claudia Shi, Jack Kussman, Avinandan Bose, Edith Elkind, Himaghna Bhattacharjee, Ariel D. Procaccia, Maximilian Nickel

Keywords: pluralistic alignment, AI alignment, disempowerment, simulation

Abstract: As AI systems are increasingly used for normative assistance—guidance on what people ought to do or think—there is growing concern about the innumerous and opaque ways that models may shape users’ beliefs and decisions. We argue that prevailing alignment paradigms are ill-suited to this setting: for normative questions without ground truth, the central failure mode is not incorrectness but reduced user agency, when a model steers users toward particular conclusions rather than helping them form their own. We propose **simulation-augmented generation (SAGE)** as an alternative paradigm in which models act as *faithful mirrors of society*. At inference time, a model selects an appropriate reference population for the prompt at hand (optionally adjustable by the user), queries generative simulations of individuals from that population for their open-ended judgments, and synthesizes these into a response while exposing who was consulted and how representative the synthesis is. By helping users access the landscape of societal perspectives—rather than serving as an arbiter of judgment—SAGE assists users in normative choices while upholding their agency.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 62