Learning to Plan with Personalized Preferences

Manjie Xu, Xinyi Yang, Wei Liang, Chi Zhang, Yixin Zhu

Keywords: preference, human-ai interaction, human-ai cooperation, few-shot planning

Abstract: Effective integration of AI agents into daily life requires them to understand and adapt to individual human preferences, particularly in assistive roles. Although recent studies on embodied intelligence have advanced significantly, they typically adopt generalized approaches that overlook personalized preferences in planning. Cognitive research has demonstrated that these preferences serve as crucial intermediate representations in human decision-making processes and, though implicitly expressed through minimal demonstrations, can generalize across diverse planning scenarios. To systematically address this gap, we introduce the Preference-based Planning (PBP) benchmark, an embodied benchmark designed to evaluate agents' ability to learn preferences from few demonstrations and adapt their planning strategies accordingly. PBP features hundreds of diverse preferences spanning from atomic actions to complex sequences, enabling comprehensive assessment of preference learning capabilities. Evaluations of SOTA methods reveal that while symbol-based approaches show promise in scalability, significant challenges remain in learning to generate plans that satisfy personalized preferences. Building on these findings, we develop agents that not only learn preferences from few demonstrations but also adapt their planning strategies based on these preferences. Experiments in PBP demonstrate that incorporating learned preferences as intermediate representations significantly improves an agent's ability to construct personalized plans, establishing preference as a valuable abstraction layer for adaptive planning.

Primary Area: datasets and benchmarks

Submission Number: 2547