ChOiRe: Characterizing and Predicting Human Opinions with Chain of Opinion Reasoning

Anonymous

TL;DR: We present a novel and state-of-the-art framework for characterizing and predicting human opinions.

Abstract: Aligning language models (LMs) with human opinion is challenging yet vital to enhance their grasp of human values, preferences, and beliefs. We present ChOiRe, a four-step solution framework to predict human opinion that differentiates between the user's \emph{explicit personae} (i.e. demographic or ideological attributes) that are manually declared, and \emph{implicit personae} inferred from user historical opinions. Specifically, it consists of (i) an LM analyzing the user's explicit personae to filter out irrelevant attributes; (ii) the LM ranking the implicit persona opinions into a preferential list; (iii) Chain-of-Opinion (CoO) reasoning, where the LM sequentially analyzes the explicit personae and the most relevant implicit personae to perform opinion prediction; (iv) and where ChOiRe executes Step (iii)'s CoO multiple times with increasingly larger lists of implicit personae to overcome insufficient personae information to infer a final result. ChOiRe achieves new state-of-the-art effectiveness with limited inference calls, improving previous LLM-based techniques significantly by 3.22\%.

Paper Type: long

Research Area: Question Answering

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English