Self-Preference Bias in LLM-as-a-Judge

Koki Wataoka, Tsubasa Takahashi, Ryokan Ri

Keywords: large language model, llm-as-a-judge, bias, fairness

TL;DR: We propose a novel quantitative metric to measure self-preference bias in LLM-as-a-judge.

Abstract:

Automated evaluation leveraging large language models (LLMs), commonly referred to as LLM evaluators or LLM-as-a-judge, has been widely used in measuring the performance of dialogue systems.
However, the self-preference bias in LLMs has posed significant risks, including promoting specific styles or policies intrinsic to the LLMs.
Despite the importance of this issue, there is a lack of established methods to measure the self-preference bias quantitatively, and its underlying causes are poorly understood.
In this paper, we introduce a novel quantitative metric to measure the self-preference bias.
Our experimental results demonstrate that GPT-4 exhibits a significant degree of self-preference bias.
To explore the causes, we hypothesize that LLMs may favor outputs that are more familiar to them, as indicated by lower perplexity.
We analyze the relationship between LLM evaluations and the perplexities of outputs.
Our findings reveal that LLMs assign significantly higher evaluations to outputs with lower perplexity than human evaluators, regardless of whether the outputs were self-generated.
This suggests that the essence of the bias lies in perplexity and that the self-preference bias occurs because the LLMs' own outputs have lower perplexity.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8884