Cognitive Bias in High-Stakes Decision-Making with LLMs

Anonymous

Abstract:

Large language models (LLMs) offer significant potential as tools to support an expanding range of decision-making tasks. However, given their training on human (created) data, LLMs can inherit both societal biases against protected groups, as well as be subject to cognitive bias. Such human-like bias can impede fair and explainable decisions made with LLM assistance.
Our work introduces \ours, a framework designed to uncover, evaluate, and mitigate cognitive bias in LLMs, particularly in high-stakes decision-making tasks. Inspired by prior research in psychology and cognitive sciences, we develop a dataset containing 16,800 prompts to evaluate different cognitive biases (e.g., prompt-induced, sequential, inherent). We test various bias mitigation strategies, amidst proposing a novel method utilising LLMs to debias their own prompts.
Our analysis provides a comprehensive picture on the presence and effects of cognitive bias across different commercial and open-source models. We demonstrate that our self-help debiasing effectively mitigate cognitive bias without having to manually craft examples for each bias type.

Paper Type: long

Research Area: Ethics, Bias, and Fairness

Contribution Types: Model analysis & interpretability, Data resources

Languages Studied: English

Preprint Status: We are considering releasing a non-anonymous preprint in the next two months (i.e., during the reviewing process).

A1: yes

A1 Elaboration For Yes Or No: 7

A2: yes

A2 Elaboration For Yes Or No: 7

A3: yes

A3 Elaboration For Yes Or No: 1

B: yes

B1: yes

B1 Elaboration For Yes Or No: 1,2,3,4,5

B2: yes

B2 Elaboration For Yes Or No: 7

B3: yes

B3 Elaboration For Yes Or No: 7

B4: n/a

B5: yes

B5 Elaboration For Yes Or No: 3

B6: yes

B6 Elaboration For Yes Or No: 5

C: yes

C1: yes

C1 Elaboration For Yes Or No: 5,7

C2: n/a

C2 Elaboration For Yes Or No: 3,4

C3: yes

C3 Elaboration For Yes Or No: 5

C4: n/a

D: no

D1: n/a

D2: n/a

D3: n/a

D4: n/a

D5: n/a

E: yes

E1: yes

E1 Elaboration For Yes Or No: 1,2,3,4,5