Brainstorm, then Select: a Generative Language Model Improves Its Creativity Score

Douglas Summers-Stay, Clare R. Voss, Stephanie M. Lukin

Keywords: generative language models, creativity

TL;DR: We can do better on the Alternate Uses Test for originality and utility than the average human by asking GPT to evaluate its own responses.

Abstract:

Creative problem solving is a crucial ability for intelligent agents. A common method that individuals or groups use to invent creative solutions is to start with a ``brainstorming" phase, where many solutions to a problem are proposed, and then to follow with a ``selection" phase, where those solutions are judged by some criteria so that the best solutions can be selected. Using the Alternate Uses Task, a test for divergent thinking abilities (a key aspect of creativity) we show that when a large language model is given a sequence of prompts that include \textit{both} brainstorming and selection phases, its performance improves over brainstorming alone. Furthermore, we show that by following this paradigm, a large language model can even achieve higher than average human performance on the same task.
Following our analysis, we propose further research to gain a clearer understanding of what counts as ``creativity" in language models.

Submission Type: archival

Presentation Type: onsite

Presenter: Douglas Summers-Stay