Brainstorm, then Select: a Generative Language Model Improves Its Creativity ScoreDownload PDF

21 Nov 2022 (modified: 05 May 2023)creativeAIReaders: Everyone
Keywords: generative language models, creativity
TL;DR: We can do better on the Alternate Uses Test for originality and utility than the average human by asking GPT to evaluate its own responses.
Abstract: Creative problem solving is a crucial ability for intelligent agents. A common method that individuals or groups use to invent creative solutions is to start with a ``brainstorming" phase, where many solutions to a problem are proposed, and then to follow with a ``selection" phase, where those solutions are judged by some criteria so that the best solutions can be selected. Using the Alternate Uses Task, a test for divergent thinking abilities (a key aspect of creativity) we show that when a large language model is given a sequence of prompts that include \textit{both} brainstorming and selection phases, its performance improves over brainstorming alone. Furthermore, we show that by following this paradigm, a large language model can even achieve higher than average human performance on the same task. Following our analysis, we propose further research to gain a clearer understanding of what counts as ``creativity" in language models.
Submission Type: archival
Presentation Type: onsite
Presenter: Douglas Summers-Stay
0 Replies