Leveraging Large Models for Evaluating Novel Content: A Case Study on Advertisement Creativity

ACL ARR 2024 December Submission1471 Authors

Abstract: Evaluating creativity is challenging, even for humans, not only because of its subjectivity but also because it involves complex cognitive processes. Inspired by work in marketing, we attempt to break down creativity into atypicality and originality. With fine-grained human annotations on these dimensions, we propose a suit of tasks specifically for such a subjective problem. We also evaluate the alignment between state-of-the-art (SoTA) vision language models (VLM) and humans on our proposed benchmark, demonstrating both the promises and challenges of using VLMs for automatic creativity assessment.

Paper Type: Short

Research Area: Resources and Evaluation

Research Area Keywords: creativity, subjective task, automatic evaluation

Contribution Types: Model analysis & interpretability, Data resources

Languages Studied: English

Submission Number: 1471