Semantic feature verification in FLAN-T5

Siddharth Suresh, Kushin Mukherjee, Timothy T. Rogers

Keywords: Large Language Models, Cognitive Science, Semantic Norms, Human conceptual structure, AI conceptual structure

TL;DR: This study investigates the potential of large language models to outsource the verification phase of semantic feature norms, finding that combining human and machine-verified data provides the most accurate estimates of human semantic structure.

Abstract: This study evaluates the potential of a large language model for aiding in generation of semantic feature norms–a critical tool for evaluating conceptual structure in cognitive science. Building from an existing human-generated dataset, we show that machine-verified norms capture aspects of conceptual structure beyond what is expressed in human norms alone, and better explain human judgments of semantic similarity amongst items that are distally related. The results suggest that LLMs can greatly enhance traditional methods of semantic feature norm verification, with implications for our understanding of conceptual representation in humans and machines.