Evaluating Humanlike Memory Effects in Transformers Using Item Recognition Tasks

Christian Clark, William Schuler

Keywords: memory, cued recall, item recognition, primacy, recency, serial position

Abstract:

Recent studies examining cued recall in Transformers have observed that these language models remember information from the beginning or end of a passage more easily than information in the middle, a pattern which is evocative of serial position effects (primacy and recency) observed in human memory. However, while these effects have been documented in humans across a range of memory tasks (e.g., serial recall, free recall, item recognition), it is less clear whether they generalize beyond cued recall in Transformers.

We address this limitation of previous work by performing novel behavioral evaluations on Transformers using a simple item recognition paradigm, which we compare against evaluations using cued recall. We find that Transformers show weak or absent recency effects in item recognition, a pattern which differs from human behavior and from Transformers' own behavior in cued recall. A subsequent experiment examines the role of Transformers' architectural biases in producing serial position effects in item recognition and cued recall.

Scope Confirmation: To the best of my judgment, this submission falls within the scope of CoNLL.

Primary Area Selection: Computational Psycholinguistics, Cognition and Linguistics

Secondary Area Selection: Theoretical Analysis and Interpretation of ML Models for NLP

Use Of Generative Artificial Intelligence Tools: Yes, for writing code

Data Collection From Human Subjects: No

Submission Type: Archival: I certify that the submission has not been previously published, nor is the material in it under review by another journal or conference. Further, no material in it will be submitted for review at another conference or journal while under review by CoNLL 2026.

Submission Number: 202