Modeling the Viewpoint Theory of Mental Rotation

Yeon Tae Chung, Vijay Marupudi, Siddhartha Krishna Vemuri, Athanassios Economou, Sashank Varma

Keywords: mental rotation, viewpoint theory, image processing, computer vision

TL;DR: Two models of mental rotation adapting computational methods from image processing and computer vision were evaluated; they were found to be inadequate as instantiations of the viewpoint and analog theories of mental rotation.

Abstract: Multiple theories have been proposed to explain mental rotation. The analog theory was inspired by the seminal experiment that introduced the Shepard–Metzler (SM) task, which found a linear trend between the angle between two objects and the time to judge them as the same or different (Shepard & Metzler, 1971). This finding was taken as evidence of objects being mentally represented and spatially transformed (e.g., rotated). Subsequent theories have questioned whether mental rotation is such a kinematic process. The viewpoint theory, in particular, explains how response time (RT) is linearly related to angular disparity in terms of the *similarity* between viewpoints. Here, we evaluated two models that instantiate this theory in an experiment on the SM task and a variant of it. One model is an algorithm from image processing and operates at the raw level of images, computing the normalized mean-shifted cross-correlation (NMSCC). The other model is based on a computational architecture adapted from computer vision and operates at the structured level of vector spaces: the cosine similarity of latent vectors in AlexNet, a convolutional neural network (CNN). Neither model demonstrated the expected performance profile consistent with the human data. In response, we introduced a process account as an alternative way to interpret the results of the experiments under the analog theory.

Paper Track: Technical paper

Submission Number: 3