Evaluating Vision-Language Models on the TriangleCOPA Benchmark

Ankur Chemburkar, Andrew S. Gordon, Andrew Feng

Abstract: The TriangleCOPA benchmark consists of 100 textual questions with videos depicting the movements of simple shapes in the style of the classic social-psychology film created by Fritz Heider and Marianne Simmel in 1944. In our experiments, we investigate the performance of current vision-language models on this challenging benchmark, assessing the capability of these models for visual anthropomorphism and abstract interpretation.