PSI: A Benchmark for Human Interpretation and Response in Traffic Interactions

TAOTAO JING, Tina Chen, Renran Tian, Yaobin Chen, Joshua Domeyer, Heishiro Toyoda, Rini Sherony, Zhengming Ding

Keywords: reasoning, explainable AI, multi-modality, autonomous driving

TL;DR: A Benchmark for Human Interpretation and Response in Traffic Interactions

Abstract: Accurately modeling pedestrian intention and understanding driver decision-making processes are critical for the development of safe and socially aware autonomous driving systems. However, existing datasets primarily emphasize observable behavior, offering limited insight into the underlying causal reasoning that informs human interpretation and response during traffic interactions. To address this gap, we introduce PSI, a benchmark dataset that captures the dynamic evolution of pedestrian crossing intentions from the driver’s perspective, enriched with human-annotated textual explanations that reflect the reasoning behind intention estimation and driving decision making. These annotations offer a unique foundation for developing and benchmarking models that combine predictive performance with interpretable and human-aligned reasoning. PSI supports standardized tasks and evaluation protocols across multiple dimensions, including pedestrian intention prediction, driver decision modeling, reasoning generation, and trajectory forecasting and more. By enabling causal and interpretable evaluation, PSI advances research toward autonomous systems that can reason, act, and explain in alignment with human cognitive processes.

Dataset URL: https://huggingface.co/datasets/psi-benchmark/PSI-Benchmark

Code URL: https://github.com/PSI-Intention2022/PSI-Dataset

Primary Area: Datasets & Benchmarks for applications in language modeling and vision language modeling

Submission Number: 221