NeurIPS 2025 Datasets and Benchmarks Track Submissions

DecoyDB: A Dataset for Graph Contrastive Learning in Protein-Ligand Binding Affinity Prediction
Yupu Zhang, Zelin Xu, Tingsong Xiao, Gustavo Seabra, Yanjun Li, Chenglong Li, Zhe Jiang
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
Zhiheng Xi, Guanyu Li, YuTao Fan, Honglin Guo, Yufang Liu, Xiaoran Fan, Jiaqi Liu, dingjinchao, Wangmeng Zuo, Zhenfei Yin, LEI BAI, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
PAC Bench: Do Foundation Models Understand Prerequisites for Executing Manipulation Policies?
Atharva Gundawar, Som Sagar, Ransalu Senanayake
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos
Kejian Zhu, Zhuoran Jin, Hongbang Yuan, Jiachun Li, Shangqing Tu, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
- 12 May 2025 (modified: 30 Oct 2025)
- Submitted to NeurIPS 2025 Datasets and Benchmarks Track
- Readers: Everyone
PuzzleJax: a Benchmark for Reasoning and Learning
Sam Earle, Yuchen Li, Graham Todd, Zehua Jiang, Ahmed Khalifa, Muhammad Umair Nasir, Andrzej Banburski-Fahey, Julian Togelius
- 12 May 2025 (modified: 30 Oct 2025)
- Submitted to NeurIPS 2025 Datasets and Benchmarks Track
- Readers: Everyone
Absence Bench: Language Models Can’t See What’s Missing
Harvey Yiyun Fu, Aryan Shrivastava, Jared Moore, Peter West, Chenhao Tan, Ari Holtzman
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
BuildingBRep-11K: Precise Multi-Storey B-Rep Building Solids with Rich Layout Metadata
YU GUO, Hongji Fang, Tianyu Fang, Zhe Cui
- 12 May 2025 (modified: 30 Oct 2025)
- Submitted to NeurIPS 2025 Datasets and Benchmarks Track
- Readers: Everyone
Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversations
Suhas BN, Andrew M. Sherrill, Rosa I. Arriaga, Christopher Wiese, Saeed Abdullah
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
From Experiments to Discovery: A Principled Approach to Measuring How Well LLMs Do Science
Kanishk Gandhi, Michael Y. Li, Lyle Goodyear, Agam Bhatia, Ying Li, Aditi Bhaskar, Mohammed Zaman, Noah Goodman
- 12 May 2025 (modified: 30 Oct 2025)
- Submitted to NeurIPS 2025 Datasets and Benchmarks Track
- Readers: Everyone
CoRe: Benchmarking LLMs’ Code Reasoning Capabilities through Static Analysis Tasks
Danning Xie, Mingwei Zheng, Xuwei Liu, Jiannan Wang, Chengpeng Wang, Lin Tan, Xiangyu Zhang
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
SutureBot: A Precision Framework & Benchmark For Autonomous End-to-End Suturing
Jesse Haworth, Juo-Tung Chen, Nigel Nelson, Ji Woong Kim, Masoud Moghani, Chelsea Finn, Axel Krieger
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
GUARD: Constructing Realistic Two-Player Matrix and Security Games for Benchmarking Game-Theoretic Algorithms
Noah Krever, Jakub Cerny, Moise Blanchard, Christian Kroer
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions
Weijie Xu, Shixian Cui, Xi Fang, Chi Xue, Stephanie Eckman, Chandan K. Reddy
- 12 May 2025 (modified: 30 Oct 2025)
- Submitted to NeurIPS 2025 Datasets and Benchmarks Track
- Readers: Everyone
PARROT: A Benchmark for Evaluating LLMs in Cross-System SQL Translation
Wei Zhou, Guoliang Li, Haoyu Wang, Yuxing Han, Wu Xufei, Fan Wu, Xuanhe Zhou
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
CLEVER: A Curated Benchmark for Formally Verified Code Generation
Amitayush Thakur, Jasper Lee, George Tsoukalas, Meghana Sistla, Matthew Zhao, Stefan Zetzsche, Greg Durrett, Yisong Yue, Swarat Chaudhuri
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
HouseLayout3D: A Benchmark and Training-free Baseline for 3D Layout Estimation in the Wild
Valentin Bieri, Marie-Julie Rakotosaona, Keisuke Tateno, Francis Engelmann, Leonidas Guibas
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Struct-Bench: A Benchmark for Differentially Private Structured Text Generation
Shuaiqi Wang, Vikas Raunak, Arturs Backurs, Victor Reis, Pei Zhou, Sihao Chen, Longqi Yang, Zinan Lin, Sergey Yekhanin, Giulia Fanti
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Can LLMs Match the Observations of Systematic Reviews?
Christopher Polzak, Alejandro Lozano, Min Woo Sun, James Burgess, Yuhui Zhang, Kevin Wu, Serena Yeung-Levy
- 12 May 2025 (modified: 30 Oct 2025)
- Submitted to NeurIPS 2025 Datasets and Benchmarks Track
- Readers: Everyone
Two Causally Related Needles in a Video Haystack
Miaoyu Li, Qin Chao, Boyang Li
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
ORBIT - Open Recommendation Benchmark for Reproducible Research with Hidden Tests
Jingyuan He, Jiongnan Liu, Vishan Vishesh Oberoi, Bolin Wu, Mahima Jagadeesh Patel, Kangrui Mao, Chuning Shi, I-Ta Lee, Arnold Overwijk, Chenyan Xiong
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Nikhil Kandpal, Brian Lester, Colin Raffel, Sebastian Majstorovic, Stella Biderman, Baber Abbasi, Luca Soldaini, Enrico Shippole, A. Feder Cooper, Aviya Skowron, Shayne Longpre, Lintang Sutawika, Alon Albalak, Zhenlin Xu, Guilherme Penedo, Loubna Ben allal, Elie Bakouch, John David Pressman, Honglu Fan, Dashiell Stander et al. (7 additional authors not shown)
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
Liyan Tang, Grace Kim, Xinyu Zhao, Thom Lake, Wenxuan Ding, Fangcong Yin, Prasann Singhal, Manya Wadhwa, Zeyu Leo Liu, Zayne Rea Sprague, Ramya Namuduri, Bodun Hu, Juan Diego Rodriguez, Puyuan Peng, Greg Durrett
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
Myeongsoo Kim, Shweta Garg, Baishakhi Ray, Varun Kumar, Anoop Deoras
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
Manish Shetty, Naman Jain, Jinjian Liu, Vijay Kethanaboyina, Koushik Sen, Ion Stoica
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Measuring Fingerprints of Web-filtered Text Datasets and Fingerprint Propagation Through Training
Youssef Mansour, Reinhard Heckel
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone