NeurIPS 2025 Datasets and Benchmarks Track Submissions

A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings
Fitsum Gaim, Hoyun Song, Huije Lee, Changgeon Ko, Eui Jun Hwang, Jong C. Park
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
AnomalyCoT: A Multi-Scenario Chain-of-Thought Dataset for Multimodal Large Language Models
Jiaxi Cheng, Yuliang Xu, Shoupeng Wang, Ma Tao, Yuchen He, Jinghe Zhang, Sihang Cai, Jiawei Zhen, Jingyi Jia, Yao Wan, Yan Xia, Zhou Zhao
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Benchmarking Large Language Models with Integer Sequence Generation Tasks
Daniel O'Malley, Manish Bhattarai, Nishath Rajiv Ranasinghe, Erick Draayer, Javier E. Santos
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Yejin Choi
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track oral
- Readers: Everyone
Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
Andy Zou, Maxwell Lin, Eliot Krzysztof Jones, Micha V. Nowak, Mateusz Dziemian, Nick Winter, Valent Nathanael, Ayla Croft, Xander Davies, Jai Patel, Robert Kirk, Yarin Gal, Dan Hendrycks, J Zico Kolter, Matt Fredrikson
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
EuroSpeech: A Multilingual Speech Corpus
Samuel Pfisterer, Florian Grötschla, Luca A Lanzendörfer, Florian Yan, Roger Wattenhofer
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient Interactions
Daeun Kyung, Hyunseung Chung, Seongsu Bae, Jiho Kim, Jae Ho Sohn, Taerim Kim, Soo Kyung Kim, Edward Choi
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
Collaborating Vision, Depth, and Thermal Signals for Multi-Modal Tracking: Dataset and Algorithm
Xue-Feng Zhu, Tianyang Xu, Yifan Pan, Jinjie Gu, Xi Li, Jiwen Lu, Xiaojun Wu, Josef Kittler
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
PANORAMA: A Dataset and Benchmarks Capturing Decision Trails and Rationales in Patent Examination
Hyunseung Lim, Sooyohn Nam, Sungmin Na, Ji Yong Cho, June Yong Yang, Hyungyu Shin, Yoonjoo Lee, Juho Kim, Moontae Lee, Hwajung Hong
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges
Kevin David Hayes, Micah Goldblum, Vikash Sehwag, Gowthami Somepalli, Ashwinee Panda, Tom Goldstein
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
A Controllable Examination for Long-Context Language Models
Yijun Yang, Zeyu Huang, Wenhao Zhu, Zihan Qiu, Fei Yuan, Jeff Z. Pan, Ivan Titov
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
Trans-EnV: A Framework for Evaluating the Linguistic Robustness of LLMs Against English Varieties
Jiyoung Lee, Seungho Kim, Jieun Han, Jun-Min Lee, Kitaek Kim, Alice Oh, Edward Choi
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
MedSG-Bench: A Benchmark for Medical Image Sequences Grounding
Jingkun Yue, Siqi Zhang, Zinan Jia, Huihuan Xu, Zongbo Han, Xiaohong Liu, Guangyu Wang
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy
Sebastian Antony Joseph, Syed Murtaza Husain, Stella S. R. Offner, Stéphanie Juneau, Paul Torrey, Adam S. Bolton, Juan P. Farias, Niall Gaffney, Greg Durrett, Junyi Jessy Li
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions
Weizhe Yuan, Jane Yu, Song Jiang, Karthik Padthe, Yang Li, Dong Wang, Ilia Kulikov, Kyunghyun Cho, Yuandong Tian, Jason E Weston, Xian Li
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
ModuLM: Enabling Modular and Multimodal Molecular Relational Learning with Large Language Models
Zhuo Chen, YIZHEN ZHENG, Huan Yee Koh, hongxin xiang, Linjiang Chen, Wenjie Du, Yang Wang
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception
Karthikeyan Chandra Sekaran, Markus Geisler, Dominik Rößle, Adithya Mohan, Daniel Cremers, Wolfgang Utschick, Michael Botsch, Werner Huber, Torsten Schön
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMs
Mohammad Shahab Sepehri, Berk Tinaz, Zalan Fabian, Mahdi Soltanolkotabi
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
HawkBench: Investigating Resilience of RAG Methods on Stratified Information-Seeking Tasks
Hongjin Qian, Zheng Liu, Chao Gao, Yankai Wang, Defu Lian, Zhicheng Dou
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
Escaping the SpuriVerse: Can Large Vision-Language Models Generalize Beyond Seen Spurious Correlations?
Yiwei Yang, Chung Peng Lee, Shangbin Feng, Dora Zhao, Bingbing Wen, Anthony Zhe Liu, Yulia Tsvetkov, Bill Howe
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
AGENTIF: Benchmarking Large Language Models Instruction Following Ability in Agentic Scenarios
Yunjia Qi, Hao Peng, Xiaozhi Wang, Amy Xin, Youfeng Liu, Bin Xu, Lei Hou, Juanzi Li
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversations
Vardhan Dongre, Chi Gui, Shubham Garg, Hooshang Nayyeri, Gokhan Tur, Dilek Hakkani-Tür, Vikram S. Adve
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
All that structure matches does not glitter
Maya Martirossyan, Thomas Egg, Philipp Höllmer, George Karypis, Mark Transtrum, Adrian Roitberg, Mingjie Liu, Richard Hennig, Ellad B. Tadmor, Stefano Martiniani
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding
hongyong han, Wei Wang, Gaowei Zhang, Mingjie Li, Yi Wang
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track oral
- Readers: Everyone
BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks
Anna Sokol, Elizabeth M. Daly, Michael Hind, David Piorkowski, Xiangliang Zhang, Nuno Moniz, Nitesh V Chawla
- Published: 18 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone