NeurIPS 2025 Workshop LLM Evaluation Submissions

Progress over Points: Reframing LM Benchmarks Around Scientific Objectives
Alwin Jin, Sean M. Hendryx, Vaskar Nath
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Recovery-Bench: Evaluating Agentic Recovery from Mistakes
Shangyin Tan, Kevin Lin, Koushik Sen, Matei Zaharia
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
In-Context Learning for Esoteric Programming Languages: Evaluating and Enhancing LLM Reasoning Without Fine-Tuning
Saraswathy Amjith, Michael X. Wang, Arul Kolla, Jayson Lynch, Neil Thompson
- Published: 24 Sept 2025, Last Modified: 26 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Beyond the Singular: Revealing the Value of Multiple Generations in Benchmark Evaluation
Wenbo Zhang, Hengrui Cai, Wenyu Chen
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Improving Automated LLM Evaluation by Introducing Personas in LLM Red-Teaming
Wesley Deng, Sunnie S. Y. Kim, Akshita Jha, Ken Holstein, Motahhare Eslami, Lauren Wilcox, Leon Alexander Gatys
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation
Xunyi Jiang, Dingyi Chang, Xin Xu
- Published: 24 Sept 2025, Last Modified: 09 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Evaluating Language Models' Evaluations of Games
Katherine M. Collins, Cedegao E. Zhang, Graham Todd, Lance Ying, Mauricio Barba da Costa, Ryan Liu, Adrian Weller, Ionatan Kuperwajs, Lionel Wong, Joshua B. Tenenbaum, Thomas L. Griffiths
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
On Evaluating Methods vs. Evaluating Models
Olawale Elijah Salaudeen, Florian E. Dorner, Peter Hase
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Oral
- Readers: Everyone
A Benchmark for Description-Based Evaluation of Social Bias in LLMs
Jinhao Pan, Kyle Li, Bowen Wei, Ziwei Zhu
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
RULERv2: From Basic Retrieval to Complex Reasoning, A Bottom-Up Benchmark for Long-Context Evaluation
Cheng-Ping Hsieh, Faisal Ladhak, Krishna C Puvvada, Boris Ginsburg
- Published: 24 Sept 2025, Last Modified: 09 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Reward Model Overoptimisation in Iterated RLHF
Lorenz Wolf, Robert Kirk, Mirco Musolesi
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
HypoTermInstruct: Instructing Large Language Models not to Hallucinate
CEM ULUOGLAKCI, Tugba Taskaya Temizel
- Published: 24 Sept 2025, Last Modified: 27 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Who Routes the Router: Rethinking the Evaluation of LLM Routing Systems
Jiayi Yuan, Yifan Lu, Rixin Liu, Yu-Neng Chuang, Hongyi Liu, Shaochen Zhong, Yang Sui, Guanchu Wang, Jiarong Xing, Xia Hu
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Automatic agent chaining for multimodal task support
Ramesh Manuvinakurike, Celal Savur, Emanuel Moss, Elizabeth Anne Watkins, Saurav Sahay, Giuseppe Raffa
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
UQ: Assessing Language Models on Unsolved Questions
Fan Nie, Ken Liu, Zihao Wang, Rui Sun, Wei Liu, Weijia Shi, Huaxiu Yao, Linjun Zhang, Andrew Y. Ng, James Zou, Sanmi Koyejo, Yejin Choi, Percy Liang, Niklas Muennighoff
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
The Shepherd Test: How Will SuperIntelligent Agents Balance Care and Control in Asymmetric Relationships?
Djallel Bouneffouf, Matthew Riemer, Kush R. Varshney
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
In-Context Meta-Learning with Large Language Models for Automated Model and Hyperparameter Selection
Youssef Attia El Hili, Albert Thomas, Abdelhakim Benechehab, Corentin Léger, Corinne Ancourt, Balázs Kégl
- Published: 24 Sept 2025, Last Modified: 30 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
CCWise: Carbon–Cost Aware Regional LLM Orchestration for Next-Gen Sustainable AI
Ratul Kishore Saha, Dheeraj Chahal, Rekha Singhal, Manoj Nambiar
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
MAGNET: Mathematical Assurance of Generative AI Network Evaluation Toolkit
Jon Crall, David Joy, Roderic Collins, Benjamin Fenelon, Anthony Hoogs, Brian H Hu
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
Olawale Elijah Salaudeen, Anka Reuel, Ahmed M Ahmed, Suhana Bedi, Zachary Robertson, Sudharsan Sundar, Benjamin W. Domingue, Angelina Wang, Sanmi Koyejo
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Silent Tokens, Loud Effects: Padding in LLMs
Rom Himelstein, Amit LeVi, Yonatan Belinkov, Avi Mendelson
- Published: 24 Sept 2025, Last Modified: 05 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
The Narcissus Hypothesis: Descending to the Rung of Illusion
Riccardo Cadei, Christian Internò
- Published: 24 Sept 2025, Last Modified: 21 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
LinguaMark: Do Multimodal Models Speak Fairly? A Benchmark-Based Evaluation
Ananya Raval, Aravind Narayanan, Vahid Reza Khazaie, Shaina Raza
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Data Centric Guard (DC-Guard) - A Framework for Trustworthy LLM Evaluation
Vishnu Vardhan Yadoji
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
From Acceleration to Saturation: Scaling Behavior of Bootstrapped Language Model Pretraining
Seng Pei Liew, Takuya Kato
- Published: 24 Sept 2025, Last Modified: 09 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone