NeurIPS 2025 Workshop LLM Evaluation Submissions

Mind the Gap: Evaluating Model- and Agentic-Level Vulnerabilities in LLMs with Action Graphs
Ilham Wicaksono, Zekun Wu, Rahul Patel, Theo King, Adriano Koshiyama, Philip Colin Treleaven
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
JOINTMMSAFE: A Combinatorial Safety Benchmark for Multimodal Foundation Models
Shruti Palaskar, Leon Alexander Gatys, Mona Abdelrahman, Mar Jacobo, Laurence F Lindsey, Gunnar Lund, Yang Xu, Navid Shiee, Jeffrey P. Bigham, Charles Maalouf, Joseph Yitan Cheng
- Published: 24 Sept 2025, Last Modified: 07 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Evaluation and Benchmarking Suite for Financial Large Language Models and Agents
Shengyuan Lin, Jaisal Patel, Qinchuan Zhang, Kaiwen He, Keyi Wang, Yan Wang, Matt White, Kairong Xiao, Xiao-Yang Liu
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
GUARD: Guiding Unbiased Alignment through Reward Debiasing
Advay Samnerkar, Doelle Bhattacharya, Kailash Ranganathan, Ashwinee Panda, Kevin Zhu
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Culturally-Aware Conversations: A Framework & Benchmark for LLMs
Shreya Havaldar, Young Min Cho, Sunny Rai, Lyle Ungar
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
A Statistical Framework for Game-Based AI Evaluation
Felipe Maia Polo, Leshem Choshen, Yuekai Sun, Kristjan Greenewald
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Evaluating Cultural and Linguistic Alignment Across the LLMs
Yunxi Liu, Fuxiao Liu, Clara Fangfang Ma
- Published: 24 Sept 2025, Last Modified: 27 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Do LLMs Know What They Are Capable Of?
Casey O. Barkan, Sidney Black, Oliver Sourbut
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale
Bowen Jiang, Zhuoqun Hao, Young Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo Jose Taylor, Dan Roth
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Beyond Steering: Evaluating Fine-Grained and Multi-Concept Control in LLMs
Arya Labroo, Ivaxi Sheth, Vyas Raina, Amaani Ahmed, Mario Fritz
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Bayesian Evaluation of Blackbox LLM Behavior
Rachel Longjohn, Shang Wu, Catarina G Belém, Saatvik Kher, Padhraic Smyth
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Automatically Extracting Scientific Metrics with LLMs: A Case Study of ImageNet Papers
Mengli Duan, Michael Guerzhoy
- Published: 24 Sept 2025, Last Modified: 10 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Self-Correction Bench: Revealing the Self-Correction Blind Spot in LLMs
Ken Tsui
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
When LLM Meets Time Series: Can LLMs Perform Multistep Time Series Reasoning and Inference
Wen Ye, Jinbo Liu, Defu Cao, Wei Yang, Yan Liu
- Published: 24 Sept 2025, Last Modified: 13 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
Liana Patel, Negar Arabzadeh, Harshit Gupta, Ankita Sundar, Ion Stoica, Matei Zaharia, Carlos Guestrin
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Domain-Aware Scaling Laws Uncover Data Synergy
Kimia Hamidieh, Lester Mackey, David Alvarez-Melis
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
FEval-TTC: Fair Evaluation Protocol for Test-Time Compute
Pavel Rumiantsev, Soumyasundar Pal, Yingxue Zhang, Mark Coates
- Published: 24 Sept 2025, Last Modified: 23 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Haystack Engineering: Context Engineering Meets the Long-Context Challenge in Large Language Models
Mufei Li, Dongqi Fu, Limei Wang, Si Zhang, Hanqing Zeng, Kaan Sancak, Ruizhong Qiu, Haoyu Peter Wang, Xiaoxin He, Xavier Bresson, Yinglong Xia, Chonglin Sun, Pan Li
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Breaking the Mirror: Examining Self-Preference in LLM Evaluators through Activation-Based Representations
Dani Roytburg, Matthew Bozoukov, Hongyu Fu, Matthew Nguyen, Jou Barzdukas, Narmeen Fatimah Oozeer
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Smarter Sampling for LLM Judges: Reliable Evaluation on a Budget
Alyssa Unell, Natalie Dullerud, Nigam Shah, Sanmi Koyejo
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
How to Get Your LLM to Generate Challenging Problems for Evaluation
Arkil Patel, Siva Reddy, Dzmitry Bahdanau
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
RELIC: Evaluating Compositional Instruction Following via Language Recognition
Jackson Petty, Michael Y. Hu, Wentao Wang, Shauli Ravfogel, William Merrill, Tal Linzen
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
ADCA: Artifact-Based Dataset Creativity Assessment
Harrison Sims, Gabriel Ganberg, Robert McCormack, Svitlana Volkova
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
An Evaluation Study of Hybrid Methods for Multilingual PII Detection
Harshit Rajgarhia, Suryam Gupta, Asif Shaik, Gulipalli Praveen Kumar, Y Santhoshraj, Sanka Nithya Tanvy Nishitha, Abhishek Mukherji
- Published: 24 Sept 2025, Last Modified: 24 Sept 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone
Learning from Generalization Patterns: An Evaluation-Driven Approach to Enhanced Data Augmentation for Fine-Tuning Small Language Models
Huan Song, Deeksha Razdan, Yiyue Qian, Arijit Ghosh Chowdhury, Parth Patwa, Aman Chadha, Shinan Zhang, Sharlina Keshava, Hannah R Marlowe
- Published: 24 Sept 2025, Last Modified: 20 Oct 2025
- NeurIPS 2025 LLM Evaluation Workshop Poster
- Readers: Everyone