Automated Benchmarking of LLMs: Applying Regression to Estimate LLM Accuracy

Suryaansh Jain, Umair Z. Ahmed, Shubham Sahai, Ben Leong

Keywords: LLM Benchmarking, Precision Estimate, Regression

TL;DR: A novel framework to estimate LLM's performance on subjective tasks by leveraging LLM based evaluators and limited human annotations

Confirmation Of Submission Requirements: I submit an abstract. It uses the template provided on the submission page and is no longer than 2 pages.

Submission Number: 242