CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization

Deep Shah, Sanket Badhe, Nehal Kathrotia, Priyanka Tiwari

Track: long paper (up to 10 pages)

Keywords: LLMs, Automatic Prompt Optimization, Efficient Inference, Textual Regularization, Chain-of-Thought Reasoning

TL;DR: CROP is an automatic prompt optimization framework that integrates a token-length regularization into the refinement loop to discover prompts that elicit highly compressed reasoning traces without sacrificing task accuracy.

Abstract: Large Language Models utilizing reasoning techniques improve task performance but incur significant latency and token costs due to verbose generation. Existing automatic prompt optimization(APO) frameworks target task accuracy exclusively at the expense of generating long reasoning traces. We propose Cost-Regularized Optimization of Prompts (CROP), an APO method that introduces regularization on response length by generating textual feedback in addition to standard accuracy feedback. This forces the optimization process to produce prompts that elicit concise responses containing only critical information and reasoning. We evaluate our approach on complex reasoning datasets, specifically GSM8K, LogiQA and BIG-Bench Hard. We achieved an 80.6\% reduction in token consumption while maintaining competitive accuracy, seeing only a nominal decline in performance. This presents a pragmatic solution for deploying token-efficient and cost-effective agentic AI systems in production pipelines.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 173