From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMs

ACL ARR 2026 January Submission4947 Authors

Keywords: Automated Data Augmentation, Large Language Models (LLMs), Performance-Guided Optimization, Supervised Fine-Tuning (SFT), Image Transformation

Abstract: Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches. We introduce a performance-aware, closed-loop solution in the NNGPT ecosystem of projects that enables LLMs to autonomously engineer optimal transformations by internalizing empirical performance cues. We fine-tune LLMs with Low-Rank Adaptation on a novel repository of 6,000+ empirically evaluated PyTorch augmentation functions, each annotated solely by downstream model accuracy. Training uses pairwise performance ordering (better–worse transformations), enabling alignment through empirical feedback without reinforcement learning, reward models, or symbolic objectives. This reduces the need for exhaustive search, achieving up to 600$\times$ fewer evaluated candidates than brute-force discovery while maintaining competitive peak accuracy and shifting generation from random synthesis to task-aligned design. Ablation studies show that structured Chain-of-Thought prompting introduces syntactic noise and degrades performance, whereas direct prompting ensures stable optimization in performance-critical code tasks. Qualitative and quantitative analyses demonstrate that the model internalizes semantic performance cues rather than memorizing syntax. These results show that LLMs can exhibit task-level reasoning through non-textual feedback loops, bypassing explicit symbolic rewards. Upon acceptance, we will publicly release all code, prompts, and generated artifacts to support reproducibility and community use.

Paper Type: Long

Research Area: Code Models

Research Area Keywords: Generation, Language Modeling, Resources and Evaluation,

Contribution Types: Approaches to low-resource settings, Approaches low compute settings-efficiency, Data resources, Data analysis

Languages Studied: English

Submission Number: 4947