Verifying Prompt-Induced Search-Space Shifts in LLM-Generated Mathematical Functions

Shervin Ardeshir

Keywords: AI for Math, Automated Machine Learning Research, AI for Scientific Discovery

TL;DR: Prompt wording significantly shapes the diversity of mathematical functions generated by language models. This bias could be induced by platform and domain related key-words, and this shift can limit true exploration.

Abstract: A core step in automated discovery and agentic ML research is generating diverse mathematical functions (hypotheses), to try to solve varied problems. Large language models (LLMs) are natural tools for this task, but often regurgitate familiar patterns, especially when prompted with explicit references to known roles (e.g., 'activation function') or frameworks (e.g., PyTorch). Such inductive biases can collapse the functional search space and hinder exploration. Here we investigate how prompt phrasing induces domain-specific and platform-specific inductive biases in function generation. We compare four prompting styles across three LLMs, generating 12,000 scalar-to-scalar functions. Our analysis quantifies shifts in mathematical characteristics, revealing how seemingly minor prompt differences can significantly alter the space of functions explored.

Submission Number: 6