As Generative Models Improve, People Adapt Their Prompts

Eaman Jahani, Benjamin S. Manning, Joe Zhang, Hong-Yi TuYe, Mohammed Alsobay, Christos Nicolaides, Siddharth Suri, David Holtz

Abstract: As generative AI systems rapidly improve, a key question emerges: How do users keep up-and what happens if they fail to do so. Drawing on theories of dynamic capabilities and IT complements, we examine prompt adaptation-the adjustments users make to their inputs in response to evolving model behavior-as a mechanism that helps determine whether technical advances translate into realized economic value. In a preregistered online experiment with 1,893 participants, who submitted over 18,000 prompts and generated more than 300,000 images, users attempted to replicate a target image in 10 tries using one of three randomly assigned models: DALL-E 2, DALL-E 3, or DALL-E 3 with automated prompt rewriting. We find that users with access to DALL-E 3 achieved higher image similarity than those with DALL-E 2-but only about half of this gain (51%) came from the model itself. The other half (49%) resulted from users adapting their prompts in response to the model's capabilities. This adaptation emerged across the skill distribution, was driven by trial-and-error, and could not be replicated by automated prompt rewriting, which erased 58% of the performance improvement associated with DALL-E 3. Our findings position prompt adaptation as a dynamic complement to generative AI-and suggest that without it, a substantial share of the economic value created when models advance may go unrealized.