Discovering Hidden Patterns in the Data: The Value of Input and Output Analysis in Optimizing LLM Prompt Chains

Isabel Deibel

Track: Innovations in AI for Education (Day 1)

Paper Length: short-paper (2 pages + references)

Keywords: Prompt Engineering, Hallucinations, Large Language Models, Data Analysis, AI in Education

TL;DR: Input and Output analysis provides important insights for prompt optimization and can inform prompting strategy.

Abstract: This study investigates how prompt engineering can optimize for distractor plausibility in GPT-generated fill-in-the-blank language exercises by comparing output from three different prompt chains. The findings suggest that clarity and conciseness in prompt chains may outperform more complex ways of prompting, and that linguistic patterns in the input and output provide insightful data that may be crucial for better prompting success. These insights help us to understand the impact of prompt engineering on complex prompt chains and to adjust prompting strategy in order to generate more optimal outputs.

Submission Number: 11