Instructions for Fill-in-the-blank Evaluation Task
Your goal is to analyze how good an artificial intelligence is at generating text that makes sense with respect to the text before and after it. You will be shown the start of a passage of text where the AI's continuation has been highlighted in yellow. You will then be asked:
Does the highlighted text make sense?
Answer "not at all" if:
the text is very ungrammatical OR
though the text is grammatical, the highlighted section makes no sense with respect to the rest of the passage.
Answer "completely" if:
the text is grammatical and smooth-flowing AND
the contents of the highlighted section seems completely reasonable given the rest of the passage.
You must adjust every slider to be able to submit the HIT. The questions you've already worked on will be marked with a ✅.