Identifying Vulnerabilities and Fortifying Defenses in Automated Short-Answer Grading

Yiyun Zhou, Victoria Yaneva, Peter Baldwin, Le An Ha, Saed Rezayi, Polina Harik

Track: Responsible AI for Education (Day 2)

Paper Length: short-paper (2 pages + references)

Keywords: Automated Scoring, Adversarial Attacks, Responsible AI, Gaming Strategies

TL;DR: This paper identifies potential adversarial attacks and defense strategies in the context of a transformer-based short-answer scoring system intended for use in medical education.

Abstract: Exploitable weaknesses in automated scoring models can greatly impact the responsible use of AI systems in education. This paper identifies several potential adversarial attacks and defense strategies in the context of a transformer-based short-answer scoring system intended for use in medical education. Attacks were designed to resemble tactics that test takers might employ when they are not certain of the correct answer to a given test question. Initial results corroborate previous findings on the susceptibility of transformer-based grading systems to such attacks and show that adversarial training can significantly improve a system’s robustness.

Submission Number: 36