Do Users Write More Insecure Code with AI Assistants?

Neil Perry, Megha Srivastava, Deepak Kumar, Dan Boneh

Keywords: code generation models, language models, security, user study, safety, human-AI interaction

TL;DR: We conduct a user study examining whether users interacting with an AI Code assistant are more likely to introduce security vulnerabilities in code than those without.

Abstract:

We conduct the first large-scale user study examining how users interact with an AI Code assistant to solve a
variety of security related tasks across different programming
languages. Overall, we find that participants who had access
to an AI assistant based on OpenAI’s \texttt{codex-davinci-002}
model wrote less secure code than those without
access. Additionally, participants with access to an AI assistant
were more likely to believe they wrote secure code than those
without access to the AI assistant. Furthermore, we find that
participants who trusted the AI less and engaged more with
the language and format of their prompts (e.g. re-phrasing,
adjusting temperature) provided code with fewer security
vulnerabilities. Finally, in order to better inform the design
of future AI Assistants, we provide an in-depth
analysis of participants’ language and interaction behavior, as
well as release our user interface as an instrument to conduct
similar studies in the future.

Submission Number: 62