Human Perception of AI Capabilities at Classifying Perturbed Roadway Signs

Katherine R. Garcia, Jing Chen, Yanru Xiao, Scott Mishler, Cong Wang, Bin Hu

Abstract: Artificial Intelligence (AI) is crucial to numerous functions required for driving automation systems, including the computer vision techniques used to detect the roadway environment and make real-time decisions. However, the images used as inputs to the AI system may be maliciously perturbed, or manipulated, causing the AI system to make an incorrect classification. In this study, we examined humans’ perception of the AI’s computer vision capability of classifying various road sign images, including the original images, images with two different types of malicious attacks, and images that are scrambled randomly at the pixel level. Our results showed that participants rated the AI agent to be less capable than themselves of classifying the road signs. However, they overestimated the AI’s computer vision capability for correctly classifying images with malicious attacks that should cause the AI system to misclassify the image. These findings suggest that people lack an accurate understanding of the vulnerabilities of AI computer vision technologies and tend to overtrust AI in driving automation systems.

External IDs:dblp:journals/thms/GarciaCXMWH25