Applying Computer Vision to Analyze Self-Injurious Behaviors in Children with Autism Spectrum Disorder

Ayda Eghbalian, Md Mushfiqur Azam, Katie Holloway, Leslie Neely, Kevin Desai

Abstract: Computer vision has immense potential to advance healthcare and behavioral research, particularly for conditions such as autism spectrum disorder (ASD), which is characterized by early-emerging social communication deficits and repetitive sensory-motor behaviors. Early detection of these behavioral patterns is crucial for timely diagnosis and intervention. While existing studies primarily focus on facial expressions and eye-tracking in children with ASD, the study of body gestures remains underexplored, largely due to the lack of datasets capturing full-body movements. In this work, we present a novel dataset of videos capturing full-body gestures in children with ASD, recorded during behavioral therapy sessions. The dataset specifically includes frames capturing severe behaviors such as head hitting and head banging. To establish benchmarks, we conducted baseline experiments using state-of-the-art image classification models on these action-specific frames. Furthermore, we evaluated kinematic pose estimation models on the same frames, analyzing their performance and highlighting the unique challenges faced in applying computer vision techniques to children with ASD. This dataset and our findings aim to bridge critical gaps in current research, providing a foundational resource and insights for advancing ASD-focused behavioral assessments through computer vision.

External IDs:dblp:conf/wacv/EghbalianAHND25