Interpretable Time-Series and Trajectory Analysis for Detection of Autism-Related Motor Behaviors

Olga Isaeva, Ivan Maltsev, Anna Varlamova, Sergei Strijhak, Konstantin Gnidko, Olga Krivorotko

Keywords: autism spectrum disorder, stereotypical behavior patterns, video, dataset, skeletal keypoints, threshold-based classifiers, machine learning, PCA, FFT, metrics

TL;DR: This work focuses on the detection of stereotypical movements in children with autism spectrum disorder (ASD) based on the analysis of kinematic and spectral characteristics of motion.

Abstract:

Early detection of autism spectrum disorder (ASD) in children remains one of the most pressing challenges in modern medicine. The importance of this problem is conditioned by the fact that the effectiveness of treatment strongly depends on the age at diagnosis: early correction initiated at the youngest possible age yields the most favorable outcomes. To date, clinical experts have identified 19 stereotypical behaviors characteristic of children with ASD. Most of these behaviors are associated with cyclic movements (e.g., walking along a circular trajectory or hand flapping) as well as self-injurious actions (such as hitting the head with the hand). The manifestation of at least two stereotypical behaviors in a child’s activity is considered an indication for further clinical assessment.
The data used in this study consist of videos of children recorded by experts in controlled playroom environments. The experiments were conducted on a dataset comprising 103,000 frames from 42 videos of 18 children. Using the YOLOv11Pose model, the child’s pose is extracted from each video frame in the form of skeletal keypoints. Temporal localization of behavioral patterns is performed using a sliding window approach. The resulting skeletal keypoint time series are segmented and provided as input to classifiers for the detection of stereotypical actions.
Due to the cyclic nature of many stereotypical behaviors, the analysis of trajectories of the center of mass and individual body parts can be reduced to three main characteristics: trajectory circularity, the presence of repetitive movements, and periodicity. Each characteristic is represented by a set of interpretable features. For example, circular motion is described using features such as the mean radius, radius standard deviation, and coefficient of variation of the radius. Repetitive movements are assessed through similarity measures between trajectory segments and their one-dimensional projections obtained via Principal Component Analysis (PCA). Periodicity is characterized using frequency-domain features derived from Fast Fourier Transform (FFT) and autocorrelation analysis, including the dominant frequency, energy ratio, and the number of cycles.
The proposed approach aims to detect ASD-related stereotypical motor behaviors by extracting interpretable kinematic and spectral features from skeletal motion data. Based on the computed features, threshold-based decision rules are defined, and classical machine learning models (logistic regression and gradient boosting) as well as a fully connected neural network are trained to identify repetitive behaviors. Several actions were analyzed, including shaking hands, swinging the whole body, walking and jumping in a circular trajectory, etc. By classifying the segments of the hand trajectories (shaking/no shaking) using the threshold method, the metrics Accuracy: 0.763, Precision: 0.762, Recall: 0.800, F1: 0.780 were obtained. The use of machine learning algorithms has increased the F1 metric to values of ~ 0.8 – 0.84. The contribution of individual features to the classifiers is analyzed using feature importance measures, such SHAP. The software stack included python libraries numpy, pandas, ultralytics, sklearn и scipy.

Submission Number: 18