A Machine Learning Framework for Automated Computational Ethology Using Markerless Pose Estimation

Prateek Verma, Minh-Hao Van, Christopher McEnaney, Amy R. Poe, Xintao Wu

Submission Track: Short papers presenting ongoing research or work submitted to other venues (up to 5 pages, excluding references)

Keywords: computational ethology, behavioral classification, pose estimation, machine learning, framework, SLEAP

TL;DR: We present an end-to-end framework that bridges pose estimation with machine learning classification for automated behavioral analysis of animals.

Abstract: Quantitative behavioral analysis is fundamental to ethological research, yet automated approaches remain limited by the gap between pose estimation and meaningful behavioral classification. Most existing methods focus on either pose detection or behavior recognition in isolation, lacking integrated frameworks for comprehensive behavioral analysis. We present an end-to-end framework that bridges markerless pose estimation with machine learning classification for automated behavioral analysis. Our framework integrates SLEAP pose estimation, systematic feature engineering, multiple machine learning algorithms, and robust validation strategies into a unified pipeline. We demonstrate the framework on Drosophila larvae videos, automatically classifying three behavioral states (feeding, sleeping, crawling) from pose trajectories. We evaluate five machine learning models across three validation strategies and engineer twelve position-invariant features from four anatomical landmarks. The framework provides computational ethology researchers with practical tools for pose-based behavioral classification, comprehensive model evaluation, and deployment guidance for real-world applications.

Submission Number: 23