Transferring Movement Understanding for Parkinson’s Therapy by Generative Pre-Training

Emily Napier, Gavia Gray, Tristan Loria, Veronica Vuong, Michael Thaut, Sageev Oore

Keywords: language modeling, Parkinson's, motion capture, sequence modeling, GPT, scaling laws

TL;DR: Uniformly quantize motion to turn it into text and train language models on it for fine-tuning on clinical tasks

Abstract:

Motion data is a modality of clinical importance for Parkinson's research but modeling it typically requires careful design of the machine learning system. Inspired by recent advances in autoregressive language modeling, we investigate the extent to which these modeling assumptions may be relaxed. We quantize motion capture data into discrete tokens and apply a generic autoregressive model
to learn a model of human motion. Representing both positions and joint angles in a combined vocabulary, we model forward and inverse kinematics in addition to autoregressive prediction in 3D and angular space. This lets us pre-train on a 1B token, 40 hour dataset of motion capture, and then finetune on one hour of clinically relevant data in a downstream task. Despite the naivety of this approach, the model is able to perform clinical tasks and we demonstrate high performance classifying 5 hours of dance data.

Submission Number: 51