Imitation learning of dexterous hand control uncovers muscle-level representations in primate sensorimotor cortex

Alessandro Marin Vargas, Adriana Perez Rotondo, Alberto Silvio Chiappa, Mackenzie W Mathis, Alexander Mathis

Keywords: Imitation learning, representation learning, sensorimotor representations, cortical dynamics, neural decoding, dexterous manipulation, motor control, embodied control

Abstract: Dexterous grasping requires the integration of proprioceptive feedback with predictive motor commands. Yet, how cortical circuits combine afferent feedback with efference signals during grasp remains poorly understood. Here we combine deep reinforcement learning, biomechanics, and neural recordings to build a closed-loop, muscle-level controller of primate grasping. A neural network policy, trained via imitation learning on a 39-muscle musculoskeletal hand, reproduces naturalistic pre-contact shaping and develops internal dynamics that explain single-neuron activity in primary motor (M1) and somatosensory (S1) cortex. Three principles emerged. First, muscle-based control aligns more closely with cortical dynamics than joint-based control, even when the latter achieves higher tracking accuracy. Second, imitation learning drives the emergence of internal representations that capture both trial-to-trial variability and object-specific tuning. Third, trajectory embeddings learned within the modular policy can be decoded from M1 to directly drive the controller from only tens of neurons. Together, these findings establish a stimulus-computable framework that reveals integrated, muscle-centric and goal-latent codes in S1/M1 while opening a novel route for creating brain-body models.

Submission Number: 36