Disentangled Skill Representations for Predictive Human Modeling

Mariah L Schrum, Srijan Srivatsa, Deepak Edakkattil Gopinath, Guy Rosman, Tiffany L. Chen

Keywords: Representation Learning, human skill modeling, Interpretable Latents, Counterfactual Training

TL;DR: We introduce a method for learning disentangled, interpretable skill representations from behavior, using expert–novice basis blending and counterfactual subskill swaps to yield stable embeddings that predict and generalize across domains.

Abstract: Understanding human skill is essential for AI systems that collaborate with, coach, or assist people. Unlike typical latent variable estimation problems—which rely on single observations or explicit labels—skill is a persistent, compositional, and behaviorally grounded construct that must be inferred from patterns over time. We introduce Skill Abstraction with Interpretable Latents (SAIL), a method for learning disentangled skill representations from naturalistic behavioral data. Our approach produces a skill embedding that is robust to spurious performance fluctuations and captures core, transferable representation of human subskills. Furthermore, SAIL supports skill-informed behavior prediction that generalizes across a variety of contexts. We represent each individual with a persistent skill embedding that controls a blend between expert and novice behavior bases and is trained using counterfactual subskill swaps for disentanglement. This design yields a representation that is both robust to performance variation and structured for interpretability. We demonstrate that SAIL outperforms prior methods across two domains—high-performance driving and baseball batting—producing skill representations that are stable, predictive, and interpretable.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 6393