CausalAffect: Causal Discovery for Facial Affective Understanding

ICLR 2026 Conference Submission20783 Authors

Keywords: Affective Analysis, emotion recognition, action units, basic emotions

Abstract:

Understanding human affect from facial behavior requires not only accurate recognition but also structured reasoning over the latent dependencies that drive muscle activations and their expressive outcomes.
Although Action Units (AUs) have long served as the foundation of affective computing, existing approaches rarely address how to infer psychologically plausible causal relations between AUs and expressions directly from data.
We propose CausalAffect, the first framework for causal graph discovery in facial affect analysis.
CausalAffect models AU$\rightarrow$AU and AU$\rightarrow$Expression dependencies through a two-level polarity and direction aware causal hierarchy that integrates population-level regularities with sample-adaptive structures.
A feature-level counterfactual intervention mechanism further enforces true causal effects while suppressing spurious correlations.
Crucially, our approach requires neither jointly annotated datasets nor handcrafted causal priors, yet it recovers causal structures consistent with established psychological theories while revealing novel inhibitory and previously uncharacterized dependencies.
Extensive experiments across six benchmarks demonstrate that CausalAffect advances the state of the art in both AU detection and expression recognition, establishing a principled connection between causal discovery and interpretable facial behavior.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 20783