Causal Physics Steering in Video World Models via Concept Activation Vectors

Nahid Alam

Keywords: video world models, interpretability, steering, controllable world models, physics reasoning, concept activation vectors

TL;DR: We extract Concept Activation Vectors from a frozen VideoMAE's Physics Emergence Zone and inject them at inference time to achieve the first effective, bidirectional, training-free causal control of physical plausibility in a video world model.

Abstract: Video world models learn rich internal representations of physical dynamics, yet steering what physics governs a predicted scene at inference time remains unsolved. Recent interpretability work identified a Physics Emergence Zone (PEZ), a narrow band of middle transformer layers in VideoMAE where physical plausibility is encoded in a direction-centric population code, nearly orthogonal to other visual features. We present physics steering: a training-free method that extracts a Concept Activation Vector (CAV) from a lightweight linear probe at PEZ layers and injects it — scaled by strength $\alpha$ — into hidden states at inference time, causally shifting the model's physical expectations without modifying any weights. On the IntPhys benchmark (O1/O2/O3), physics steering achieves a 75% flip rate in physical plausibility predictions at $\alpha = +5$ and drives $P(\text{impossible})$ to 1.0 at $\alpha = +10$, with directional purity 1.00 at the PEZ. A layer-specificity ablation confirms that identical interventions at non-PEZ layers produce zero effect (flip rate 0.00 at layers 6–11), establishing the PEZ as causally necessary. Subspace analysis reveals physics and motion direction are encoded at 90° — perfectly orthogonal in representation space — confirming that physics can be steered without corrupting the model's motion representations.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 5