Value Identifiable, Credit Not: A Credit-First Identifiability Frontier for Confounded Sequential Decision-Making

Teresa Zhang

Keywords: Causal Reinforcement Learning, Temporal Credit Assignment, Proximal Causal Inference, Multi-Agent Systems, Partial Identification

Abstract: Temporal credit assignment seeks to isolate the specific decision-time interventions responsible for downstream returns. In observational trajectories, however, agent actions and environmental outcomes are frequently entangled by unobserved confounders. Consequently, standard correlation-based credit signals---such as temporal difference (TD) errors, advantage estimates, or multi-agent Shapley values---can yield systematically misleading attributions, even when aggregate policy value predictions appear highly accurate. While proximal causal inference mitigates hidden confounding for value-centric objectives via bridge-function moment restrictions, we demonstrate that value identifiability does not inherently guarantee credit identifiability. In this paper, we formalize per-timestep temporal credit as a sequential interventional estimand constrained by an explicit continuation policy. By analyzing identifiability through a discrete proximal moment equation $M\theta = y$, we establish a rigorous functional criterion: a linear functional $f^\top \theta$ is identifiable if and only if $f$ resides within the row space of the moment matrix, $f \in \text{Range}(M^\top)$. We provide an explicit finite-dimensional proof separating value identifiability from temporal-credit identifiability, demonstrating an irreducible identification width for credit even in the limit of infinite data. Empirically, we validate this predicted shrinkage-versus-plateau signature by comparing finite-sample interval widths against the oracle population width. Furthermore, we introduce a temporal responsibility metric alongside a computable row-space residual to serve as an operational ``credit identifiability risk'' diagnostic. Finally, using a controlled, tabular SCM illustrative environment, we illustrate the fundamental structural vulnerabilities of representative multi-agent reinforcement learning (MARL) baselines, showing they can exhibit systematic, sign-level misattribution under latent confounding, whereas our proximal interval credit remains demonstrably robust.

Paper Type: Full (minimum of 10 pages and a maximum of 16 excluding references)

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 11