A Unifying Framework for Causal Imitation Learning with Hidden Confounders

Daqian Shao, Thomas Kleine Buening, Marta Kwiatkowska

Track: regular paper (up to 6 pages)

Keywords: causal imitation learning, hidden confounders, instrumental variables

TL;DR: We propose a unifying framework for causal imitation learning and design an algorithm to address suprious correlations and hidden confounders via IV regression.

Abstract: We propose a general and unifying framework for causal Imitation Learning (IL) with hidden confounders that subsumes several existing settings. Our framework accounts for two types of hidden confounders: (a) those observed by the expert but not the imitator, and (b) confounding noise hidden to both. By leveraging trajectory histories as instruments, we reformulate causal IL into Conditional Moment Restrictions (CMRs). We propose DML-IL, an algorithm that solves these CMRs via instrumental variable regression, and upper bound its imitation gap. Empirical evaluation on continuous state-action environments, including Mujoco tasks, shows that DML-IL outperforms state-of-the-art causal IL methods.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 5