Adapting, Fast and Slow: Transportable Circuits for Few-Shot Learning

Kasra Jalaldoust, Elias Bareinboim

Keywords: Causal inference, Transportability, Domain Generalization, Domain Adaptation, Few-shot Learning

TL;DR: a causal framework for compositional generalization and supervised domain adaptation.

Abstract: Generalization across the domains is not possible without asserting a structure that constrains the unseen target domain w.r.t. the source domain. Building on causal transportability theory, we design an algorithm for zero-shot compositional generalization which relies on access to qualitative domain knowledge in form of a causal graph for intra-domain structure and discrepancies oracle for inter-domain mechanism sharing. Circuit-TR learns a collection of modules (i.e., local predictors) from the source data, and transport/compose them to obtain a circuit for prediction in the target domain if the causal structure licenses. Furthermore, circuit transportability enables us to design a supervised domain adaptation scheme that operates without access to an explicit causal structure, and instead uses limited target data. Our theoretical results characterize classes of few-shot learnable tasks in terms of graphical circuit transportability criteria, and connects few-shot generalizability with the established notion of circuit size complexity; controlled simulations corroborate our theoretical results.

Primary Area: causal reasoning

Submission Number: 22323