Better Counterfactual Model Reasoning with Submodular Quadratic Component Models
Keywords: Component Attribution, Component Modeling, Boolean Analysis, Submodular Maximization
Abstract: In this work, we propose a new heuristic for *component modeling*---the task of building a model that estimates the effect of model components on model behavior. Empirically, we show that even at scale, *actively* learning a quadratic model instead of a linear model increases accuracy and decreases sample complexity. Prior work has favored a linear model, because outside of accuracy, interpretability is a key desiderata for a component model. By exploiting properties of the degree 2 Fourier representation, we derive an individual influence for each point, that strictly generalizes the coefficient in a linear data model. This notion corresponds to the discrete derivative of the function at a given point, and has the benefit that it incorporates information about the rest of the dataset into the individual influence estimate. We also introduce the idea of enforcing submodularity, which theoretically can allow for better and nontrivial optimization for interesting counterfactual reasoning tasks with set cardinality constraints.
Submission Number: 86
Loading