Error-controlled interaction discovery in deep neural networks

Winston Chen, Yifan Jiang, William Noble, Yang Young Lu

Keywords: Interpretable AI, Explainable AI, Trustworthy AI, interaction detection, FDR, reproducibility

TL;DR: The first paper to detect interactions from deep neural networks subject to error control.

Abstract: The complexity of deep neural networks (DNNs) makes them powerful but also makes them challenging to interpret, hindering their applicability in error-intolerant domains. Existing methods attempt to reason about the internal mechanism of DNNs by identifying feature interactions that influence prediction outcomes. However, such methods typically lack a systematic strategy to prioritize interactions while controlling confidence levels, making them difficult to apply in practice for scientific discovery and hypothesis validation. In this paper, we introduce a method, Diamond, to address this limitation using knockoffs, which are dummy variables that are designed to mimic the dependence structure of a given set of features while being conditionally independent of the response. Together with a novel DNN architecture involving a pairwise-coupling layer, Diamond jointly controls the false discovery rate (FDR) and maximizes statistical power. In addition, we identify a challenge in correctly controlling FDR using off-the-shelf feature interaction importance measures. Diamond overcomes this challenge by proposing a calibration procedure applicable to any existing interaction importance measures to maintain FDR control at the target level. Finally, we validate the effectiveness of Diamond through extensive experiments on simulated and real datasets.

Track: Main track

Submitted Paper: No

Published Paper: No

Submission Number: 13