A simple measure of conditional dependence

Mona Azadkia, Sourav Chatterjee

Abstract:

We propose a coefficient of conditional dependence between two random variables Y and Z given a set of other variables
X
1
,
…
,
X
p
, based on an i.i.d. sample. The coefficient has a long list of desirable properties, the most important of which is that under absolutely no distributional assumptions, it converges to a limit in
[
0
,
1
]
, where the limit is 0 if and only if Y and Z are conditionally independent given
X
1
,
…
,
X
p
, and is 1 if and only if Y is equal to a measurable function of Z given
X
1
,
…
,
X
p
. Moreover, it has a natural interpretation as a nonlinear generalization of the familiar partial
R
2
statistic for measuring conditional dependence by regression. Using this statistic, we devise a new variable selection algorithm, called Feature Ordering by Conditional Independence (FOCI), which is model-free, has no tuning parameters, and is provably consistent under sparsity assumptions. A number of applications to synthetic and real data sets are worked out.