Exploiting the implicit independence assumption for learning directed graphical models

Limin Wang, Junyang Wei, Kuo Li, Jiaping Zhou

Abstract: Bayesian network classifiers (BNCs) provide a sound formalism for representing probabilistic knowledge and reasoning with uncertainty. Explicit independence assumptions can effectively and efficiently reduce the size of the search space for solving the NP-complete problem of structure learning. Strong conditional dependencies, when added to the network topology of BNC, can relax the independence assumptions, whereas the weak ones may result in biased estimates of conditional probability and degradation in generalization performance. In this paper, we propose an extension to the k -dependence Bayesian classifier (KDB) that achieves the bias/variance trade-off by verifying the rationality of implicit independence assumptions implicated. The informational and probabilistic dependency relationships represented in the learned robust topologies will be more appropriate for fitting labeled and unlabeled data, respectively. The comprehensive experimental results on 40 UCI datasets show that our proposed algorithm achieves competitive classification performance when compared to state-of-the-art BNC learners and their efficient variants in terms of zero-one loss, root mean square error (RMSE), bias and variance.