Class-specific attribute value weighting for Naive Bayes

Huan Zhang, Liangxiao Jiang, Liangjun Yu

Abstract: Naive Bayes (NB) is one of the top 10 data mining algorithms. However, its assumption of conditional independence rarely holds true in real-world applications. To alleviate this assumption, numerous attribute weighting approaches have been proposed. However, few of these simultaneously pay attention to the horizontal granularity of attribute values and vertical granularity of class labels. In this study, we propose a new paradigm for fine-grained attribute weighting, named class-specific attribute value weighting. For each class, this approach discriminatively assigns a specific weight to each attribute value. We refer to the resulting improved model as class-specific attribute value weighted NB (CAVWNB). In CAVWNB, the class-specific attribute value weight matrix is learned by either maximizing the conditional log-likelihood (CLL) or minimizing the mean squared error (MSE). Thus, two versions are proposed, which we denote as CAVWNBCLL and CAVWNBMSE, respectively. Extensive experimental results on a large number of datasets show that both CAVWNBCLL and CAVWNBMSE significantly outperform NB and all the other existing state-of-the-art attribute weighting approaches used for comparison.