Auto-Encoding Independent Attribute Transformation for Naive Bayesian Classifier

Guiliang Ou, Yulin He, Philippe Fournier-Viger, Joshua Zhexue Huang

Abstract: To alleviate the attribute independence assumption, this paper proposes an auto-encoding naive Bayesian classifier (AE-NBC) which is trained based on transformed independent condition-attributes using an auto-encoding strategy. First, an auto-encoding neural network (AENN) with single hidden-layer is created to map the original data set to a high-dimensional data space. To guarantee the independence among auto-encoding condition-attributes (AECAs), an efficient and convergent objective function is elaborated to optimize the input-layer and output-layer weights of the AENN by considering both the auto-encoding error and attribute independence degree. Second, independent component analysis is iteratively carried out to remove the redundant attributes in AECAs so that the optimal transformed condition-attributes (TCAs) are obtained for the training of the NBC. Experiments were conducted on 29 benchmark data sets to validate the rationality, feasibility, and effectiveness of AE- NBC. Experimental results show that the independence among TCAs is stronger than among the original condition-attributes, and that AE- NBC can obtain higher training and testing accuracies than six Bayesian classifiers.