Extracting decision trees from trained neural networks

Olcay Boz

Abstract: Neural Networks are successful in acquiring hidden knowledge in datasets. Their biggest weakness is that the knowledge they acquire is represented in a form not understandable to humans. Researchers tried to address this problem by extracting rules from trained Neural Networks. Most of the proposed rule extraction methods required specialized type of Neural Networks; some required binary inputs and some were computationally expensive. Craven proposed extracting MofN type Decision Trees from Neural Networks. We believe MofN type Decision Trees are only good for MofN type problems and trees created for regular high dimensional real world problems may be very complex. In this paper, we introduced a new method for extracting regular C4.5 like Decision Trees from trained Neural Networks. We showed that the new method (DecText) is effective in extracting high fidelity trees from trained networks. We also introduced a new discretization technique to make DecText be able to handle continuous features and a new pruning technique for finding simplest tree with the highest fidelity.