Using Logic Programming and Kernel-Grouping for Improving Interpretability of Convolutional Neural Networks

Parth Padalkar, Huaduo Wang, Gopal Gupta

Abstract: Within the realm of deep learning, the interpretability of Convolutional Neural Networks (CNNs), particularly in the context of image classification tasks, remains a formidable challenge. To this end, we present a neurosymbolic framework, NeSyFOLD-G that generates a symbolic rule-set using the last layer kernels of the CNN to make its underlying knowledge interpretable. We find groups of similar kernels in the CNN (kernel-grouping) using the cosine-similarity score between the feature maps generated by various kernels. Once such kernel groups are found, we binarize each kernel group’s output and use it to generate a rule-set using a Rule Based Machine Learning (RBML) algorithm called FOLD-SE-M. We present a novel kernel grouping algorithm and show that grouping similar kernels leads to a significant reduction in the size of the rule-set generated by FOLD-SE-M, consequently, improving the interpretability. The rule-set can be viewed as a stratified Answer Set Program wherein each predicate’s truth value depends on a kernel group in the CNN. Each predicate in the rule-set is mapped to a semantic concept using a novel semantic labeling algorithm that utilizes a few semantic segmentation masks of the images used for training. The last layers of the CNN can then be replaced by this rule-set to obtain the NeSy-G model which can then be used for the image classification task. The goal-directed ASP system s(CASP) can be used to obtain the justification of any prediction made using the NeSy-G model.