Neural Disjunctive Normal Form: Vertically Integrating Logic With Deep Learning For Classification

Jialin Lu, Martin Ester

Keywords: neuro-symbolic, hybrid, interpretability

Abstract:

We present Neural Disjunctive Normal Form (Neural DNF), a hybrid neuro- symbolic classifier that vertically integrates propositional logic with a deep neural network. Here, we aim at a vertical integration of logic and deep learning: we utilize the ability of deep neural networks as feature extractors to extract intermediate representation from data, and then a Disjunctive Normal Form (DNF) module to perform logical rule-based classification; we also seek this integration to be tight that these two normally-incompatible modules can be learned in an end-to-end manner, for which we propose the BOAT algorithm.
Compared with standard deep classifiers which use a linear model or variants of additive model as the classification head, Neural DNF provides a new choice of model based on logic rules. It offers interpretability via an explicit symbolic representation, strong model expressity, and a different type of model inductive bias. Neural DNF is particularly suited for certain tasks that require some logical composition and provides extra interpretability.

One-sentence Summary: The manuscript proposes a two-stage hybrid neuro-symbolic classifier, where a regular neural network produces binarized features that are fed into a logical rule-based classification stage, and a joint learning algorithm is provided.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=QzYC1NQnBf