Neuro-Symbolic Rule Lists

Sascha Xu, Nils Philipp Walter, Jilles Vreeken

Keywords: Neuro-Symbolic;Rule Induction; Intepretability

TL;DR: We introduce NyRules, an end-to-end trainable model that unifies discretization, rule learning, and rule order into a single differentiable framework.

Abstract:

Machine learning models deployed in sensitive areas such as healthcare must be interpretable to ensure accountability and fairness.
Rule lists (_**If**_ $\texttt{Age} < 35 \wedge \texttt{Priors} > 0$ _**then**_ $\texttt{Recidivism} = $True, _**else if**_ "Next Condition" ...)
offer full transparency, making them well-suited for high-stakes decisions.
However, learning such rule lists presents significant challenges. Existing methods based on combinatorial optimization require feature pre-discretization and impose restrictions on rule size. Neuro-symbolic methods use more scalable continuous optimization yet place similar pre-discretization constraints and suffer from unstable optimization. To address the existing limitations, we introduce NyRules, an end-to-end trainable model that unifies discretization, rule learning, and rule order into a single differentiable framework.
We formulate a continuous relaxation of the rule list learning problem that converges to a strict rule list through temperature annealing.
NyRules learns both the discretizations of individual features, as well as their combination into conjunctive rules without any pre-processing or restrictions.
Extensive experiments demonstrate that NyRules consistently outperforms both combinatorial and neuro-symbolic methods,
effectively learning simple and complex rules, as well as their order, across a wide range of datasets.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11239