TT-Sparse: Learning Sparse Rule Models with Differentiable Truth Tables

Hans Farrell Soegeng, Sarthak Ketanbhai Modi, Thomas Peyrin

TL;DR: TT-Sparse is a differentiable neural block built on Learnable Truth Tables and a novel relaxed TopK operator, enabling end-to-end learning of exact Boolean rule sets that establish a new performance-complexity Pareto frontier across tabular tasks.

Abstract: Interpretable machine learning is essential in high-stakes domains where decision-making requires accountability, transparency, and trust. While rule-based models offer global and exact interpretability, learning rule sets that simultaneously achieve high predictive performance and low, human-understandable complexity remains challenging. To address this, we introduce TT-Sparse, a flexible neural building block that leverages differentiable truth tables as nodes to learn sparse, effective connections. A key contribution of our approach is a new soft TopK operator with straight-through estimation for learning discrete, cardinality-constrained feature selection in an end-to-end differentiable manner. Crucially, the forward pass remains sparse, enabling each node (and the entire model) to be transformed exactly into compact, globally interpretable DNF/CNF Boolean formulas via Quine--McCluskey minimization. Extensive empirical results across 28 datasets spanning binary, multiclass, and regression tasks show that the learned sparse rules exhibit superior predictive performance with lower complexity compared to existing state-of-the-art methods.

Lay Summary: As AI is widely adopted in decision-making, it becomes increasingly important now to understand why it makes its decisions, especially in critical applications such as approving loans, diagnosing diseases, assessing criminal risk, etc. Most high-performing models are "black-boxes" whose decisions cannot be explained. We developed TT-Sparse, a system that learns compact sets of logical rules (e.g. "if blood pressure is above 120 AND age is over 60, then the risk of heart disease is elevated by x") from data by using a neural network training process. A key innovation is a new method for selecting which features each rule examines, keeping rules short and human-readable while maintaining strong predictive accuracy. After training, the learned rules can be extracted exactly -- completely reproducing the model's predictions -- allowing us to use the understandable rules in place of the model. Across tabular datasets spanning different domains and task types (classification or regression), TT-Sparse produces rules that are both more accurate and simpler than existing interpretable methods while staying competitive with the state-of-the-art black box tabular model.

Link To Code: https://github.com/hansfarrell/tt-sparse

Primary Area: General Machine Learning

Keywords: Interpretable AI, top-k, neuro-symbolic AI, rule learning, differentiable truth tables, sparse neural networks, Boolean logic

Submission Number: 1698