Interpretable by Design: Boosting Neural Network Performance with Rule-Augmented Features

Keywords: interpretable machine learning, rule-augmented neural networks, explainable AI, neural-symbolic integration, tabular classification, decision stumps, hybrid architectures, accuracy-interpretability trade-off

Abstract: Deep learning models achieve high accuracy but lack interpretability, while rule-based models are interpretable but often sacrifice performance. This work addresses the accuracy-interpretability trade-off by proposing a novel pipeline that combines rule mining with neural networks for tabular classification. Our approach automatically extracts decision stump rules from training data, selects a sparse subset of effective rules, and integrates them into hybrid neural architectures. We introduce two hybrid models: HybridConcat, which concatenates rule outputs with raw features, and HybridResidual, which combines linear rule combinations with residual MLPs. Our method provides a quantifiable Pareto frontier between interpretability and performance. Experimental results on synthetic tabular data demonstrate that our hybrid models achieve superior performance compared to MLP baselines while using fewer than 6 interpretable rules. Specifically, our HybridConcat model achieves 86.32\% accuracy (+3.85\% improvement) with 3 interpretable rules providing 74.2\% sample coverage. This work contributes a systematic framework for creating interpretable yet accurate models, offering practitioners a principled approach to balance model transparency with predictive power in critical applications requiring explainable AI.

Submission Number: 260