A Modular Abstraction for Integrating Domain Rules into Deep Learning Models

TMLR Paper5656 Authors

Abstract: Domain-specific knowledge can often be expressed as suggestive rules defined over subgroups of data. Such rules, when encoded as hard constraints, are often not directly compatible with deep learning frameworks that train neural networks over batches of data. Also, domain-experts often use heuristics that should not be encoded as logical rules. In this work, we propose a framework to capture domain-experts' knowledge as domain-specific rules over subgroups of data, and to leverage such rules in training deep learning models using the modular components of regularization, data augmentation, and parameter optimization. This translation of domain knowledge into custom primitives that can be augmented to existing state-of-the-art deep learning models improves the ability of domain experts to interpret and express model behavior, intervene through changes in the modeling specifications, and improve the overall performance of the model as compared to existing frameworks that incorporate deterministic declarative predicates. On one synthetic and three real-world tasks, we show that our method allows iterative refinement and is demonstrably more accurate.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

In this revision, we have made the following key updates:

1. **Additional Experimental Results:** We have expanded our empirical evaluation by adding new tables in the Appendix reporting training errors and overfitting trends across the synthetic, MIMIC-III, and CarRacing-v0 tasks. These results complement the main experiments and address reviewer feedback on model overfitting and training performance.

2. **New Background and Notations Section:** We have added a formal subsection introducing definitions for empirical risk minimization, adversarial robustness, and probabilistic robustness to improve theoretical clarity and provide a unified foundation for our proofs.

3. **Reorganization and Clarity Improvements:** We have moved implementation details and proofs to the main paper for better accessibility, improved notation consistency, and conducted a thorough proofreading of the manuscript.

4. **Expanded Related Work:** We have added new references to prior work in neuro-symbolic learning and domain-specific modeling to better situate our framework within existing literature.

Assigned Action Editor: ~Emanuele_Sansone1

Submission Number: 5656