An Explainable Deep Learning Model for Dental Caries Detection and Segmentation

Fadoua Drira, Imen JDEY, walid Brahmi

Abstract:

Timely diagnosis of dental caries is fundamental to preventive oral care; however, manual
interpretation of panoramic radiographs remains labor-intensive and susceptible to diagnostic
subjectivity. While Deep Learning (DL) has demonstrated high performance in medical imaging,
its clinical integration is significantly hindered by the "black-box" nature of neural networks and a
lack of alignment with clinical risk priorities, such as the high cost of false negatives. To address
these limitations, we present a clinical-risk-aware framework for automated caries detection
and instance segmentation utilizing the YOLO11-seg architecture. This pipeline enhances
generalization under real-world conditions by integrating Bayesian hyperparameter optimization
via the Optuna framework with an augmentation-robust strategy tailored for radiographic noise.
The model is rigorously evaluated on the COCO-Caries dataset, comprising 2,668 tooth-level
cropped radiographs.
Our optimized YOLO11-seg model demonstrates superior performance over YOLOv8-
seg and vanilla YOLO11-seg baselines, achieving a box-level precision of 93.8%, a recall of
75.4%, and an mAP@50 of 85.4%. Critically, these gains are realized while maintaining a
rapid inference speed of 5.2 ms and a reduced parameter count of 2.83M, facilitating realtime chairside deployment. By incorporating dual explainability techniques—Grad-CAM for
spatial saliency and LIME for model-agnostic local interpretations—the framework provides
transparent visual rationales for its predictions. This synthesis of clinical-risk-aware optimization
and interpretable Artificial Intelligence (AI) establishes a robust pipeline for dental diagnostics,
effectively bridging the gap between high-performance deep learning and clinical trust.