Integrating Segmentation Geometry and Large Language Models for Automated Crack Inspection

Keonvin Park, Yong Ann Voeurn, Doyun Lee

Track: long paper (up to 10 pages)

Keywords: Crack segmentation, Structural inspection, Geometric feature extraction, Large language models, Automated inspection reporting

Abstract:

Automated crack inspection is an important task in structural health monitoring and infrastructure maintenance.
Although recent deep learning methods have achieved strong performance in crack segmentation, transforming segmentation outputs into interpretable inspection reports remains challenging.
Most existing approaches primarily focus on visual detection accuracy and do not address how geometric defect information can be systematically integrated with reasoning systems.
In this work, we propose a crack inspection pipeline that integrates segmentation-derived geometric features with large language models (LLMs).
A SegFormer-based model is first used to detect crack regions from inspection images.
From the predicted crack masks, geometric attributes such as crack area, orientation, and estimated length are extracted to form structured defect descriptors.
These geometry-based descriptors are then provided to an LLM to generate structured inspection reports describing crack characteristics.
To further analyze the consistency between the measured crack geometry and the generated reports, we introduce an LLM-as-a-Judge evaluation framework that measures the alignment between the extracted geometric attributes and the language-based descriptions.
Experiments on the OmniCrack30K dataset demonstrate that the proposed pipeline can transform visual crack detections into structured textual descriptions based on geometric measurements.
Overall, this study highlights the potential of combining vision-based defect detection with language models for interpretable automated inspection workflows.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 162