UAV Imaging: Correlation Between Contrast and F1-Score for Vision-Based Crack Detection

Tran Hiep Dinh, Cong Hieu Le, Quang Phuc Ha

Abstract: Image segmentation is one of the critical tasks in UAV-based infrastructure monitoring. The evaluation of a segmentation is conducted by comparing the ground truth annotated by a human expert to a resulting image by an algorithm. In the absence of the ground truth, it is not easy to evaluate the performance of the deployed models. Hence come empirical goodness metrics, where segmented images are compared with the original one using statistical measures. In this paper, we analyze the correlation between the $F_{1}$ score and a well-known ground-truth-independent metric, the region contrast $C$, from the segmentation results of 9 crack detection models on the SYDCrack dataset. Experimental results have confirmed the alignment between $c$ and $F_{1}$ regarding model ranking. However, the correlation analysis does not show a linear dependence between the two metrics on observations of all models. In fact, the region number of segmentation is highly correlated to both evaluation metrics. This might shed some light on developing an empirical goodness metric for vision-based crack detection.