FlameFinder: Illuminating Obscured Fire Through Smoke With Attentive Deep Metric Learning

Hossein Rajoli, Sahand Khoshdel, Fatemeh Afghah, Xiaolong Ma

Abstract: FlameFinder, a novel deep metric learning (DML) framework, accurately detects RGB-obscured flames using thermal images from firefighter drones during wildfire monitoring. In contrast to RGB, thermal cameras can capture smoke-obscured flame features but they lack absolute thermal reference points, detecting many nonflame hot spots as false positives. This issue suggests that extracting features from both modalities in unobscured cases can reduce the model’s bias to relative thermal gradients. Following this idea, our proposed model utilizes paired thermal-RGB images captured onboard drones for training, learning latent flame features from smoke-free samples. In testing, it identifies flames in smoky patches based on their equivalent thermal-domain distribution, improving performance with supervised and distance-based clustering metrics. The approach includes a flame segmentation method and a DML-aided detection framework with center loss (CL), triplet CL (TCL), and triplet cosine CL (TCCL), to find the optimal cluster representatives for classification. Evaluation of FLAME2 and FLAME3 datasets shows the method’s effectiveness in diverse fire and no-fire scenarios. However, the CL dominates the two other losses, resulting in the model missing features that are sensitive to them. To overcome this issue, an attention mechanism is proposed making nonuniform feature contribution possible and amplifying the critical role of cosine and triplet loss in the DML framework. Plus, the attentive DML shows improved interpretability, class discrimination, and decreased intraclass variance exploiting several other flame-related features. The proposed model surpasses the baseline with a binary classifier by 4.4% in FLAME2 and 7% in FLAME3 datasets for unobscured flame detection accuracy. It also demonstrates enhanced class separation in obscured scenarios compared to fine-tuned VGG19, ResNet18, and three other backbone models tailored for flame detection.