Segmentation of Low-Level Temporal Plume Patterns From IR Video

Rajeev Bhatt, M. Gökhan Uzunbas, Thai Hoang, Ozge C. Whiting

Abstract: In this paper, a method to segment out gas or steam plumes in IR videos collected from fixed cameras is presented. We propose a spatio-temporal U-Net architecture that captures deforming blobs of gas/steam plumes that have a unique temporal signature. In this task, the blob shapes are not semantically meaningful and change from frame to frame with no consistency across different exemplar plumes; however, there is spatial and temporal continuity in the way blobs deform suggesting a need for a low-level spatio-temporal segmentation network. The proposed method is compared to an LSTM-based segmentation network on a challenging IR video dataset collected in a controlled environment. In the controlled dataset there is motion due to steam plumes with deforming blob patterns as well as due to walking people with more structured high-level patterns. The experiments show that plume patterns are successfully segmented out with no confusion to moving people and the proposed spatiotemporal U-Net outperforms LSTM-based network in terms of pixelwise accuracy of output masks.