Exploring a Distillation with Embedded Prompts for Object Detection in Adverse Environments

Hao Fu, Long Ma, Jinyuan Liu, Xin Fan, Risheng Liu

Abstract: Efficient and robust object detection in adverse environments is crucial and challenging for autonomous agents. The current mainstream approach is to use image enhancement or restoration as a means of image preprocessing to reduce the domain shift between adverse and regular scenes. However, these image-level methods cannot guide the model to capture the spatial and semantic information of object instances, resulting in only marginal performance improvements. To overcome this limitation, we explore a Prompts Embedded Distillation framework, called PED. Specifically, a spatial location prompt module is proposed to guide the model to learn the easily missed target position information. Considering the correlation between object instances in the scene, a semantic mask prompt module is proposed to constrain the global attention between instances, making each aggregated instance feature more discriminative. Naturally, we propose a teacher model with embedded cues and finally transfer the knowledge to the original student model through focal distillation. Extensive experimental results demonstrate the effectiveness and flexibility of our approach.