Task-driven Sensing with Coarse-to-Fine Glimpse-based Active Perception

Oleh Kolner, Thomas Ortner, Stanisław Woźniak, Angeliki Pantazi

Keywords: active vision, active sensing, brain-inspired

Abstract: Modern computer vision models commonly rely on passive sensing and process images in their entirety. Lacking the ability to zoom-in to task-relevant regions for detailed analysis, this approach becomes limited for high-resolution, cluttered scenes where only a small area is relevant for the task at hand. A particularly challenging problem in this context is instance detection that involves localizing specific object instances given a few visual examples. We introduce an active sensing model that uses a brain-inspired coarse-to-fine strategy to glimpse over the image by steering a retina-like sensor. The sensor uses a log-polar pixel layout that facilitates precise localization of task-relevant regions. Our model can be integrated with various state-of-the-art instance detectors. It improves their performance by up to 90%, making even small models developed for edge-devices, perform on par or even better than their large counterparts. In light of these performance gains, our model can become a complementary part in sensor hardware designs enabling active, task-driven sensing.

Submission Number: 2