Neuromorphic luminance-edge contextual preprocessing of naturally obscured targets

Alexander James White, Chou P. Hung, Andre V. Harrison, Chung-Chuan Lo

Abstract: Contextual grouping mechanisms in early visual cortex are thought to aid in perception of ambiguous textures, including partially obscured targets under real-world high dynamic range (HDR) luminance. Yet, deep neural networks struggle with naturalistic obscuration and illumination while requiring millions of neurons and power-hungry GPUs for processing. We hypothesized that contextual grouping mechanisms for edge and luminance processing may aid in localization of targets under natural obscuration and illumination. To address this issue, we developed a novel small (< 10,000 neurons) spiking neural network (SNN) that uses spike time correlations to leverage the combined luminance and orientation similarity of nearby image regions for image pre-processing, to support downstream deep neural network (DNN) target localization. The network has leaky integrate-and-fire neurons with current based (CuBa) synapses and is simulated using the Nengo LOIHI API, with potential application via Intel's LOIHI neuromorphic hardware. We collected 89 HDR images of a target dummy in a heavily wooded environment under varying occlusion and illumination. We used SNN preprocessing to adjust local image contrast based on the grouping mechanism, followed by a DNN classifier (Detectron2) to localize the target. Results show that a small SNN for image preprocessing can aid image segmentation and localization of occluded targets, marking an initial step towards more efficient and accurate target recognition under natural illumination and occlusion.