On Representation of Natural Image Patches

Cheng Guo

Keywords: neural coding, sparse binary code, natural image, edge detection, generative model

Abstract:

To optimize survival, organisms need to accurately and efficiently relay new information throughout their systems for processing and responses.
Furthermore, they benefit from predicting environmental occurrences, or in mathematical terms, understanding the probability distribution of their environment,
based on both personal experiences and inherited evolutionary memory.
These twin objectives of information transmission and learning environmental probabilistic distributions form the core of an organism's information processing system.
While the early vision neuroscience field has primarily focused on the former, employing information theory as a guiding framework,
the latter is largely explored by the machine learning community via probabilistic generative models.
However, the relationship between these two objectives has not been thoroughly investigated.
In this paper, we study a biologically inspired information processing model and prove that these two objectives can be achieved independently.
By evenly partitioning the input space to model input probability, our model bypasses the often intractable normalization factor computation.
When applied to image patches, this model produces a sparse, nonlinear binary population code similar to early visual systems,
with features like edge-detection and orientation-selective units.
Our results not only offer potential new insights into the functioning of neurons in early vision systems,
but also present a novel approach to represent natural image patches.

Submission Number: 12155