Self-Organizing Maps for the Reconstruction of Images in Pixel Permuted Image Stacks

Connor Price, Michael Kirby, Christopher Scott Peterson, David Kott

Track: Full Paper (8 pages)

Keywords: Self-Organizing Maps (SOMs), Image Reconstruction, Unsupervised Learning, Pixel Permutation, Noise Resilience, Clustering, Topology

Abstract: A color digital photograph, of resolution $a\times b$, is typically stored as an $a \times b \times 3$ array. The vector of length $3$, sitting at a pixel, encodes its color in terms of intensity measurements of the red, green, and blue color bands. We call this length $3$ vector a "pixel pole". Suppose we are given the $ab$ individual pixel poles as a jumbled collection of length $3$ vectors and we wish to reconstruct the image, with no advanced knowledge concerning its content. In other words, we are given $ab$ color squares and we wish to "solve the pixel puzzle" meaning we want to reconstruct the original unknown image. As one can imagine, this is a difficult problem. In this paper, we show how to rebuild the images in a stack of $N$ distinct $a\times b$ color digital images from the $ab$ stacked pixel poles. More precisely, this paper shows how to use "Self Organizing Maps" (SOMs) to algorithmically reconstruct, unsupervised, a stack of distinct $a\times b$ color images from the collection of $1\times 1\times 3N$ stacked pixel poles using no apriori information about the original images. We evaluate the accuracy of the reconstructions as a function of $N$, we determine the effectiveness of the algorithm when the individual images are corrupted by noise, and we assess the model's performance when pure noise images are included in the stack.

Submission Number: 34