Cells Are Not Rectangles: A CNN Equivariant to the Symmetries of 2D Images of Things with Irregular Boundary

Anonymous

Abstract: We propose a convolutional neural network (CNN) architecture that is tailored to 2D images of "things" with irregular boundaries, such as: cells, inhomogenous materials, and flatlanders. The CNN is equivariant to E(2), i.e., (continuous) translations, rotations, and reflections in 2D Euclidean space, thus, having features and filters that represent different geometric tensors (i.e., not only scalars, but also quantities related to gradients, elongation, "pointiness", etc.). Each pixel is additionally given a "type", either interior or exterior, and retain this knowledge throughout the convolution layers. Separate convolution filters are learned for passing information within the interior, within the exterior, and in both directions across the interface. To the best of our knowledge, such a fully-equivariant treatment of the boundary of images is new. Moreover, whereas CNNs equivariant to E(2) have already been studied, they often not employed in practical situations of relevance, e.g., even in "state-of-the-art" analysis of cell images. We hope that, by describing these ideas in a didactic (and somewhat whimsical) manner, this short format paper can convince more people to use them.