Model-Agnostic Shift-Equivariant Downsampling
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Shift equivariance, Shift invariance, Downsampling, Convolutional neural networks
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: The performance of convolutional neural networks (CNNs) are thought to be insensitive to image shifts. However, recent studies have revealed that downsampling layers in CNNs result in inconsistent outputs for shifted input images. In this
study, we present an approach for performing downsampling that ensures absolute shift equivariance. By employing model-agnostic downsampling method that leverages origin selection functions obtained from coordinate-independent statistics of the feature map, we can achieve perfect shift equivariance, while still adhering to the conventional downsampling procedures. Our method allows CNNs to exhibit both improved accuracy and perfect shift invariance for image classification, while also achieving shift equivariance in semantic segmentation benchmarks. Furthermore, we introduce a methodology for achieving shift equivariance without the need for any additional training process. This is accomplished by transferring pretrained weights and replacing existing layers with shift-equivariant
counterparts. Additionaly, we show that fine-tuning of the modified CNNs leads superior performance compared to previously proposed models.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 9459
Loading