Adapting ConvNets for New Cameras without Retraining
Keywords: Convolutional Networks, Pretrained, Wide FOV, Fisheye, Segmentation, Rectification
TL;DR: In this work we adapt pretrained CNNs to improve performance for wide-fov cameras, without any additional training or data required.
Abstract: In the vast majority of research, it is assumed images will be perspective or can be rectified to a perspective projection. However, in many applications it is beneficial to use non conventional cameras, such as fisheye cameras, that have a larger field of view (FOV). The issue arises that these large FOV images can't be rectified to a perspective projection without significant cropping of the original image. To address this issue we propose Rectify Convolutions (RectConv); a new approach for adapting pre-trained convolutional networks to operate with new non-perspective images, without any retraining. Replacing the convolutional layers of the network with RectConv layers allows the network to see both rectified patches and the entire FOV. We demonstrate RectConv adapting multiple pre-trained networks to perform segmentation and detection on fisheye imagery from two publicly available datasets. Our method shows improved results over both direct application of the network and naive pre-rectification of imagery. Our approach requires no additional data or training, and we develop a software tool that transforms existing pre-trained networks to operate on new camera geometries. We believe this work is a significant step toward adapting the vast resources available for perspective images to operate across a broad range of camera geometries. Code available upon acceptance.
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 9231
Loading