Transferable Recognition-Aware Image Processing

Zhuang Liu, Tinghui Zhou, Zhiqiang Shen, Bingyi Kang, Trevor Darrell

Keywords: Image Recognition, Image Processing

TL;DR: We propose simple and effective approaches to enhance the machine interpretability of image processing outputs; the improvement on recognition accuracy is transferable among different recognition architectures, object categories, and tasks.

Abstract: Recent progress in image recognition has stimulated the deployment of vision systems (e.g. image search engines) at an unprecedented scale. As a result, visual data are now often consumed not only by humans but also by machines. Meanwhile, existing image processing methods only optimize for better human perception, whereas the resulting images may not be accurately recognized by machines. This can be undesirable, e.g., the images can be improperly handled by search engines or recommendation systems. In this work, we propose simple approaches to improve machine interpretability of processed images: optimizing the recognition loss directly on the image processing network or through an intermediate transforming model, a process which we show can also be done in an unsupervised manner. Interestingly, the processing model's ability to enhance the recognition performance can transfer when evaluated on different recognition models, even if they are of different architectures, trained on different object categories or even different recognition tasks. This makes the solutions applicable even when we do not have the knowledge about future downstream recognition models, e.g., if we are to upload the processed images to the Internet. We conduct comprehensive experiments on three image processing tasks with two downstream recognition tasks, and confirm our method brings substantial accuracy improvement on both the same recognition model and when transferring to a different one, with minimal or no loss in the image processing quality.

Code: https://drive.google.com/open?id=1aMJwg7UrJ9f1aQHYBC10ZZWcZDwFYmex

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1910.09185/code)