Mt-Gan: A Training Framework To Enhance Image Classification Task With Image Translation

Qun Li, Changbo Hu, Keng-hao Chang, Ruofei Zhang

Abstract: The emerging e-commerce ecosystem has made image understanding, particularly fashion image attributes (e.g., pattern, color, material, category) recognition and classification, a prerequisite for many downstream applications such as visually similar product retrieval and recommendation. We propose a training framework named Multi-Task Generative Adversarial Network (MT-GAN) that trains an image classifier as its discriminator and proves to improve the multi-class multilabel image classification task performance by 11.54% on average. A key contributor to the performance improvement is the feeding of the conditional input image to the image classifier in addition to the image to be classified, which enhances the classification performance by 7.36% relatively compared to the same classifiers trained in the same MT-GAN setting without being fed the conditional input. The proposed MTGAN training framework is GAN-agnostic and can be applied to image classification tasks in a supervised learning manner with various state-of-the-art designs of the generator.