Enhancing Dataset Sufficiency for Attributes Through Text-Driven Generative Data Augmentation

Masatoshi Sekine, Daisuke Shimbara, Tomoyuki Myojin, Eri Imatani

Student Lead Author Indication: No

Keywords: Data quality assessment, evaluation metric, attribute, text-driven, automated data augmentaion

TL;DR: We propose a method to evaluate the quality of an image dataset by introducing the classification accuracy for each word in image captions and implement an automated data augmentation strategy to optimize this method.

Abstract: Training datasets for deep learning models, including foundation models, must be both diverse and comprehensive. Therefore, the dataset should be improved to ensure overall mean accuracy and mean accuracy per classification class as well as fine-tuning attributes in the dataset. The accuracy of conventional evaluation methods based on class-wise classification can degrade for attributes other than the class, even if each class achieves high classification accuracy. Therefore, in this study, a novel evaluation metric called attribute-wise classification accuracy was proposed for classification tasks. In this model, an automated data augmentation method that constructs subsets based on individual words in image captions was used to compute and maximize classification accuracy. The proposed method generates captions corresponding to the original image dataset and expresses attribute values in the text format. Furthermore, we introduced generative automated image data augmentation based on text-driven attribute manipulation (GANDAM), an automated data augmentation method that generates interpretable new data by manipulating text. GANDAM manipulates attribute values to ensure sufficient and complete data coverage for attribute values. By learning an optimal policy to manipulate text in a manner that maximizes classification accuracy for each attribute value and maintains the naturalness of the generated text data, GANDAM optimizes data augmentation. Performance evaluation confirmed that the proposed method improved the attribute-wise classification accuracy and its mean.

Submission Number: 6