Optimizing Dataset Evaluation In Vision Tasks: Redundancy Is Out, Diversity Is In

Ziyang Li, Qin Liu, Fengshan Zhao, Yujie Wang, Takeshi Ikenaga

Abstract: Traditional evaluation methods primarily focus on how datasets enhance model performance for specific tasks, often overlooking the intrinsic quality and composition of the datasets themselves. A new dataset evaluation framework could remove redundancy and increase diversity, offering a more effective measure of dataset quality for vision tasks. This paper introduces a novel framework that refines the dataset evaluation process through the integration of data augmentation and pruning techniques, quantified by two innovative metrics: Streamline Efficiency (SE) and Enhancement Efficacy (EE). Applied to image classification and high dynamic range (HDR) reconstruction tasks, our framework first optimizes datasets using SE and EE with the Vision Transformer (ViT) and DRCT models, respectively. The optimized datasets are then validated on verification models—ResNet50 for classification and HAT for HDR reconstruction—demonstrating enhanced model performance compared to non-optimized datasets. These results confirm the effectiveness of our evaluation framework in optimizing datasets, providing actionable insights for constructing more efficient and robust datasets.

External IDs:dblp:conf/prcv/LiLZWI25