[Re] Pure Noise to the Rescue of Insufficient DataDownload PDF

Published: 02 Aug 2023, Last Modified: 02 Aug 2023MLRC 2022 OutstandingPaperReaders: Everyone
Keywords: data augmentation, computer vision
TL;DR: Reproduce result on using pure noise to mitigate class imbalance problem
Abstract: Scope of Reproducibility — We examine the main claims of the original paper [1], which states that in an image classification task with imbalanced training data, (i) using pure noise to augment minority‐class images encourages generalization by improving minority‐ class accuracy. This method is paired with (ii) a new batch normalization layer that normalizes noise images using affine parameters learned from natural images, which improves the model’s performance. Moreover, (iii) this improvement is robust to vary‐ ing levels of data augmentation. Finally, the authors propose that (iv) adding pure noise images can improve classification even on balanced training data. Methodology — We implemented the training pipeline from the description of the paper using PyTorch and integrated authors’ code snippets for sampling pure noise images and batch normalizing noise and natural images separately. All of our experiments were run on a machine from a cloud computing service with one NVIDIA RTX A5000 Graphics Card and had a total computational time of approximately 432 GPU hours. Results — We reproduced the main claims that (i) oversampling with pure noise improves generalization by improving the minority‐class accuracy, (ii) the proposed batch nor‐ malization (BN) method outperforms baselines, (iii) and this improvement is robust across data augmentations. Our results also support that (iv) adding pure noise images can improve classification on balanced training data. However, additional experiments suggest that the performance improvement from OPeN may be more orthogonal to the improvement caused by a bigger network or more complex data augmentation. What was easy — The code snippet in the original paper was thoroughly documented and was easy to use. The authors also clearly documented most of the hyperparameters that were used in the main experiments. What was difficult — The repo linked in the original paper was not populated yet. As a re‐ sult, we had to retrieve the CIFAR‐10‐LT dataset from previous works [2, 3], re‐implement WideResNet [4], and the overall training pipeline. Communication with original authors — We contacted the authors for clarifications on the implementation details of the algorithm. Prior works had many important implemen‐ tation details such as linear learning rate warmup or deferred oversampling, so we con‐ firmed with the authors on whether these methods were used.
Paper Url: https://proceedings.mlr.press/v162/zada22a
Paper Venue: ICML 2022
Supplementary Material: zip
Confirmation: The report pdf is generated from the provided camera ready Google Colab script, The report metadata is verified from the camera ready Google Colab script, The report contains correct author information., The report contains link to code and SWH metadata., The report follows the ReScience latex style guides as in the Reproducibility Report Template (https://paperswithcode.com/rc2022/registration)., The report contains the Reproducibility Summary in the first page., The latex .zip file is verified from the camera ready Google Colab script
Latex: zip
Journal: ReScience Volume 9 Issue 2 Article 45
Doi: https://www.doi.org/10.5281/zenodo.8173763
Code: https://archive.softwareheritage.org/swh:1:dir:41b1ddbd87720da65e78d56dfc86b8eb81dbba56
0 Replies