[Re] Pure Noise to the Rescue of Insufficient Data

Seungjae Ryan Lee, Seungmin Lee

Keywords: data augmentation, computer vision

TL;DR: Reproduce result on using pure noise to mitigate class imbalance problem

Abstract:

Scope of Reproducibility — We examine the main claims of the original paper [1], which
states that in an image classification task with imbalanced training data, (i) using pure
noise to augment minority‐class images encourages generalization by improving minority‐
class accuracy. This method is paired with (ii) a new batch normalization layer that
normalizes noise images using affine parameters learned from natural images, which
improves the model’s performance. Moreover, (iii) this improvement is robust to vary‐
ing levels of data augmentation. Finally, the authors propose that (iv) adding pure noise
images can improve classification even on balanced training data.

Methodology — We implemented the training pipeline from the description of the paper
using PyTorch and integrated authors’ code snippets for sampling pure noise images
and batch normalizing noise and natural images separately. All of our experiments were
run on a machine from a cloud computing service with one NVIDIA RTX A5000 Graphics
Card and had a total computational time of approximately 432 GPU hours.

Results — We reproduced the main claims that (i) oversampling with pure noise improves
generalization by improving the minority‐class accuracy, (ii) the proposed batch nor‐
malization (BN) method outperforms baselines, (iii) and this improvement is robust
across data augmentations. Our results also support that (iv) adding pure noise images
can improve classification on balanced training data. However, additional experiments
suggest that the performance improvement from OPeN may be more orthogonal to the
improvement caused by a bigger network or more complex data augmentation.

What was easy — The code snippet in the original paper was thoroughly documented and
was easy to use. The authors also clearly documented most of the hyperparameters that
were used in the main experiments.

What was difficult — The repo linked in the original paper was not populated yet. As a re‐
sult, we had to retrieve the CIFAR‐10‐LT dataset from previous works [2, 3], re‐implement
WideResNet [4], and the overall training pipeline.

Communication with original authors — We contacted the authors for clarifications on the
implementation details of the algorithm. Prior works had many important implemen‐
tation details such as linear learning rate warmup or deferred oversampling, so we con‐
firmed with the authors on whether these methods were used.

Paper Url: https://proceedings.mlr.press/v162/zada22a

Paper Venue: ICML 2022

Confirmation: The report pdf is generated from the provided camera ready Google Colab script, The report metadata is verified from the camera ready Google Colab script, The report contains correct author information., The report contains link to code and SWH metadata., The report follows the ReScience latex style guides as in the Reproducibility Report Template (https://paperswithcode.com/rc2022/registration)., The report contains the Reproducibility Summary in the first page., The latex .zip file is verified from the camera ready Google Colab script

Journal: ReScience Volume 9 Issue 2 Article 45

Doi: https://www.doi.org/10.5281/zenodo.8173763

Code: https://archive.softwareheritage.org/swh:1:dir:41b1ddbd87720da65e78d56dfc86b8eb81dbba56