Self-Pretraining for Small Datasets by Exploiting Patch Information

Zhang Chunyang

Keywords: Learning with Small Datasets, Self-Pretraining

Abstract: Deep learning tasks with small datasets are often tackled by pretraining models with large datasets on relevent tasks. Although pretraining methods mitigate the problem of overfitting, it can be difficult to find appropriate pretrained models sometimes. In this paper, we proposed a self-pretraininng method by exploiting patch information in the dataset itself without pretraining on other datasets. Our experiments show that the self-pretraining method leads to better performance than training from scratch both in the condition of not using other data.

One-sentence Summary: Pretraining the model using patch information in the small dataset itself

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=-UT2ecnQ6