A Base Model Selection Methodology for Efficient Fine-Tuning

Yosuke Ueno, Masaaki Kondo

Keywords: transfer learning, fine-tuning, parameter transfer

TL;DR: We propose several metrics to estimate the transferability of pre-trained CNN models for a given target task.

Abstract:

While the accuracy of image classification achieves significant improvement with deep Convolutional Neural Networks (CNN), training a deep CNN is a time-consuming task because it requires a large amount of labeled data and takes a long time to converge even with high performance computing resources.
Fine-tuning, one of the transfer learning methods, is effective in decreasing time and the amount of data necessary for CNN training. It is known that fine-tuning can be performed efficiently if the source and the target tasks have high relativity.
However, the technique to evaluate the relativity or transferability of trained models quantitatively from their parameters has not been established. In this paper, we propose and evaluate several metrics to estimate the transferability of pre-trained CNN models for a given target task by featuremaps of the last convolutional layer.
We found that some of the proposed metrics are good predictors of fine-tuned accuracy, but their effectiveness depends on the structure of the network. Therefore, we also propose to combine two metrics to get a generally applicable indicator.
The experimental results reveal that one of the combined metrics is well correlated with fine-tuned accuracy in a variety of network structure and our method has a good potential to reduce the burden of CNN training.

Code: https://www.dropbox.com/s/ry1gh8bf4yy4qe7/iclr2020_code_submission.tar.gz?dl=0