A STUDY ON POLARITY DISTRIBUTIONS FOR NETWORK LEARNING

Aoife Igoe, Arindam Biswas, BISWAJIT BASU

Keywords: Polarities, weight distribution, network training

Abstract: Transfer learning is one of the most important techniques in modern deep learning. The knowledge gained from transferring weights helps networks to learn fast achieving high accuracy. Recent work has shown that transferring the polarity of the weights plays a fundamental role in transfer learning. In this work, we concentrate on the polarity distribution and study its effects on the learning accuracy. Our results on benchmark datasets show that only the knowledge of the polarity distribution (percentage of weights having polarity positive, negative or zero) is sufficient to achieve comparable accuracy within a short training period.

Submission Number: 209