A STUDY ON POLARITY DISTRIBUTIONS FOR NETWORK LEARNING

Published: 19 Mar 2024, Last Modified: 31 May 2024Tiny Papers @ ICLR 2024 ArchiveEveryoneRevisionsBibTeXCC BY 4.0
Keywords: Polarities, weight distribution, network training
Abstract: Transfer learning is one of the most important techniques in modern deep learning. The knowledge gained from transferring weights helps networks to learn fast achieving high accuracy. Recent work has shown that transferring the polarity of the weights plays a fundamental role in transfer learning. In this work, we concentrate on the polarity distribution and study its effects on the learning accuracy. Our results on benchmark datasets show that only the knowledge of the polarity distribution (percentage of weights having polarity positive, negative or zero) is sufficient to achieve comparable accuracy within a short training period.
Submission Number: 209
Loading