Compressive Hyperspherical Energy Minimization

Sep 25, 2019 ICLR 2020 Conference Withdrawn Submission readers: everyone
  • Abstract: Minimum hyperspherical energy (MHE) has demonstrated its potential in regularizing neural networks and improving the generalization. MHE was inspired by the Thomson problem in physics where the distribution of multiple propelling electrons on a unit sphere can be modeled via minimizing some potential energy. Despite its practical effectiveness, MHE suffers from some difficulties in optimization as the dimensionality of the space becomes higher, therefore limiting the potential to improve network generalization. To address these problems, we propose the compressive minimum hyperspherical energy (CoMHE) as a more effective regularization for neural networks. Specifically, CoMHE utilizes a projection mapping to reduce the dimensionality of neurons and minimizes their hyperspherical energy. According to different constructions for the projection mapping, we propose two major variants: random projection CoMHE and angle-preserving CoMHE. As a novel extension, We further consider adversarial projection CoMHE and group CoMHE. We also provide some theoretical insights to justify the effectiveness. Our comprehensive experiments show that CoMHE consistently outperforms MHE by a considerable margin, and can be easily applied to improve different tasks such as image recognition and point cloud recognition.
0 Replies

Loading