DistHD: A Learner-Aware Dynamic Encoding Method for Hyperdimensional Classification

Junyao Wang, Sitao Huang, Mohsen Imani

Abstract: The Internet of Things (IoT) has become an emerging trend that connects heterogeneous devices and enables them with new capabilities. Many applications exploit machine learning methodology to dissect collected data, and edge computing was introduced to enhance the efficiency and scalability in resource-constrained computing environments. Unfortunately, popular deep learning algorithms involve intensive computations that are overcomplicated for edge devices. Brain-inspired Hyperdimensional Computing (HDC) has been considered a promising approach to address this issue. However, existing HDC methods use static encoders, and thus require extremely high dimensionality and hundreds of training iterations to achieve reasonable accuracy. This results in a huge loss of efficiency and severely impedes the application of HDC algorithms in power-limited machines. In this paper, we propose DistHD, a novel HDC framework with a unique dynamic encoding technique consisting of two parts: top-2 classification and dimension regeneration. Our top-2 classification provides top-2 labels for each data sample based on cosine similarity, and dimension regeneration identifies and regenerates dimensions that mislead the classification and reduce the learning quality. The highly parallel algorithm of DistHD effectively accelerates the learning process and achieves the desired accuracy with considerably lower dimensionality. Our evaluation on a wide range of practical classification tasks shows that DistHD is capable of achieving on average 2.12% higher accuracy than state-of-the-art (SOTA) HDC approaches while reducing dimensionality by 8.0×. It delivers 5.97× faster training and 8.09× faster inference than SOTA learning algorithms. Additionally, the holographic distribution of patterns in high dimensional space provides DistHD with 12.90× higher robustness against hardware errors than SOTA DNNs. DistHD has been open-sourced to enable future research in this field. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>