Linearithmic Clean-up for Vector-Symbolic Key-Value Memory with Kroneker Rotation Products

Ruipeng Liu, Qinru Qiu, Simon Khan, Garrett Ethan Katz

Keywords: Vector-Symbolic Architectures, Holographic Reduced Representations, Clean-up

TL;DR: Contributes a novel codebook representation and linearithmic clean-up mechanism for holographic reduced representations

Track: Main Track

Abstract: A computational bottleneck in current Vector-Symbolic Architectures (VSAs) is the "clean-up" step, which decodes the noisy vectors retrieved from the architecture. Clean-up typically compares noisy vectors against a "codebook" of prototype vectors, incurring computational complexity that is quadratic or similar. We present a new codebook representation that supports efficient clean-up, based on Kroneker products of rotation-like matrices. The resulting clean-up time complexity is linearithmic, i.e. $\mathcal{O}(N\text{log}N)$, where $N$ is the vector dimension and also the number of vectors in the codebook. Clean-up space complexity is $\mathcal{O}(N)$. Furthermore, the codebook is not stored explicitly in computer memory: It can be represented in $\mathcal{O}(\text{log}N)$ space, and individual vectors in the codebook can be materialized in $\mathcal{O}(N)$ time. At the same time, asymptotic memory capacity remains comparable to standard approaches. Computer experiments confirm these results, demonstrating several orders of magnitude more scalability than baseline VSA techniques.

Paper Type: Long Paper

Submission Number: 14