Magnum: Tackling High-Dimensional Structures with Self-Organization

PoYuan Mao, Tham Yik Foong, Danilo Vasconcellos Vargas

Keywords: self-organization, chunking, time-series data

Abstract: A big challenge for dealing with real world problems is scalability. In fact, this is partially the reason behind the success of deep learning over other learning paradigms. Here, we tackle the scalability of a novel learning paradigm proposed in 2021 based solely on self organizing principles. This paradigm consists of only dynamical equations which self-organize with the input to create attractor-repeller points that are related to the patterns found in data. To achieve scalability for such a system, we propose the Magnum algorithm, which utilizes many self-organizing subsystems (SubSigma) each with subsets of the problem's variables. The main idea is that by merging SubSigmas, Magnum builds over time a variable correlation by consensus, capable of accurately predicting the structure of large groups of variables. Experiments show that Magnum surpasses or ties with other unsupervised algorithms in all of the high-dimensional chunking problems, each with distinct types of shapes and structural features. Moreover, SubSigma alone outperforms or ties with other unsupervised algorithms in six out of seven basic chunking problems. Thus, this work sheds light in how self-organization learning paradigms can be scaled up to deal with high dimensional structures and compete with current learning paradigms.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: General Machine Learning (ie none of the above)