The Gaussian-Multinoulli Restricted Boltzmann Machine: A Potts Model Extension of the GRBM

TMLR Paper6636 Authors

Abstract: Many real-world tasks, from associative memory to symbolic reasoning, benefit from discrete, structured representations that standard continuous latent models can struggle to express. We introduce the Gaussian–Multinoulli Restricted Boltzmann Machine (GM-RBM), a generative energy-based model that extends the Gaussian–Bernoulli RBM (GB-RBM) by replacing binary hidden units with $q$-state categorical (Potts) units, yielding a richer latent state space for multivalued concepts. We provide a self-contained derivation of the energy, conditional distributions, and learning rules, and describe the contrastive-divergence training procedure used in our recall and image-generation experiments. To separate architectural effects from parameter count, we evaluate GM-RBM under fixed visible-to-hidden weight budgets and additional hidden-size sweeps against GB-RBM baselines. On hetero-associative recall benchmarks, GM-RBM achieves competitive recall and, in several regimes, improved recall under fixed visible-to-hidden weight budgets in these experiments. The discrete $q$-ary formulation preserves standard RBM block updates. These results clarify when categorical hidden units provide a simple alternative to binary latents for discrete inference within tractable RBMs.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Resolved missing cross-reference to table in appendix

Code: https://github.com/ucsb-biomimetic/gmrbm

Assigned Action Editor: ~Masha_Itkina1

Submission Number: 6636