Multilayer Matrix Factorization via Dimension-Reducing Diffusion Variational Inference

Junbin Liu, Farzan Farnia, Wing-Kin Ma

Abstract:

Multilayer matrix factorization (MMF) has recently emerged as a generalized model of, and potentially a more expressive approach than, the classic matrix factorization.
This paper considers MMF under a probabilistic formulation, and our focus is on inference methods under variational inference.
The challenge in this context lies in determining a variational process that leads to a computationally efficient and accurate approximation of the maximum likelihood inference.
One well-known example is the variational autoencoder (VAE), which uses neural networks for the variational process.
In this work, we take insight from variational diffusion models in the context of generative models to develop variational inference for MMF.
We propose a dimension-reducing diffusion process that results in a new way to interact with the layered structures of the MMF model.
Experimental results demonstrate that the proposed diffusion variational inference method leads to improved performance scores compared to several existing methods, including the VAE.

Lay Summary: Matrix factorization is a computational technique that helps researchers analyze high-dimensional data and uncover simpler, more understandable patterns. Recently, researchers developed multilayer matrix factorization (MMF), which uses multiple layers to retrieve patterns from complex data. The challenge is that these multilayer models are computationally difficult to handle. To address this, we drew inspiration from diffusion models, a powerful generative AI technique that has revolutionized image generation. We developed a new MMF method that borrows key insights from diffusion models. Our approach systematically breaks down the problem layer by layer. We tested our method against established techniques like the variational autoencoders (VAE), and our approach delivered promising results. This advancement may give researchers a more powerful tool for uncovering hidden structures in complex high-dimensional data. The potential applications are broad, from analyzing remote sensing images to isolating individual voices in audio recordings.

Link To Code: https://github.com/Szanzang/DRDVI

Primary Area: Probabilistic Methods->Variational Inference

Keywords: Multilayer matrix factorization, Variational inference, Variational diffusion models, Dimension reduction

Submission Number: 16092