Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing

Danru Xu, Sebastien Lachapelle, Sara Magliacane

TL;DR: We provide a series of identififiability results from observations only when assuming the latent variables follows potentially degenerate Gaussian mixture models with piecewise affine mixing functions.

Abstract: Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method’s effectiveness in recovering the ground-truth latent variables.

Code Dataset Promise: Yes

Code Dataset Url: https://github.com/danrux/Identi-pdGMM

Format Confirmation: I agree that I have read and followed the formatting instructions for the camera ready version.

Submission Number: 567