Identifiability of potentially degenerate Gaussian Mixture Models with piecewise affine mixing

Danru Xu, Sebastien Lachapelle, Sara Magliacane

TL;DR: We provide a series of identififiability results from observations only when assuming the latent variables follows potentially degenerate Gaussian mixture models with piecewise affine mixing functions.

Abstract:

Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function.
We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation, scaling and translation, we leverage a sparsity regularization on the learned representation.
Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method’s effectiveness in recovering the ground-truth latent variables.

Submission Number: 567