When Post-Hoc Rotation Cannot Replace Non-Negativity: A Boundary for Low-Rank Factor Recovery Through a Nonlinear Link

Keywords: latent space models, non-negative matrix factorization, network embeddings, mixed membership, Varimax rotation, Bernoulli--Poisson link, identifiability

TL;DR: Through a nonlinear link, post-hoc Varimax rotation of an unconstrained fit recovers a non-negative factor model's column directions but not its sparsity or heavy tails — the link destroys the leptokurtic structure Rohe–Zeng identifiability needs.

Abstract:

Rohe \& Zeng (2023) showed that an unconstrained spectral fit followed by a Varimax rotation recovers the latent factors of a broad class of linear low-rank models---including the stochastic blockmodel and latent Dirichlet allocation---when the factors are leptokurtic. We ask whether this recipe survives a nonlinear link between latent factors and observation, using two rank-$K$ factor models for binary matrices that share an inner-product core but differ in link and constraint: the latent space model (signed, logistic link) of Hoff
et al. (2002) and the edge partition model (non-negative, Bernoulli--Poisson link) of Zhou (2015). Post-hoc Varimax aligns the LSM fit with the correct axes in moderately dense networks containing anchor nodes, but the resulting embedding remains partly negative, only weakly sparse, and Gaussian-like in shape at every density we test, never matching the non-negative, sparse, heavy-tailed loadings of the constrained fit. Although the LSM-MLE Gram is empirically close to a nonlinear transformation of $\Theta\Theta^\top$ (Gram-vs-Gram correlation $0.82$--$0.88$), the column-wise leptokurtosis of $\Theta$ on which the identifying assumption depends is not inherited by the LSM-MLE fit; Varimax cannot manufacture leptokurtic axes when the column space it operates on does not contain any. This gives a boundary for the spectral-plus-rotation recipe and a structural reason why constraint-based factorizations cannot be fully replaced by post-hoc rotation of an unconstrained fit.

Submission Number: 56