[Re] Identifying Through Flows for Recovering Latent Representations

Fengyuan Sun, Max van den Heuvel, Roel Klein, Tim Stolp

Keywords: identifiability, iFlow, normalizing flows, iVAE

Abstract:

Scope of Reproducibility
iFlow is more identifiable than iVAE is the main claim of the original paper. Other claims proposed are the preservation of geometry of latent sources, more accurate data modelling, and consistent identifiability across dimensions.

Methodology
The author's code was available and used to generate synthetic data and train the iFlow models, requiring only minor adjustments. Reproduced results were compared to the papers results and the model was extended upon to further test the robustness of their claims. The models were trained on an Nvidia RTX 2080 and using around 50+ GPU hours.

Results
The reproduced MCC scores in 2D do not support the claim that iFlow yields identifiability improvements over iVAE due to iVAE scoring higher. iFlow might preserve the geometry of the latent space. In contradiction to the paper, there was no case where iVAE collapsed. The results of our iFlow performance study are almost identical to that of the original paper suggesting that iFlow indeed yields improved identifiability and modelling of data distributions. The results of the final reproduction suggests in favour of the improved identifiability across dimensions of latent space. Using the best performing seed, iFlow scored around 0.25 higher on the correlation coefficient. Our extended experiments show more stability of iFlow when increasing the data complexity.

What was easy
Code was provided, making training the models generally straightforward. The code was well-structured and included descriptions of the hyperparameters. The paper provided sufficient information on the hyperparameters used.

What was difficult
No code was provided for creating the figures of the paper. Creating these ourselves required a thorough understanding of both the paper and the code. One experiment also required uncommenting some lines of code, which was changed to make the code more dynamic. Implementing a generative model using the iFlow framework proved difficult, because there is currently no method to generate new samples.

Communication with original authors
We were unable to come in contact with the authors.

Paper Url: https://openreview.net/forum?id=SklOUpEYvB