Nonparametric Bayesian Deep Visualization

Haruya Ishizuka, Daichi Mochihashi

Abstract:

Visualizationmethodssuchast-SNE[1]havehelpedinknowl- edge discovery from high-dimensional data; however, their performance may degrade when the intrinsic structure of observations is in low-dimensional space, and they cannot estimate clusters that are often useful to under- stand the internal structure of a dataset. A solution is to visualize the latent coordinates and clusters estimated using a neural clustering model. However, they require a long computational time since they have numer-
ous weights to train and must tune the layer width, the number of latent dimensions and clusters to appropriately model the latent space. Addi- tionally, the estimated coordinates may not be suitable for visualization since such a model and visualization method are applied independently. We utilize neural network Gaussian processes (NNGP) [2] equivalent to a neural network whose weights are marginalized to eliminate the ne- cessity to optimize weights and layer widths. Additionally, to determine latent dimensions and the number of clusters without tuning, we pro- pose a latent variable model that combines NNGP with automatic rele- vance determination [3] to extract necessary dimensions of latent space and infinite Gaussian mixture model [4] to infer the number of clusters. We integrate this model and visualization method into nonparametric Bayesian deep visualization (NPDV) that learns latent and visual co- ordinates jointly to render latent coordinates optimal for visualization. Experimental results on images and document datasets show that NPDV shows superior accuracy to existing methods, and it requires less training time than the neural clustering model because of its lower tuning cost. Furthermore, NPDV can reveal plausible latent clusters without labels.