Multi-scale representations for natural image modeling

Sixin Zhang

Session: Energy- and score-based models (Joakim Andén)

Keywords: high-dimensional data, maximum entropy model, wavelet scattering, texture generation, generative adversarial network

Abstract: Modeling natural images has always been a difficult problem in the field of image processing and machine learning. The main difficulty lies in the complexity and diversity caused by the high dimensionality of image data. We discuss how to establish a probabilistic statistical model of high-dimensional data from the perspective of generative models in machine learning. The first part introduces how to model non-Gaussian stationary random processes under the classical maximum entropy model framework. Focusing on texture images, the difficulty of the problem lies in how to specify translation-invariant representations. We discuss the differences and connections between the representation of deep learning networks, the representation of random convolutional networks, and the representation defined by wavelet analysis. We will focus on the advantages and disadvantages of the representation of wavelet analysis in the problem of texture generation, and then extend it to the simulation of image data such as turbulence, random point processes, and astrophysics. In the second part, we discuss how to design the discriminant network of the generative adversarial network to model a class of non-Gaussian non-stationary natural images based on wavelet scattering transform.

Submission Number: 50