A divide-and-conquer dictionary learning algorithm and its performance analysis

Subhadip Mukherjee, Chandra Sekhar Seelamantula

Abstract: We address the problem of learning a sparsifying synthesis dictionary over large datasets that occur in numerous signal and image processing applications, such as inpainting, super-resolution, etc. We develop a dictionary learning algorithm that exploits the similarity of the training examples to reduce the training time. Training datasets containing correlated examples typically occur in image processing applications, as the datasets contain the patches extracted from natural images as training vectors. Our algorithm employs a divide- and-conquer approach, where one leverages the correlation within the training examples to segment the dataset into clusters containing similar examples, and learn local dictionaries for each of them. This constitutes the divide step of the algorithm. In the conquer step, a global dictionary is trained using the atoms of the local dictionaries as the training examples. We analyze the run-time complexity and the representation error of the proposed divide-and-conquer dictionary learning algorithm, and compare the performance with the batch and online dictionary learning algorithms, both on synthesized dataset and natural images. The analysis reveals that the proposed algorithm has an asymptotic complexity that is linear and logarithmic in the number of training examples, corresponding to sequential and parallel implementations, respectively.