Adapt-Align-Combine for diffusion-based distributed dictionary learning

Dimitris Ampeliotis, Christos Mavrokefalidis, Kostas Berberidis, Sergios Theodoridis

Abstract: Diffusion-based distributed dictionary learning methods are studied in this work. We consider the classical mixed l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> cost function, that employs an l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> representation error term and an l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> sparsity promoting regularizer. First, we observe that this cost function suffers from an inherent permutation ambiguity. This ambiguity may deteriorate significantly the performance of diffusion-based schemes, since the involved combination step may combine different atoms even when the same atoms exist at all dictionaries. Thus, we propose to align the dictionaries prior to the combination step. Furthermore, we define a new problem, that we call the node-specific distributed dictionary learning problem. The proposed Adapt-Align-Combine algorithm enjoys increased convergence rate as compared with a scheme that does not align the dictionaries prior to the combination. Simulation results support our findings.