Task-driven dictionary learning in distributed online settings

Alec Koppel, Garrett Warned, Ethan Stump

Abstract: We consider task-driven dictionary learning in a decentralized dynamic setting. Here a network of agents while sequentially receiving local information aims to learn a common data-driven signal representation and model parameters. We formulate this problem as a distributed stochastic program with a non-convex objective and present a block variant of the Arrow-Hurwicz saddle point algorithm to solve it. Using Lagrange multipliers to penalize the discrepancy between them, only neighboring nodes exchange model information. We show that decisions made with this saddle point algorithm asymptotically converge to a stationarity condition in expectation under certain conditions. The learning rate depends on the signal source, network, and discriminative task. We illustrate the algorithm performance in an online multi-agent setting for a collaborative image classification task, demonstrating that the performance is comparable to the centralized case and depends on the network topology over which it is run.