Restricted subgradient descend method for sparse signal learning

Jiajun Wen, Wai Keung Wong, Xiao-Li Hu, Hong-Lin Chu, Zhihui Lai

Abstract: The sparse signal learning is essentially a sparse solution optimization problem. This technique is especially applicable to the field of signal recovery, e.g. image reconstruction. Such a problem can be solved by the gradient or subgradient descend method. However, conventional method normally needs to introduce extra quadratic term to construct complex objective function, whose solution costs many iteration steps. To address this problem, this paper proposes a novel method called restricted subgradient descend to learn the sparse signals. Our idea is based on the fact that the subgradient of 1-norm function exits at any n-dimensional point, and such a function even can obtain the gradient on the point without zero coordinate components. Thus, to decrease the objective function with regard to 1-norm value, the gradient or subgradient direction can be used to search next update of estimation, which facilitates the learning of the proposed method for high quality sparse solution with quick convergence time. Specifically, two algorithms are proposed, among which the first one uses merely restricted subspace projection scheme and the refined one is based on an improved version of the pivot step of simplex algorithm. It is analyzed that the refined algorithm is able to learn exactly the source sparse signal in finite iteration steps if the subgradient condition is satisfied. This theoretical result is also verified by numerical simulation with good experimental results compared with other state-of-the-art sparse signal learning algorithms.