Iteratively Reweighted Algorithm for Fuzzy $c$-Means

Jingjing Xue, Feiping Nie, Rong Wang, Xuelong Li

Abstract: Fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> -means method (FCM) is a popular clustering method, which uses alternating iteration algorithm to update membership matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf {F}$</tex-math></inline-formula> and center matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf {M}$</tex-math></inline-formula> of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$d \times c$</tex-math></inline-formula> size. However, original FCM suffers from finding a suboptimal local minimum, which limits the performance of FCM. In this article, we propose a new optimization method for fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> -means problem. We first propose an equivalent minimization problem of FCM, then, a simple alternating iteration algorithm is proposed to solve the new minimization problem, which involves an effective and theoretically guaranteed Iteratively Reweighted (IRW) method, so we call the new optimization method IRW-FCM. Our IRW-FCM utilizes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> not <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$dc$</tex-math></inline-formula> intermediate variables to update <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf {F}$</tex-math></inline-formula> , which can decrease space complexity. Extensive experiments including objective-function value comparison and clustering-performance comparison show that IRW-FCM can obtain better local minima than FCM with fewer iterations. And according to the time-complexity analysis, it is verified IRW-FCM has the same linear time complexity with FCM. What's more, compared with other clustering methods, IRW-FCM also shows its superiority.