Estimation of Number of Communities in Assortative Sparse Networks

Neil Hwang, Jiarui Xu, Shirshendu Chatterjee, Sharmodeep Bhattacharyya

Keywords: networks, number of communities, Bethe-Hessian, sparse networks, stochastic block model

Abstract: Most community detection algorithms assume the number of communities, $K$, to be known \textit{a priori}. Among various approaches that have been proposed to estimate $K$, the non-parametric methods based on the spectral properties of the Bethe Hessian matrices have garnered much popularity for their simplicity, computational efficiency, and robust performance irrespective of the sparsity of the input data. Recently, one such method has been shown to estimate $K$ consistently if the input network is generated from the (semi-dense) stochastic block model, when the average of the expected degrees ($\tilde{d}$) of all the nodes in the network satisfies $\tilde{d} \gg \log(N)$ ($N$ being the number of nodes in the network). In this paper, we prove some finite sample results that hold for $\tilde{d} = o(\log(N))$, which in turn show that the estimation of $K$ based on the spectra of the Bethe Hessian matrices is consistent not only for the semi-dense regime, but also for the sub-logarithmic sparse regime when $1 \ll \tilde{d} \ll \log(N)$. Thus, our estimation procedure is a robust method for a wide range of problem settings, regardless of the sparsity of the network input.

One-sentence Summary: Estimation of number of communities in sparse networks using eigenvalues of the Bethe-Hessian Matrix

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=wGmJsJqSCI