Systematic Analysis of Cluster Similarity Indices: How to Validate Validation Measures

Martijn Gösgens, Liudmila Prokhorenkova, Aleksei Tikhonov

Keywords: cluster similarity indices, cluster validation, clustering, community detection, constant baseline

Abstract: There are many cluster similarity indices used to evaluate clustering algorithms, and choosing the best one for a particular task remains an open problem. We demonstrate that this problem is crucial: there are many disagreements among the indices, these disagreements do affect which algorithms are chosen in applications, and this can lead to degraded performance in real-world systems. We propose a theoretical solution to this problem: we develop a list of desirable properties and theoretically verify which indices satisfy them. This allows for making an informed choice: given a particular application, one can first make a selection of properties that are desirable for a given application and then identify indices satisfying these. We observe that many popular indices have significant drawbacks. Instead, we advocate using other ones that are not so widely adopted but have beneficial properties.

One-sentence Summary: Provide a systematic theoretical analysis of cluster similarity indices: define a number of properties that are desirable across many applications and check them for a number of known indices.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=AGKL695bq3