Summaries as Centroids for Interpretable and Scalable Text Clustering

ICLR 2026 Conference Submission15924 Authors

Keywords: Text clustering, unsupervised learning, natural language processing

TL;DR: We replace k-means centroids with readable summaries for interpretable clustering that scales to streams (mini-batch), is LLM-optional with capped cost, and matches/beats baselines with far fewer calls.

Abstract: We introduce k-NLPmeans and k-LLMmeans, text-clustering variants of k-means that periodically replace numeric centroids with textual summaries. The key idea—summary-as-centroid—retains k-means assignments in embedding space while producing human-readable, auditable cluster prototypes. The method is LLM-optional: k-NLPmeans uses lightweight, deterministic summarizers, enabling offline, low-cost, and stable operation; k-LLMmeans is a drop-in upgrade that uses an LLM for summaries under a fixed per-iteration budget whose cost does not grow with dataset size. We also present a mini-batch extension for real-time clustering of streaming text. Across diverse datasets, embedding models, and summarization strategies, our approach consistently outperforms classical baselines and approaches the accuracy of recent LLM-based clustering—without extensive LLM calls. Finally, we provide a case study on sequential text streams and release a StackExchange-derived benchmark for evaluating streaming text clustering.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 15924