Keywords: outlier detection, clustering algorithms, statistical process control, multi-criteria validation, robust clustering, adaptive parameter tuning, K-means clustering, representation learning, anomaly detection
TL;DR: Adaptive clustering with smart outlier detection that auto-tunes, self-corrects, and delivers robust clusters with minimal false alarms.
Abstract: Robust clustering requires effective outlier detection mechanisms that can adapt
to cluster-specific characteristics. We introduce an adaptive parameter tuning approach
that enhances traditional clustering with multi-criteria outlier detection
and intelligent restart-based clustering quality optimization. Our method develops
cluster-specific threshold models using adaptive scaling factors (α = 3.5σ), enabling
automatic parameter selection based on real-time performance monitoring.
We propose a multi-criteria validation framework requiring satisfaction of at least
3 out of 4 criteria, potentially reducing false positive rates compared to singlecriteria
approaches. The framework integrates Statistical Process Control (SPC)
for adaptive parameter optimization and intelligent restart-triggered silhouettebased
k-refinement that automatically searches competitive k values (k-2 to k+3)
when clustering quality is poor (silhouette < 0.25), enabling dynamic adjustment
of outlier detection sensitivity while ensuring competitive clustering structure.
Experimental evaluation on diverse datasets demonstrates that our approach
achieves ultra-conservative outlier detection (0.36-0.43% outlier rates) with competitive
precision (17.9%) among tested outlier detection algorithms and low false
positive rate (1.8%), while maintaining competitive clustering quality (silhouette
scores 0.573-0.781) and computational efficiency (18.6 seconds for 3,700 points),
making it suitable for practical clustering applications.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Submission Number: 15904
Loading