Dimensionality Reduction of Service Monitoring Time-Series: An Industrial Use Case

Farzana Anowar, Samira Sadaoui, Hardik Dalal

Abstract: Our study proposes a dimensionality reduction approach to efficiently process a service monitoring application’s high-dimensional and unlabeled time-series dataset. The approach aims to improve data quality and lower the feature space optimally. Since the dataset is vast and the reduction approach requires colossal resources, we divide it into several weekly sub-datasets. Using clustering methods and metrics, we evaluate the approach’s efficacy on each sub-dataset thoroughly and show that the information loss was minimal after data transformation. Moreover, we assess each sub-dataset’s trustworthiness and similarity to verify whether the new data acquire the same cluster labels as the initial data. Since the experiments reveal a high data quality, the industrial partner can utilize the new data in their decision-making tasks.