CD-Trees: An Efficient Index Structure for Outlier Detection

Huanliang Sun, Yubin Bao, Faxin Zhao, Ge Yu, Daling Wang

Abstract: Outlier detection is to find objects that do not comply with the general behavior of the data. Partition is a kind of method of dividing data space into a set of non-overlapping rectangular cells. There exists very large data skew in real-life datasets so that partition will produce many empty cells. The cell-based algorithms for outlier detection don’t get enough attention to the existence of many empty cells, which affects the efficiency of algorithms. In this paper, we propose the concept of Skew of Data (SOD) to measure the degree of data skew, and which approximates the percentage of empty cells under a partition of a dataset. An efficient index structure called CD-Tree and the related algorithms are designed. This paper applies the CD-Tree to detect outliers. Compared with cell-based algorithms on real-life datasets, the speed of CD-Tree-based algorithm increases 4 times at least and that the number of dimensions processed also increases obviously.