Kernel Rotation Forests for Classification

Jaewoong Shim, Seokho Kang, Sungzoon Cho

Abstract: There have been significant research efforts for developing decision tree (DT)-based ensemble methods. Such methods generally construct an ensemble by aggregating a large number of unpruned DTs, thereby yielding good classification accuracy. A recently developed method, rotation forest, is known to achieve better classification accuracy by rotating the dataset using principal component analysis (PCA). This paper describes a new method called kernel rotation forest, which is an extension of rotation forest. The proposed method applies kernel PCA instead of linear PCA to extract non-linear features when training DTs. Experimental results showed that kernel rotation forest outperforms rotation forest as well as other DT-based ensemble methods.