Abstract: Highlights•Local-based ensemble methods produce and/or exploit models specialized in each area.•Locality is often achieved via distance metrics which degrade in high dimensionality.•We propose a method that produces local experts on the fly for high dimensional data.•The models are fit over multiple partitions recursively defined by decision trees.•The proposed method outperformed several distance-based local ensemble methods.