Guided Locally Linear Embedding

Babak Alipanahi, Ali Ghodsi

Abstract: Nonlinear dimensionality reduction is the problem of retrieving a low-dimensional representation of a manifold that is embedded in a high-dimensional observation space. Locally Linear Embedding (LLE), a prominent dimensionality reduction technique is an unsupervised algorithm; as such, it is not possible to guide it toward modes of variability that may be of particular interest. This paper proposes a supervised variation of LLE. Similar to LLE, it retrieves a low-dimensional global coordinate system that faithfully represents the embedded manifold. Unlike LLE, however, it produces an embedding in which predefined modes of variation are preserved. This can improve several supervised learning tasks including pattern recognition, regression, and data visualization. Graphical abstract Visualizations of the embeddings acquired by GLLE on the USPS handwritten digits dataset. Download : Download high-res image (68KB) Download : Download full-size image Research highlights ► Locally Linear Embedding (LLE) is an unsupervised algorithm. ► It is not possible to guide LLE toward modes of variability that may be of particular interest. ► We have proposed a novel, supervised extension to Locally Linear Embedding that we call GLLE. ► We have demonstrated the effectiveness of GLLE in classification and data visualization tasks.