Dimension Reduction

Provides a tutorial overview of several foundational methods for dimension reduction. The authors divide the methods into projective methods and methods that model the manifold on which the data lies. For projective methods, they review projection pursuit, principal component analysis (PCA), kernel PCA, probabilistic PCA, canonical correlation analysis (CCA), kernel CCA, Fisher discriminant analysis, oriented PCA, and several techniques for sufficient dimension reduction. For the manifold methods, the book reviews multidimensional scaling (MDS), landmark MDS, Isomap, locally linear embedding, Laplacian eigenmaps, and spectral clustering. Although the review focuses on foundations, the author also provide pointers to some more modern techniques, and describe the correlation dimension as one method for estimating the intrinsic dimension, and point out that the notion of dimension can be a scale-dependent quantity. The Nystrm method, which links several of the manifold algorithms, is also reviewed. We use a publicly available dataset to illustrate some of the methods. The goal is to provide a self-contained overview of key concepts underlying many of these algorithms, and to give pointers for further reading.