Clustering & Dimensionality Reduction, in Practice

The unsupervised learning chapter introduced clustering and dimensionality reduction conceptually. This one gives each a concrete, standard algorithm, so the idea stops being abstract.

K-means clustering works in a simple loop. Pick K starting points, called centroids. Assign every example to whichever centroid it is closest to. Move each centroid to the average position of the examples now assigned to it. Repeat this assign-then-average cycle until the centroids stop moving much. The result is K clusters, each gathered around its own centroid.

Principal Component Analysis, or PCA, is the standard dimensionality reduction technique. It finds new, synthetic dimensions, each one a combination of the original features, ordered so the first new dimension captures as much of the data's variation as possible, the second captures as much of what is left, and so on. Keeping only the first few of these new dimensions often preserves most of the meaningful structure in the data while discarding the rest.

• -K-means answers: which group does this example belong to?
• -PCA answers: what are the few dimensions that actually matter most in this data?

Both techniques show up constantly as a first exploratory step. Clustering can reveal whether natural customer segments exist before you design a targeting strategy, and PCA can compress a dataset down to two or three dimensions just so a human can actually plot and look at it.