Chapter 4: Unsupervised Learning

What Is Unsupervised Learning?

Unsupervised Learning

Discovering the latent structure and patterns in unlabeled data.

$$\{\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n\} \rightarrow \text{structure / patterns}$$
Supervised vs. Unsupervised Supervised Learning data: x₁ → y₁ ✓ data: x₂ → y₂ ✓ data: x₃ → y₃ ✓ with labels Unsupervised Learning data: x₁ ? data: x₂ ? data: x₃ ? no labels
Figure 1: Supervised vs. unsupervised learning — the presence or absence of labels is the key difference

Goals of Unsupervised Learning

  • Grouping data (clustering)
  • Compressing / summarizing data (dimensionality reduction)
  • Detecting anomalies (anomaly detection)
  • Generating data (generative models)

Here, "generating" means learning the data distribution to produce new data (generative models such as VAEs and GANs); modern generative AI also builds on ideas from unsupervised learning. This article covers the three representative tasks of clustering, dimensionality reduction, and anomaly detection.

Clustering

Clustering

Group similar data points into clusters. A natural grouping is discovered without any labels.

Example of Clustering Raw data clustering After clustering
Figure 2: Clustering — automatically grouping similar data (× = cluster centroids)

Applications of Clustering

  • Customer segmentation: group customers by purchasing patterns
  • Image grouping: automatically gather similar images
  • Document clustering: organize news articles by topic
  • Gene expression analysis: group genes with similar expression patterns

Representative Algorithms

  • k-means: partition into k clusters (the most basic)
  • Hierarchical clustering: build a tree-structured hierarchy of clusters
  • DBSCAN: density-based; discovers clusters of arbitrary shape

Dimensionality Reduction

Dimensionality Reduction

Compress high-dimensional data into fewer dimensions while preserving the important information.

Data points (red) distributed on a 2D plane (green) that cuts diagonally through a semi-transparent cube in 3D space
3D (original data)
projection
→
The same plane viewed head-on as a 2D scatter plot, with the data points (red) lying on the plane
2D (after projection)
Figure 3: Dimensionality reduction — when the data actually lies on a 2D plane, viewing that plane head-on compresses it to 2D while preserving the information

Applications of Dimensionality Reduction

  • Visualization: plot high-dimensional data in 2D/3D
  • Preprocessing: compress features to reduce computation
  • Denoising: drop unimportant components
  • Feature extraction: discover the essential features

Representative Methods

  • PCA (principal component analysis): pick the directions in which the data spreads most (the principal components, ordered by variance) and project onto the top few, reducing dimensions while preserving information
  • t-SNE: visualization-oriented, preserving local structure
  • UMAP: faster than t-SNE and also preserves global structure

Anomaly Detection

Anomaly Detection

Find anomalous data that deviates greatly from the usual patterns.

Example of Anomaly Detection Normal region Anomaly! Anomaly!
Figure 4: Anomaly detection — flagging data that falls outside the normal region

Applications of Anomaly Detection

  • Fraud detection: fraudulent credit-card usage
  • Equipment monitoring: detecting early signs of machine failure
  • Cybersecurity: detecting unauthorized access
  • Healthcare: detecting abnormal test results

Supervised vs. Unsupervised

Aspect Supervised Unsupervised
Correct labels Required Not required
Goal Prediction (regression / classification) Structure discovery
Evaluation Compare against ground truth Hard to evaluate
Data collection Costly labeling Relatively easy

* Because unsupervised learning has no correct labels, it is hard to judge the quality of a result against an objective ground truth.

Summary

  • Unsupervised learning: discover the structure of data without labels
  • Clustering: group similar data (k-means, DBSCAN, etc.)
  • Dimensionality reduction: compress and visualize high-dimensional data (PCA, t-SNE, etc.)
  • Anomaly detection: find data that differs from the norm

Frequently Asked Questions

Q. What is unsupervised learning?

Unsupervised learning is a branch of machine learning that discovers latent patterns and structure in data that has no correct labels. Its main goals are clustering (grouping similar data), dimensionality reduction (compressing high-dimensional data), anomaly detection (finding outliers), and generative modeling (generating data).

Q. How does k-means clustering work?

k-means initializes k cluster centers (centroids) and repeats two steps until the assignments stop changing: (1) assign each data point to its nearest centroid, and (2) update each centroid to the mean of the points assigned to it. It is the most basic clustering method, and the number of clusters k must be specified in advance.

Q. What is PCA (principal component analysis) used for?

PCA is a representative dimensionality-reduction method that finds the orthogonal directions (principal components) maximizing the variance of the data and keeps only the top few components. Because it compresses dimensions while minimizing information loss, it is widely used for visualization, denoising, feature extraction, and preprocessing.

Q. If there are no labels, how does unsupervised learning actually learn?

It does not use correct labels, but it relies on statistical structure in the data—the distances, similarities, and distributions between data points—to find common patterns and groupings. For example, clustering groups nearby points together, while PCA keeps the directions of largest variance; both learn by optimizing a clear objective function.

References