[Clustering]
Hierarchical Clustering.
Builds a tree of clusters through agglomerative or divisive approaches.
SPEC SHEET
FIG — THE MECHANISM, LIVE
Where k-means picks k centres up front, hierarchical clustering would merge these points bottom-up into a tree.
C — How it actually works
Start with every point as its own cluster, then repeatedly merge the two closest clusters until one remains. The record of merges is a dendrogram - a family tree of your data. Cut the tree at any height to get any number of clusters, and see how tight each merge was.
D — The math
Agglomerative clustering with a linkage criterion: single (min pairwise distance), complete (max), average, or Ward (minimum within-cluster variance increase). Ward + Euclidean is the k-means-like default.
Training
O(n² log n) time, O(n²) memory - the practical ceiling is ~10-50k points
Inference
cut the dendrogram
E — When NOT to use it
- Large datasets - the quadratic memory wall is hard (sample first, or use MiniBatch k-means)
- You just need a fast flat partition with known k
- Streaming data - the tree does not update incrementally
F — Tuning that matters
- Ward linkage for compact clusters; average for mixed shapes; single only when you want chaining
- Choose the cut height from the biggest vertical gap in the dendrogram
- Standardise features; linkage distances inherit all the usual scale problems
- On big data: cluster a 10-20k sample hierarchically, then assign the rest to nearest cluster
G — Production pitfalls
- Single linkage chaining: two clusters merged because of one bridging point
- Reading the dendrogram without checking cophenetic distances - pretty trees can distort badly
- Trying it on a million rows and running out of RAM
- Forgetting merges are greedy and irreversible - one early bad merge propagates
H — Minimal starting point
PYTHONfrom scipy.cluster.hierarchy import linkage, dendrogram, fcluster
Z = linkage(X_scaled, method="ward")
dendrogram(Z, truncate_mode="level", p=5) # inspect the tree
labels = fcluster(Z, t=4, criterion="maxclust") # cut into 4 clustersI — The interview question
When does the dendrogram itself matter more than the final clusters?
When the nesting is the finding: taxonomies, document topic trees, gene families. The tree shows sub-structure inside clusters and how strongly each group is separated (merge heights) - information a flat k-partition throws away.
J — In the wild
Single-cell genomics pipelines cluster cells hierarchically on expression profiles: the dendrogram mirrors real biology - cell types splitting into subtypes - and reviewers expect exactly that tree in the paper.
K — Consider instead
- K-Means Clustering— scales far better when you already know roughly how many clusters you need
- DBSCAN— when noise handling and arbitrary shapes matter more than the tree