[Dimensionality Reduction]
t-SNE.
Non-linear dimensionality reduction great for visualization.
SPEC SHEET
C — How it actually works
Make a 2-D map where points that were close neighbours in high dimensions stay close. t-SNE converts distances to neighbour probabilities in both spaces and drags the map around until the two distributions agree. It is a microscope for local structure - clusters pop beautifully - but the map’s global geometry is essentially fiction.
D — The math
Minimise KL(P‖Q) between pairwise affinities: Gaussian kernel in the original space (bandwidth set by perplexity), heavy-tailed Student-t in the map (which prevents crowding). Optimised by gradient descent - non-convex, run-to-run variation is expected.
Training
O(n log n) with Barnes-Hut; practical ceiling ~50-100k points
Inference
no natural transform for new points
E — When NOT to use it
- As input features for downstream models - it distorts distances by design
- When between-cluster distances or axis directions need to mean something
- Datasets past ~100k points (use UMAP) or pipelines needing transform() on new data
F — Tuning that matters
- Perplexity ≈ 5-50, roughly "how many neighbours matter"; try 30 first
- PCA down to ~50 dims first - it denoises and speeds everything up
- Run 3-4 seeds; structure that survives all seeds is real, the rest is optimisation noise
- Let it run long enough - early stopping leaves clusters half-separated
G — Production pitfalls
- Reading cluster SIZE or inter-cluster DISTANCE as meaningful - both are artefacts
- Treating axis positions as coordinates for arithmetic
- Publishing one seed’s map when another seed tells a different story
- Using t-SNE output to train a classifier
H — Minimal starting point
PYTHONfrom sklearn.decomposition import PCA
from sklearn.manifold import TSNE
X50 = PCA(n_components=50).fit_transform(X_scaled)
XY = TSNE(perplexity=30, init="pca", random_state=0).fit_transform(X50)
# plot XY coloured by label - for eyes only, never for modelsI — The interview question
Why a Student-t distribution in the low-dimensional space?
In 2-D there is far less "room" than in high dimensions, so moderate distances get crushed together (the crowding problem). The t-distribution’s heavy tails let moderately-distant pairs sit further apart in the map, keeping clusters visually separated.
J — In the wild
Every immunology paper’s cell-atlas figure - the coloured blob map of cell populations - is t-SNE or UMAP on single-cell data. The blobs guide discovery; the actual statistics are always computed back in the original space.
K — Consider instead
- Principal Component Analysis (PCA)— when you need a fast, deterministic, invertible linear projection