Unsupervised Learning
k-means and DBSCAN clustering, PCA and LDA, t-SNE/UMAP, and autoencoder-based anomaly detection.
12 questions
JuniorTheoryVery commonHow does k-means work step by step, and which quantity does each step decrease?
How does k-means work step by step, and which quantity does each step decrease?
k-means alternates two steps — assign every point to the nearest centroid, then move each centroid to the mean of its members. Both steps lower the same objective, the within-cluster sum of squared distances, so the run converges to a local optimum.
Common mistakes
- ✗Thinking k-means converges to the global optimum
- ✗Confusing the mean-based centroid update with a median-based one
- ✗Believing the two steps optimise different objectives
Follow-up questions
- →What stops the loop, and can the objective ever increase between iterations?
- →Why does the final partition depend on the starting centroids?
JuniorTheoryVery commonHow do you choose k for k-means, and why is the elbow so often ambiguous?
How do you choose k for k-means, and why is the elbow so often ambiguous?
Plot inertia against k and look for the bend, or score every k by silhouette and take the best. Inertia falls monotonically with k, so on smooth data no sharp bend exists and the elbow is read subjectively; silhouette gives a comparable number instead.
Common mistakes
- ✗Expecting inertia to have a minimum at the true k
- ✗Reading a sharp elbow into a smooth monotone curve
- ✗Treating elbow and silhouette as interchangeable rather than complementary
Follow-up questions
- →What does a silhouette value near zero say about a point?
- →How would a business constraint on the segment count change your choice of k?
JuniorTheoryCommonIn the density-based algorithm DBSCAN, what do eps and min_samples control, and what is noise?
In the density-based algorithm DBSCAN, what do eps and min_samples control, and what is noise?
eps sets the radius of the neighbourhood and min_samples how many points must fall inside it for a point to count as core. Core points and their neighbours form clusters; a point that is neither core nor within eps of a core point is labelled noise.
Common mistakes
- ✗Swapping the meanings of eps and min_samples
- ✗Assuming every point is assigned to some cluster
- ✗Thinking noise is decided by distance to a centroid
Follow-up questions
- →How does raising eps change the number of clusters and the amount of noise?
- →What is a border point, and why can its assignment depend on processing order?
MiddleTheoryCommonRound blobs, interleaved crescents, a nested taxonomy — pick a clustering method for each.
Round blobs, interleaved crescents, a nested taxonomy — pick a clustering method for each.
Round, similarly sized blobs suit k-means, matching its spherical assumption. Interleaved crescents need DBSCAN, which follows density and marks outliers as noise. A nested taxonomy calls for hierarchical clustering, whose dendrogram exposes the levels themselves.
Common mistakes
- ✗Forcing k-means onto non-convex shapes by raising k
- ✗Treating DBSCAN as universally applicable with a single eps
- ✗Choosing an algorithm by data size rather than cluster geometry
Follow-up questions
- →Which linkage would you use for the taxonomy, and why does the choice matter?
- →What breaks in DBSCAN when the crescents have very different densities?
MiddleTheoryCommonHow many principal components do you keep, and what does explained variance really tell you?
How many principal components do you keep, and what does explained variance really tell you?
Keep enough components for a target explained variance, or cut at the scree bend. But explained variance measures reconstruction of the inputs, not usefulness for the label — a low-variance direction may carry the signal, so tune the count by model metric.
Common mistakes
- ✗Treating 95 percent explained variance as a guarantee of downstream quality
- ✗Assuming variance and predictive power are monotonically related
- ✗Fitting PCA on the full dataset before the train and test split
Follow-up questions
- →Give a case where the discriminative direction has almost no variance.
- →Where in a cross-validation pipeline must the PCA fit actually live?
MiddleTheoryCommonHow does the supervised projection LDA differ from PCA, and when is its subspace better?
How does the supervised projection LDA differ from PCA, and when is its subspace better?
PCA is unsupervised and maximises total variance; LDA uses labels and maximises between-class scatter relative to within-class scatter. When the discriminative direction has low variance LDA wins, but it is capped at C-1 components and assumes similar class covariances.
Common mistakes
- ✗Believing LDA is unsupervised or that PCA uses labels
- ✗Forgetting the C-1 cap on the number of LDA components
- ✗Assuming supervision makes LDA unconditionally better
Follow-up questions
- →How many components can LDA return on a three-class problem?
- →What happens to LDA when one class has a far wider covariance than the others?
JuniorTheoryOccasionalWhich failure of random initialisation does k-means++ fix, and what does it cost?
Which failure of random initialisation does k-means++ fix, and what does it cost?
Random seeds can drop several centroids into one dense region and leave a real cluster unclaimed, so the run settles in a poor optimum. k-means++ samples each seed with probability proportional to squared distance from the nearest seed, costing extra passes.
Common mistakes
- ✗Believing k-means++ guarantees the global optimum
- ✗Thinking it substitutes for feature scaling
- ✗Confusing distance-weighted sampling with deterministic farthest-point selection
Follow-up questions
- →Why does sampling proportional to squared distance beat picking the single farthest point?
- →Do you still need several restarts once k-means++ is enabled?
JuniorTheoryOccasionalWhat does the dimensionality-reduction method PCA do geometrically, and why scale first?
What does the dimensionality-reduction method PCA do geometrically, and why scale first?
PCA finds orthogonal directions of maximum variance and projects the data onto the leading ones. Because it maximises raw variance, a feature measured in large units dominates the components, so features on different scales must be standardised before the fit.
Common mistakes
- ✗Thinking PCA uses labels or maximises class separation
- ✗Believing PCA selects original features rather than combining them
- ✗Running PCA on unscaled features with mixed units
Follow-up questions
- →What happens to the components if one feature is in metres and another in millimetres?
- →Are the resulting components correlated with each other, and why?
MiddleTheoryOccasionalIn autoencoder anomaly detection, what is the score and how do you threshold it without labels?
In autoencoder anomaly detection, what is the score and how do you threshold it without labels?
The score is reconstruction error — the model trains on mostly normal data, and points it cannot rebuild score high. Without labels, put the threshold at a percentile of training error matching the alert budget, then tune it by how many alerts analysts can review.
Common mistakes
- ✗Training the autoencoder on data heavily contaminated with anomalies
- ✗Fixing the threshold by a rule of thumb instead of an alert budget
- ✗Believing labelled anomalies are required to produce a score
Follow-up questions
- →What happens to the score when the training data already contains many anomalies?
- →How would you re-check the threshold as normal traffic drifts over time?
MiddleTheoryOccasionalWhy does k-means fail on elongated or unevenly dense clusters — name the assumption.
Why does k-means fail on elongated or unevenly dense clusters — name the assumption.
Minimising squared Euclidean distance to a centroid assumes roughly spherical clusters of similar size and spread. Elongated or unevenly dense groups break that, so k-means cuts across them; a density-based method such as DBSCAN follows the true shape instead.
Common mistakes
- ✗Blaming scaling or the iteration count instead of the shape assumption
- ✗Assuming the correct k rescues k-means on non-spherical clusters
- ✗Not reaching for a density-based alternative when shapes are irregular
Follow-up questions
- →How does a Gaussian mixture model relax part of this assumption?
- →What does DBSCAN give up in exchange for handling arbitrary shapes?
MiddleTheoryOccasionalWhy can cluster sizes and gaps not be read off the nonlinear embeddings t-SNE or UMAP?
Why can cluster sizes and gaps not be read off the nonlinear embeddings t-SNE or UMAP?
Both preserve local neighbourhoods, not global geometry, so between-cluster distances, apparent sizes and empty gaps are artefacts of perplexity and layout. Treat the picture as qualitative only, and never feed the coordinates into a classifier as features.
Common mistakes
- ✗Interpreting cluster sizes or gap widths in the embedding
- ✗Using embedding coordinates as model features
- ✗Assuming tuned perplexity or averaged runs restore global geometry
Follow-up questions
- →Why can a t-SNE fit not be applied to new test points the way a PCA fit can?
- →What would you use instead when a compact numeric representation is genuinely needed?
SeniorDesignRareA retail app with 8M monthly buyers wants five to eight marketing segments the CRM team can act on and that stay recognisable quarter over quarter. You have two years of transactions, sessions and catalogue categories. Design the pipeline end to end — which features you build and how you handle the heavy right tail of spend, what you standardise and whether you reduce dimensionality first, which clustering algorithm you pick and how you settle on the number of segments, and how you convince a sceptical CRM lead that the segments are real rather than an artefact of the algorithm and that they hold up when you refit next quarter.
A retail app with 8M monthly buyers wants five to eight marketing segments the CRM team can act on and that stay recognisable quarter over quarter. You have two years of transactions, sessions and catalogue categories. Design the pipeline end to end — which features you build and how you handle the heavy right tail of spend, what you standardise and whether you reduce dimensionality first, which clustering algorithm you pick and how you settle on the number of segments, and how you convince a sceptical CRM lead that the segments are real rather than an artefact of the algorithm and that they hold up when you refit next quarter.
Build RFM-style features, log-transform skewed spend, standardise, then cluster with k-means and pick k by silhouette plus business usability. Show the segments are real by profiling them on held-out behaviour, and stable by refit agreement across time slices.
Common mistakes
- ✗Clustering unscaled features with a heavy-tailed spend column
- ✗Choosing k by the inertia minimum instead of a scored criterion plus usability
- ✗Confirming stability on the same data rather than across time slices
Follow-up questions
- →How would you keep segment identities aligned between two quarterly refits?
- →What would make you drop a mathematically clean segment as unusable for CRM?
