Statistics
Central tendency, distribution shape, the central limit theorem, and Simpson's paradox.
18 questions
JuniorTheoryCommonWhat are the mode, median, and arithmetic mean?
What are the mode, median, and arithmetic mean?
The mode is the most frequent value. The median is the middle value of the sorted data — the 50th percentile, splitting it into two equal halves. The arithmetic mean is the sum of all values divided by their count, the balance point of the data.
Common mistakes
- ✗Confusing the median with the mean when the data is skewed
- ✗Thinking the mode must be unique — a dataset can have several modes
- ✗Believing all three measures always coincide
Follow-up questions
- →Which central measure is most robust to outliers, and why?
- →When would you report the median instead of the mean for income data?
JuniorTheoryCommonNULL, NaN, and zero in a dataset — why are they three different statements?
NULL, NaN, and zero in a dataset — why are they three different statements?
NULL means the value is unknown or absent, NaN is an undefined result like 0/0, and zero is a real measured quantity. Treating them alike corrupts an average — counting NULLs as zeros drags the mean down and inflates the denominator, and a stray NaN poisons the sum.
Common mistakes
- ✗Imputing NULLs as zero and silently biasing the mean downward
- ✗Letting one NaN propagate through a sum instead of masking it
- ✗Assuming every blank means the same thing across all columns
Follow-up questions
- →How does SQL AVG treat NULL, and how does that differ from pandas mean?
- →When is imputing zero the right choice rather than a source of bias?
JuniorTheoryCommonWhy report p50 and p95 latency instead of the mean — what do percentiles capture?
Why report p50 and p95 latency instead of the mean — what do percentiles capture?
A percentile is the value below which that share of observations falls: p95 is the latency 95% of requests beat. The mean hides the tail — slow calls inflate it and a bimodal mix averages to a value no request sees. p50 is the typical case, p95 the near-worst users feel.
Common mistakes
- ✗Reading p95 as the value 95% of requests exceed rather than beat
- ✗Averaging percentiles across servers, which is not meaningful
- ✗Reporting only the mean and missing a heavy or bimodal tail
Follow-up questions
- →Why can't you average p95 across shards to get an overall p95?
- →How would you estimate a percentile from a streaming data source?
JuniorTheoryCommonWhat makes a sample representative, and why does a convenience sample of power users mislead?
What makes a sample representative, and why does a convenience sample of power users mislead?
A representative sample gives every member of the target population a known, non-zero chance of selection, so its composition mirrors the whole. A convenience sample of your most active users over-weights heavy usage, so metrics skew high and no extra data removes the bias.
Common mistakes
- ✗Believing a large sample is automatically representative
- ✗Assuming selection bias averages out with more data
- ✗Trusting active-user behaviour as a proxy for all users
Follow-up questions
- →How would you reweight a convenience sample toward the true population?
- →What is survivorship bias, and how does it relate to this problem?
JuniorTheoryCommonStandard error vs standard deviation — why does the SE shrink with n but the SD not?
Standard error vs standard deviation — why does the SE shrink with n but the SD not?
The standard deviation measures spread in the raw data — a population property that does not shrink with n. The standard error is the SD of a statistic's sampling distribution, SD/√n for the mean, so it falls as n grows. The SD describes individuals; the SE, precision.
Common mistakes
- ✗Confusing the SE with the SD and quoting the wrong one on an estimate
- ✗Thinking more data narrows the spread of the raw values themselves
- ✗Forgetting the SE falls only as the square root of n
Follow-up questions
- →To halve the standard error of a mean, by what factor must n grow?
- →How do you compute the standard error of a proportion?
JuniorTheoryCommonVariance vs standard deviation — what does each measure, and why report the SD?
Variance vs standard deviation — what does each measure, and why report the SD?
Variance is the mean squared deviation from the mean, so it lives in squared units. The standard deviation is its square root, back in the data's units and comparable to the mean. You report the SD because '±5 kg' is interpretable, while '25 kg²' is not.
Common mistakes
- ✗Reporting variance in squared units as if it were on the data's scale
- ✗Thinking the SD and variance measure the center rather than the spread
- ✗Forgetting that variances of independent variables add but SDs do not
Follow-up questions
- →When do you divide by n versus n−1 to estimate the variance?
- →How does the standard deviation behave for a heavily skewed distribution?
MiddleTheoryCommonA single ₽10M order moves the mean 20% — trim, winsorize, cap, or keep the outlier?
A single ₽10M order moves the mean 20% — trim, winsorize, cap, or keep the outlier?
First decide whether it is real or a data-entry error. A genuine ₽10M order is signal — keep it, and report the median beside the mean so one whale does not distort the typical value. Trimming or winsorizing biases real totals low; exclude only true errors.
Common mistakes
- ✗Deleting an extreme value without checking whether it is genuine
- ✗Winsorizing real revenue and silently biasing totals low
- ✗Reporting only the mean when one whale distorts it
Follow-up questions
- →How does winsorizing differ from trimming in what it does to the data?
- →When is a robust statistic like the median preferable to cleaning outliers?
JuniorTheoryOccasionalWhat is the central limit theorem and where is it used in analytics?
What is the central limit theorem and where is it used in analytics?
The central limit theorem says the distribution of the sample mean of independent identically-distributed values with finite variance approaches a normal distribution as the sample size grows, whatever the original shape. It justifies normal-based confidence intervals and z/t-tests on means, and underpins A/B significance testing.
Common mistakes
- ✗Claiming the raw data itself becomes normal, not the sample mean
- ✗Forgetting the finite-variance and independence conditions
- ✗Assuming a fixed n like 30 makes it exact rather than approximate
Follow-up questions
- →How does the convergence change for a heavily skewed source distribution?
- →Why does the standard error of the mean shrink as the square root of n?
JuniorTheoryOccasionalOrder mode, median, and mean in a right-skewed distribution.
Order mode, median, and mean in a right-skewed distribution.
For a right-skewed distribution (long right tail, like incomes), the order is mode < median < mean. The long tail of large values pulls the mean up the most, the median less, and the mode sits at the dense low end. Left-skew reverses the order.
Common mistakes
- ✗Reversing the order, which describes left-skew not right-skew
- ✗Assuming skew shifts spread but not the relative center positions
- ✗Forgetting the mean is the measure most pulled by the tail
Follow-up questions
- →For the exponential distribution, what are the mode, median, and mean?
- →Why is the median preferred over the mean for skewed salary data?
JuniorTheoryOccasionalName several probability distributions and where each applies.
Name several probability distributions and where each applies.
Common ones — normal (measurement errors), uniform (equally likely outcomes), Bernoulli/binomial (success counts, like conversions), Poisson (event counts per interval, like arrivals), exponential (waiting times), and log-normal (quantities like income). Each is discrete or continuous.
Common mistakes
- ✗Listing measures of spread instead of named distributions
- ✗Assuming everything is normal because of the CLT
- ✗Mixing discrete and continuous without noting the use case
Follow-up questions
- →Which distribution models the number of clicks on a banner per hour?
- →When would you model a metric as log-normal rather than normal?
MiddleTheoryOccasionalHow do you size a sample to detect a 2% lift — what four inputs drive the formula?
How do you size a sample to detect a 2% lift — what four inputs drive the formula?
Four inputs set it: the metric's baseline variance, the minimum detectable effect (the 2% lift), the significance level α, and the power 1−β. n grows with variance and (z_α+z_β)² and falls with the square of the effect — so halving the lift quadruples n.
Common mistakes
- ✗Treating power or variance as an output instead of an input
- ✗Assuming sample size scales linearly with the effect size
- ✗Ignoring the baseline variance when planning the test
Follow-up questions
- →Why does halving the minimum detectable effect roughly quadruple the sample?
- →How does switching to a one-sided test change the required n?
JuniorTheoryRareLaw of large numbers vs the central limit theorem — which two theorems get conflated?
Law of large numbers vs the central limit theorem — which two theorems get conflated?
The law of large numbers says the sample mean converges to the true mean as n grows — it is about where the average lands. The central limit theorem says the distribution of that sample mean approaches a normal shape whatever the source — it is about the error around it.
Common mistakes
- ✗Swapping which theorem promises convergence and which describes shape
- ✗Thinking the raw data becomes normal rather than the sample mean
- ✗Assuming either theorem needs a normal source distribution
Follow-up questions
- →Which theorem justifies a confidence interval, and which a large-n point estimate?
- →Does the law of large numbers still hold when the variance is infinite?
JuniorTheoryRareAverage check rose in every category — can the platform-wide average fall?
Average check rose in every category — can the platform-wide average fall?
Yes. The platform-wide average can fall though each category's average rose, because the category mix shifted toward the cheaper one. This is Simpson's paradox — a weighted aggregate depends on the weights, so it can't be judged from per-category trends alone.
Common mistakes
- ✗Answering that the platform average must also rise
- ✗Ignoring that category mix (weights) changes the aggregate
- ✗Treating the overall average as a plain sum of category averages
Follow-up questions
- →What concrete category-mix shift would make the platform average drop here?
- →How do you guard analysis against Simpson's paradox?
MiddleTheoryRareIce-cream sales correlate with drownings — name the confounder and say how you would control for it.
Ice-cream sales correlate with drownings — name the confounder and say how you would control for it.
The confounder is hot weather: summer heat drives both ice-cream sales and swimming, so the two move together, no causal link. Control for it by conditioning — stratify or regress on temperature or season. If the association vanishes once weather is held fixed, it was spurious.
Common mistakes
- ✗Reading the raw correlation as proof of causation
- ✗Failing to name a plausible third variable
- ✗Adjusting for a variable on the causal path instead of a confounder
Follow-up questions
- →How does a confounder differ from a mediator on the causal path?
- →Why can adjusting for a collider create a spurious association?
MiddleTheoryRarePearson's r ≈ 0 for two variables that clearly move together — what can correlation not see?
Pearson's r ≈ 0 for two variables that clearly move together — what can correlation not see?
Pearson's r measures only the strength of a linear relationship. A U-shaped or non-monotonic bond — like a dose effect that rises then falls — averages to r ≈ 0 even though the two are tightly linked. Plot the scatter, and use Spearman for non-linear dependence.
Common mistakes
- ✗Reading r ≈ 0 as proof of independence
- ✗Skipping the scatter plot before trusting a correlation
- ✗Forgetting outliers alone can drive r up or down
Follow-up questions
- →How does Spearman's rank correlation handle a monotone non-linear link?
- →What does a low r say about a strong quadratic relationship?
MiddleTheoryRareMCAR, MAR, and MNAR — how does the missingness mechanism decide whether dropping rows biases the result?
MCAR, MAR, and MNAR — how does the missingness mechanism decide whether dropping rows biases the result?
MCAR means missingness is unrelated to anything, so dropping rows only costs power. MAR means it depends on observed variables — safe to drop once you condition on them, biased otherwise. MNAR means it depends on the unseen value itself, so dropping always biases the estimate.
Common mistakes
- ✗Treating all missingness as MCAR and dropping rows by default
- ✗Imputing the mean under MNAR and assuming the bias is gone
- ✗Confusing how much is missing with why it is missing
Follow-up questions
- →How would you probe whether missingness is MAR versus MNAR?
- →When does multiple imputation beat a plain complete-case analysis?
SeniorDebuggingRareAn A/A test came back significant on a clean sample — selection bias, peeking, or bad luck?
An A/A test came back significant on a clean sample — selection bias, peeking, or bad luck?
An A/A test has no real effect, so p = 0.036 is expected — at α = 0.05, 5% of clean A/A tests hit significance by chance. The tell is peeking: stopping at the first p < 0.05 before the planned horizon inflates the false-positive rate. A balanced split rules out selection bias.
Open full question →Common mistakes
- ✗Reading any single significant A/A result as a broken pipeline
- ✗Treating repeated daily peeks as independent, harmless looks
- ✗Forgetting the 5% false-positive rate applies to A/A tests too
Follow-up questions
- →How do sequential testing methods control the peeking problem?
- →What sample-ratio check would confirm the split is truly balanced?
SeniorTheoryRareWhy does a bootstrap (resampling) confidence interval work at all, and when does it fail?
Why does a bootstrap (resampling) confidence interval work at all, and when does it fail?
Resampling uses the sample as a population proxy, so the statistic's spread approximates its sampling distribution — giving a standard error and CI with no formula. It works because the empirical distribution approaches the real one, and fails for tail-dependent statistics like the max and for tiny samples.
Common mistakes
- ✗Thinking the bootstrap assumes normality of the data
- ✗Bootstrapping an extreme-quantile or max statistic and trusting the interval
- ✗Applying it to a tiny sample that cannot represent the population
Follow-up questions
- →How does the percentile bootstrap differ from the BCa interval?
- →Why does the bootstrap struggle to estimate the sample maximum?