Feature Engineering
Encoding, scaling, imputation, outliers, transforms, feature selection, multicollinearity and leak-free temporal features.
12 questions
JuniorTheoryVery commonStandardization, min-max or robust scaling — and which models ignore scale?
Standardization, min-max or robust scaling — and which models ignore scale?
Standardization centres a feature and divides by its standard deviation; min-max maps it into a fixed range; robust scaling uses median and interquartile range. Distance and gradient based models need scaling; trees split on order alone and do not.
Common mistakes
- ✗Scaling features for a gradient boosted tree and expecting a different fit
- ✗Using standardization on a heavy-tailed feature where outliers dominate the standard deviation
- ✗Fitting the scaler on the full dataset before the train/test split
Follow-up questions
- →Why does the distance method k-nearest-neighbours change its answer when one feature is unscaled?
- →Does scaling the target change what a squared-error loss actually optimizes?
MiddleTheoryVery commonMean, model-based or an is-missing flag — when does the flag win?
Mean, model-based or an is-missing flag — when does the flag win?
The median resists outliers that drag the mean; a model-based fill borrows other columns but can invent structure. Add an is-missing flag when missingness itself carries signal — an empty field often predicts the target better than any imputed value.
Common mistakes
- ✗Imputing with the mean on a heavy-tailed column where a few extremes shift it
- ✗Discarding the missingness pattern by filling silently and adding no indicator
- ✗Fitting the imputer on the full table so test statistics reach the training rows
Follow-up questions
- →How do gradient boosted trees handle a missing value natively, without any imputation?
- →When does filling with a sentinel value out of range beat a statistical estimate?
JuniorTheoryCommonWhen do binning and explicit interaction features help a linear model?
When do binning and explicit interaction features help a linear model?
A linear model sees only additive effects, so bins let it bend a non-linear response and a product term lets one feature modulate another. A boosted tree builds both itself — thresholds give the bins, stacked splits give the interactions.
Common mistakes
- ✗Adding hand-made interaction columns to a gradient boosted tree that already finds them
- ✗Binning a feature so coarsely that a real monotone trend inside a bin disappears
- ✗Believing a linear model picks up curvature without any transformed column
Follow-up questions
- →How would you choose bin edges — equal width, equal frequency, or from a fitted tree?
- →When does an interaction term help a tree after all, for example a ratio of two counts?
JuniorTheoryCommonWhy log-transform a skewed feature or target, and what does that change?
Why log-transform a skewed feature or target, and what does that change?
A log transform compresses a long right tail, so a few huge values stop dominating the fit and the relation becomes closer to linear. On the target it changes the objective — squared error on logs penalises relative error, so the model fits ratios.
Common mistakes
- ✗Forgetting that a log target needs back-transforming, and that a naive exponential is biased low
- ✗Logging a feature with zeros or negatives instead of using a shift such as log1p
- ✗Expecting a log to help a tree model, which is invariant to any monotone rescaling
Follow-up questions
- →How does the power transform Box-Cox differ from a plain log?
- →What bias appears when you exponentiate a prediction made on a log target?
JuniorTheoryCommonWhen is one-hot, label or ordinal encoding the right choice for a category?
When is one-hot, label or ordinal encoding the right choice for a category?
One-hot suits nominal categories — each level gets its own binary column. Ordinal encoding suits genuinely ranked levels, where that order is real. Label encoding hands out arbitrary integers, so a linear model reads a false order and false spacing.
Common mistakes
- ✗Label-encoding a nominal feature and letting a linear model read the integers as an order
- ✗Assuming one-hot is free — a high-cardinality feature explodes the column count
- ✗Treating ordinal encoding as safe when the levels have no real ranking
Follow-up questions
- →How would you encode a high-cardinality feature such as postal code?
- →Do tree models suffer from one-hot the way linear models suffer from label encoding?
JuniorTheoryCommonHow do you detect outliers, and when do you keep them instead of clipping?
How do you detect outliers, and when do you keep them instead of clipping?
An interquartile range rule flags points far outside the middle half; a z-score, points many standard deviations from the mean; isolation forest isolates a point in few random splits. Keep extremes that carry real signal; clip only measurement error.
Common mistakes
- ✗Dropping every flagged point, deleting the rare events the model exists to catch
- ✗Using a z-score on a heavy-tailed feature, where the outliers inflate the standard deviation itself
- ✗Clipping outliers with thresholds computed on the full dataset including the test split
Follow-up questions
- →Why does an interquartile rule survive a skewed feature better than a z-score?
- →How would you treat an outlier in the target rather than in a feature?
MiddleCodeCommonFind the step in this pandas pipeline that fits before the train/test split.
Find the step in this pandas pipeline that fits before the train/test split.
The scaler and the imputer are fitted on the full frame, so test statistics leak into training features and the score comes out optimistic. Split first, fit only on the training part, then transform both parts with those statistics.
Open full question →Common mistakes
- ✗Calling fit_transform on the test part instead of transform with the training statistics
- ✗Thinking a leak requires touching the target, when feature statistics leak on their own
- ✗Splitting first but re-fitting the scaler separately on each part
Follow-up questions
- →How does the sklearn helper Pipeline make this mistake structurally impossible?
- →Where does the same leak reappear under cross-validation rather than one split?
MiddleTheoryCommonHow do you detect multicollinearity, and what exactly does it break?
How do you detect multicollinearity, and what exactly does it break?
A correlation matrix or a high variance inflation factor exposes collinearity. In a linear model coefficients turn unstable and uninterpretable, signs can flip, but predictions stay fine. Trees keep accuracy; importance splits between the twins.
Common mistakes
- ✗Assuming collinearity ruins a linear model's predictions rather than only its coefficients
- ✗Reading tree feature importances as stable when two correlated columns share the credit
- ✗Treating a high variance inflation factor as a scaling problem
Follow-up questions
- →How does the penalty method ridge regression change the picture for correlated predictors?
- →Why can dropping one of a correlated pair still hurt a model that was interpretable?
MiddleTheoryCommonHow does target encoding leak the label, and how do out-of-fold means fix it?
How does target encoding leak the label, and how do out-of-fold means fix it?
Replacing a category with a mean target computed on the same rows puts each row's own label into its feature, so the model memorises it and validation looks too good. Compute the mean out of fold and smooth rare categories toward the global mean.
Common mistakes
- ✗Fitting the encoding on all training rows at once, so each row's own label enters its feature
- ✗Trusting a validation score produced under in-fold target encoding
- ✗Leaving a rare category encoded by the mean of one or two rows without smoothing
Follow-up questions
- →How does the smoothing prior trade a category mean against the global mean?
- →Why does the boosting library CatBoost use an ordered target statistic instead?
JuniorCodeOccasionalEncode hour of day so that 23:00 and 00:00 come out close together.
Encode hour of day so that 23:00 and 00:00 come out close together.
Map the value onto a circle — take the angle two pi times hour over 24 and emit its sine and cosine as two columns. Because the circle closes, hour 23 and hour 0 land next to each other, which a plain integer column cannot express.
Open full question →Common mistakes
- ✗Leaving the raw integer hour in place, so the model still sees midnight as far from 23:00
- ✗Emitting only sine, which collides two different hours onto the same value
- ✗Hard-coding a period of 24 and reusing the helper for month or weekday
Follow-up questions
- →Why does a single sine column collide two different hours onto one value?
- →Does a gradient boosted tree gain anything from this encoding?
MiddleTheoryOccasionalHow do you tell missing completely at random, at random and not at random apart?
How do you tell missing completely at random, at random and not at random apart?
Completely at random means missingness depends on nothing, so any unbiased fill works. At random means it depends on observed columns, so impute from them. Not at random means the hidden value drives its own absence — model the missingness.
Common mistakes
- ✗Assuming a mean fill is safe when the value hid itself because it was extreme
- ✗Believing you can prove missing-not-at-random from the observed data alone
- ✗Dropping rows with missing values and silently biasing the remaining sample
Follow-up questions
- →What diagnostic separates missing at random from missing completely at random?
- →Why is a censored measurement, such as a saturating sensor, missing not at random?
SeniorDesignOccasionalA churn model is retrained nightly from a feature store. A customer's label is only known 30 days after the event, and the feature tables — support tickets, usage counters, plan changes — are updated continuously and sometimes back-filled several days late. The last run scored 0.91 offline but 0.68 in production. Design the feature pipeline so every training row sees only what was actually known at its own prediction time. Explain what the point-in-time (as-of) join must guarantee, how a back-filled record that arrives after the label is handled, and how training features and serving features stay identical.
A churn model is retrained nightly from a feature store. A customer's label is only known 30 days after the event, and the feature tables — support tickets, usage counters, plan changes — are updated continuously and sometimes back-filled several days late. The last run scored 0.91 offline but 0.68 in production. Design the feature pipeline so every training row sees only what was actually known at its own prediction time. Explain what the point-in-time (as-of) join must guarantee, how a back-filled record that arrives after the label is handled, and how training features and serving features stay identical.
Read every feature as of the prediction timestamp, not today. The as-of join takes the last value whose event and arrival time precede it, so late back-fills stay invisible to old rows. Label lag forces a 30-day gap; serving shares that definition.
Common mistakes
- ✗Joining on event time only and ignoring when the record actually landed in the table
- ✗Training on features that already reflect the outcome window the label covers
- ✗Writing separate training and serving feature code, so the two definitions drift apart
Follow-up questions
- →How would you audit an existing training table for point-in-time violations?
- →What does the 30-day label lag imply for the boundary between validation folds?
