10 Features, Data Prep and Leakage Interview Questions and Answers

Short answer: Leakage means information unavailable at real prediction time seeps into training. Validation metrics look great; in production the model falls apart.

In depth:

1. Preprocessing fit on the full dataset before the split — a scaler, imputer, feature selection, or SMOTE fit on train+test has already "seen" the test distribution.
2. Features from the future or post-outcome — churn_reason, aggregates over the customer's entire history: the value is only known after the target happens.
3. Duplicates and linked rows across splits — the same user in train and test: the model memorizes them instead of generalizing.

# Wrong: scaler fit before the split
X = StandardScaler().fit_transform(X)
X_tr, X_te, y_tr, y_te = train_test_split(X, y)

# Right: everything inside a Pipeline, fit on train only
pipe = Pipeline([("scaler", StandardScaler()),
                 ("clf", LogisticRegression())])
pipe.fit(X_tr, y_tr)

Fix: all preprocessing inside a Pipeline fit on train folds only; time-aware splits for temporal data, GroupKFold for repeated users.

⚠️ Common mistake: saying "I split the data and train" without stating "split BEFORE preprocessing" — that ordering is exactly what the interviewer is probing.

Short answer: There is no universal encoding: the choice is driven by the model family and the feature's cardinality. One-hot for linear models and low cardinality, label encoding only for trees, target encoding and hashing for high cardinality.

In depth:

Two questions before choosing: (1) is the model sensitive to order and distances — a false order hurts linear and metric models, trees don't care; (2) how many unique values — at thousands of categories one-hot blows up the matrix, so target/hashing/embeddings.

⚠️ Common mistake: label encoding for logistic regression — the model starts believing that the city coded 17 is "greater than" the city coded 5.

Short answer: Target encoding replaces a category with the target's mean over that category — i.e. it puts the target straight into a feature. The naive version (mean over the full dataset) leaks and inflates the CV score. The safe scheme: out-of-fold encoding + smoothing toward the global mean + optionally noise.

In depth:

1. The leak mechanism — a row's own target participates in its category's mean; rare categories nearly memorize the answer.
2. Out-of-fold — encoding values for a fold are computed only on the other folds; for the test set, on the whole train.
3. Smoothing (regularization) — shrinkage toward the global mean: enc = (n·mean_cat + m·mean_global) / (n + m); rare categories are pulled to the global mean instead of their own 2–3 observations.
4. CatBoost — ordered target statistics: encodes each row using only "earlier" rows within a permutation — the same idea out of the box.

for tr, val in KFold(5).split(df):
    means = df.iloc[tr].groupby("cat")["y"].mean()
    df.loc[df.index[val], "cat_te"] = df.iloc[val]["cat"].map(means)

⚠️ Common mistake: proposing target encoding without saying "out-of-fold" — to the interviewer that flags a candidate this leak has never actually burned in practice.

Short answer: Scaling matters for models that rely on distances, dot products, or gradient descent: kNN, SVM, k-means, PCA, regularized linear models, neural nets. Trees, forests, and boosting don't need it — they are invariant to monotone transforms.

In depth:

And the key point: the scaler is part of the model. Fit on train only, transform valid/test — otherwise test statistics leak into training.

⚠️ Common mistake: scaling everything "just in case" before the split — a harmless habit turns into leakage; and conversely, wasting time scaling for XGBoost.

Short answer: First understand the missingness mechanism, then pick the method: median/mean, model-based imputation, a separate category, plus a "was missing" indicator column. Boosting handles NaN natively. If missingness depends on the value itself (MNAR), it carries signal — you can't just paint over it.

In depth:

In practice:

1. Median/mean/mode — quick baseline; median is robust to outliers.
2. Missingness indicator — an is_missing column preserves the MNAR signal.
3. Model-based imputation (kNN, iterative) — more accurate, more expensive; fit on train only.
4. Native handling — XGBoost/LightGBM/CatBoost learn which split branch to route NaN into.

⚠️ Common mistake: "drop the rows with missing values" as the whole answer: you lose data and bias the sample wherever missingness isn't MCAR.

Short answer: Detect with IQR, z-score (for roughly normal data), isolation forest for multivariate cases. But the first question is not "how do I delete them" — it's "is this an error or a signal": in fraud and anomaly detection the outliers ARE the target.

In depth:

1. Diagnose — a data error (height 250 cm from a typo, price −1) or a real rare event (a customer spending 100×)?
2. Errors — fix or remove, logging the reason.
3. Real extremes — don't delete; reduce their influence:

4. Clipping bounds and statistics are computed on train — otherwise yet another leak.

⚠️ Common mistake: auto-"cleaning" outliers in a fraud or anomaly task — throwing away exactly the objects the model is supposed to catch.

Short answer: Filter methods score features independently of the model (correlation, mutual information), wrappers search subsets by training the model (RFE), embedded methods select during training itself (L1, tree importances). Critically: selection is part of training and must live inside the CV loop — otherwise the estimate is inflated by leakage.

In depth:

Where the leak is: picking top features by correlation with the target on ALL the data and then cross-validating — the features were already chosen while peeking into the test folds, so CV tells a fairy tale. Correct: the selector inside a Pipeline, re-selected on every train fold.

⚠️ Common mistake: "first I'll select features, then run an honest cross-validation" — the selection itself was already dishonest.

Short answer: Impurity/gain importances are biased toward continuous and high-cardinality features (more split candidates), computed on train (overfit splits look "important"), and correlated features split importance arbitrarily. More reliable: permutation importance on validation, and SHAP.

In depth:

In practice:

1. Permutation on a held-out set — importance as the metric drop when a column is shuffled.
2. SHAP — per-object contributions plus global aggregation.
3. Cluster correlated features (correlation-based clustering) and assess groups, not single features.

⚠️ Common mistake: reading feature_importances_ as truth and using it to drop features or draw business conclusions.

Short answer: Highly correlated features make linear-model coefficients unstable: huge standard errors, signs flipping from sample to sample. Predictive quality barely suffers — it's interpretation that breaks. Diagnose with VIF; treat by dropping/combining features, ridge, or PCA.

In depth:

1. What breaks — the XᵀX matrix is near-singular → coefficient estimates have huge variance; you cannot read feature "importance" off a coefficient.
2. Who hurts — linear and logistic regression as inference tools. Trees and boosting barely care for accuracy, but importances get smeared across duplicate features.
3. Diagnosis — VIF_j = 1/(1−R²_j) from regressing feature j on the rest; VIF > 5–10 is a red flag; plus the correlation matrix.
4. Treatment — drop or combine duplicates (sum, ratio), ridge (L2 stabilizes the estimates), PCA into orthogonal components — at the price of interpretability.

⚠️ Common mistake: "multicollinearity ruins predictions" — no, the forecast is usually fine; what suffers is the coefficients, their signs, and confidence intervals.

Short answer: Start not with features but with definitions: the prediction date and the label window (e.g. "will they churn within 30 days after date X"). Then as-of-time discipline: every feature is computed only on data strictly before the prediction date. Then windowed aggregates (RFM), trends, tenure — and time-based validation.

In depth:

1. Target design — prediction date + label window; one dataset row = (customer, prediction date).
2. As-of-time — joins only "as of the date"; no aggregate peeks to the right.
3. Windowed aggregates (RFM) — recency (days since last event), frequency (events in 7/30/90 days), monetary (spend within the window).
4. Dynamics — deltas and trends: activity in the last 7 days vs the previous 30 — the drop-off is precisely the churn precursor.
5. Statics — tenure, plan, number of support contacts.
6. Time-based validation — train on early prediction dates, test on later ones; not a random split.

──── event logs ────────────►│ prediction date │──── label window ───►
  features: only from here   ▲                    target: churn here
  (7/30/90-day windows, RFM) └─ touch nothing right of this point

⚠️ Common mistake: any feature peeking past the prediction date (an "all-history" aggregate, subscription status at month end) — a model with 0.99 AUC offline and useless in production.