13 ML Engineering Interview Questions and Answers

Short answer: Separate the deterministic from the stochastic. Transforms, losses and metrics get ordinary unit tests on hand-computable cases; training is covered by smoke tests (overfit a tiny batch, shape and gradient-flow checks) and behavioral tests with tolerances.

In depth:

1. Test the code, not the model — deterministic pieces (features, losses) deserve exact asserts; stochastic ones get tolerances and invariants.
2. Smoke test in CI — a short 2–3 step run: the pipeline assembles, the loss is finite and decreasing.

⚠️ Common mistake: "we look at test accuracy" — that's model evaluation, not code testing. A dataloader bug can cost a couple of metric points and never surface.

Short answer: Go from data to architecture, not the other way around. First eyeball a batch and its labels, then run the decisive experiment — overfit a single batch. Only then LR, training-loop wiring bugs, and initialization.

In depth:

1. Eyeball the data: a raw batch, the labels, X ↔ y alignment
2. Overfit ONE batch to ~100% — the decisive experiment:
   can't → the wiring is broken; can → data or LR
3. LR sweep: ×10 up and down — a too-large LR
   also looks like "loss won't go down"
4. Wiring: forgotten optimizer.zero_grad(),
   wrong loss reduction, labels misaligned with logits
   (shift, class order), frozen parameters,
   missing model.train()
5. Initialization and input normalization

1. The order matters more than the list — the interviewer is grading exactly this: you check data before architecture, because data breaks orders of magnitude more often.
2. The one-batch overfit — a single check that bisects the hypothesis space: failing to memorize 10 examples rules out "the task is hard" as an excuse.

⚠️ Common mistake: reaching for architecture changes or extra layers first — it's almost always the data or the loop wiring, not model capacity.

Short answer: Schema (columns and dtypes), value bounds, null fractions, categorical domains, row count relative to the last run, and the label distribution. The checks are code in the pipeline that fails loudly BEFORE training — not an analyst's eyes after.

In depth:

def validate(df, ref):
    assert set(df.columns) == set(ref.columns)         # schema
    assert df["age"].between(18, 100).all()            # value bounds
    assert df["income"].isna().mean() < 0.02           # null fraction
    assert set(df["region"]) <= ref.region_domain      # categorical domain
    assert 0.5 < len(df) / ref.row_count < 2.0         # volume vs last run
    assert abs(df["y"].mean() - ref.y_rate) < 0.05     # label distribution

1. Fail loudly vs quarantine — a schema or label violation kills the whole pipeline; isolated broken rows can go to quarantine with an alert, but the allowed fraction is itself a check.
2. Reference from the previous run — half the checks are relative: comparison against pinned statistics of the last successful run.
3. Tools — Great Expectations, pandera; but the interviewer cares about the concept of "validation as a gate", not the library name.

⚠️ Common mistake: validating only the schema. The dtypes match, but a column silently became 40% null — training will "succeed" on garbage.

Short answer: The seed only controls random number generators. What remains: non-deterministic CUDA kernels, cuDNN autotuning, dataloader worker scheduling, non-associative float arithmetic, and library versions. The full answer: seeds + deterministic flags + a pinned environment + versioned data/code/config.

In depth:

1. The cost of determinism — deterministic kernels are noticeably slower; in production teams often pin everything except them and accept noise within a tolerance.
2. Reproducible ≠ bit-exact — the practical standard: the same metric within a stated spread, with full lineage recoverable.

⚠️ Common mistake: "I set seed=42, so it's reproducible" — on a different GPU or cuDNN version the numbers will diverge.

Short answer: Everything the result is built from: the code's git commit, the data version/hash, the full config with hyperparameters, the environment, metrics over steps, and artifacts (checkpoints). Logging metrics alone is the trap: you can't reproduce the winner afterwards.

In depth:

1. Code — the exact commit hash plus a dirty-working-tree flag: an uncommitted patch makes the commit useless.
2. Data — a dataset version or hash (DVC, snapshot): "the same table" a month later is different data.
3. The whole config — every hyperparameter, split, feature version; not just the knobs you turned in this experiment.
4. Environment — a dependency lock file or image, CUDA/driver version.
5. Metrics over steps — train/val curves, not one final number: divergence and early stopping become visible.
6. Artifacts — the checkpoint, tokenizer, preprocessor — attached to the run.

run = code (commit) + data (hash) + config + env
      └─► metrics over steps + artifacts (checkpoint)

⚠️ Common mistake: comparing runs by metric when they silently used different data or splits — the comparison is meaningless without identical lineage.

Short answer: First prove who holds the step instead of guessing: time the wait for each batch against the step itself, then run the same step on one reused batch with no loader. If the step runs back to back and utilization jumps without the loader, the input pipeline is the bottleneck; if not, look for sync points and tiny kernels inside the step.

In depth:

import time
import torch

def time_steps(model, loader, step, n=50, reuse_batch=False):
    # mean per step: waiting for data vs GPU work
    it = iter(loader)
    fixed = next(it)                      # one real batch to reuse
    wait = work = 0.0
    for _ in range(n):
        t0 = time.perf_counter()
        batch = fixed if reuse_batch else next(it)
        t1 = time.perf_counter()
        step(model, batch)                # forward + backward + opt.step
        torch.cuda.synchronize()          # wait for the GPU: honest timing
        t2 = time.perf_counter()
        wait, work = wait + t1 - t0, work + t2 - t1
    return wait / n, work / n

1. Data wait is significant — the loader can't keep up: more num_workers, persistent_workers=True, prefetch_factor; move heavy work out of __getitem__ and offline (pre-tokenize, pre-resize) or decode on the GPU (NVIDIA DALI).
2. Storage — millions of tiny files on a network disk → sharded formats (WebDataset, TFRecord) and a local cache.
3. Wait ≈ 0 but utilization is low — the problem is inside the step: .item(), .cpu() and logging every step are hidden sync points; many tiny kernels → a bigger batch, torch.compile, CUDA graphs.
4. Confirm in the timeline — torch.profiler shows the gaps between kernels and what causes them.

⚠️ Common mistake: asking for a bigger GPU first — that speeds up only the consumer, which is already idle 70% of the time.

Short answer: num_workers spawns separate processes (not threads — because of the GIL) that read and decode batches in parallel with prefetching. pin_memory places the ready batch in page-locked memory, which enables asynchronous copies to the GPU via non_blocking=True.

In depth:

loader = DataLoader(ds, batch_size=64,
    num_workers=8,     # 8 processes decode and prefetch batches
    pin_memory=True)   # batches land in page-locked (pinned) memory

x = batch.to("cuda", non_blocking=True)  # async H2D:
# the copy overlaps with compute — only from pinned memory

1. Processes, not threads — the Python GIL prevents parallel decoding in threads; workers are separate processes that send finished batches back through a queue.
2. Pinned memory — the OS can't page it out, so the DMA copy to the GPU runs without a staging buffer and asynchronously.
3. The cost of workers — each gets its own copy of the Dataset object: pickled under spawn/forkserver, and under fork the copy-on-write pages get copied gradually as Python refcounts touch them. A dataset of millions of Python objects means RAM ×N; keep the data in memory-mapped files and open heavy handles lazily inside each worker.

⚠️ Common mistake: "set num_workers=32, it'll be faster" — more workers than CPU cores means context switching and degradation; tune by measurement, starting from the core count.

Short answer: Each step is idempotent, exchanges only explicit versioned artifacts with its neighbors, is parametrized by the run date (for backfills), and survives retries. Plus two gates: data validation BEFORE training and eval BEFORE promotion to the registry.

In depth:

ingest ──► validate ──► features ──► train ──► eval ──► promote
              │ gate: fail            │          │ gate: metrics
              ▼ before training       ▼          ▼ ≥ threshold
            stop               checkpoint     registry

every step: idempotent · reads/writes versioned
artifacts · parametrized by date · retry-safe

1. Idempotency — re-running a step for the same date yields the same result and duplicates nothing: write to a versioned/dated path, don't append.
2. Explicit artifacts — steps talk through storage (parquet, a checkpoint), not process memory: any step can be re-run in isolation.
3. Date parametrization — run(date) instead of "today": a month-long backfill is just 30 runs.
4. Gates — broken data never reaches training, a weak model never reaches the production registry; promotion always compares against the current champion.

⚠️ Common mistake: listing Airflow operator names instead of step properties — the interviewer is asking about invariants; the orchestrator is secondary.

Short answer: Before saying "drift", rule out code and data divergence between training and serving: two feature code paths, point-in-time violations in the training join, mismatched preprocessing versions, stale online features, serialization losses.

In depth:

1. The delta vs the data scientist's answer — "drift" is a statistical hypothesis; the senior engineering answer first names the code paths where training and prod see different data.

⚠️ Common mistake: retraining "on fresh data" right away — if serving computes the features differently, retraining fixes nothing.

Short answer: For each important feature and for the model's scores, periodically compare a window of production traffic with a reference (the training set or a quiet past period) using an effect-size metric — for example, PSI over the reference's quantile bins. A KS-test p-value on millions of events is significant for any negligible shift, so an alert on it fires constantly.

In depth:

import numpy as np
from scipy import stats

def psi(ref, cur, bins=10, eps=1e-4):
    # Population Stability Index of window cur against reference ref
    cuts = np.quantile(ref, np.linspace(0, 1, bins + 1)[1:-1])
    r = np.bincount(np.searchsorted(cuts, ref), minlength=bins) / len(ref)
    c = np.bincount(np.searchsorted(cuts, cur), minlength=bins) / len(cur)
    r, c = np.maximum(r, eps), np.maximum(c, eps)   # empty bin: no log(0)
    return float(np.sum((c - r) * np.log(c / r)))

rng = np.random.default_rng(0)
ref = rng.normal(size=200_000)                      # reference window
for shift in (0.02, 0.1, 0.5):                      # mean shift in σ
    cur = rng.normal(shift, size=1_000_000)         # one day of traffic
    p = stats.ks_2samp(ref, cur).pvalue
    print(f"{shift:<4} PSI={psi(ref, cur):.4f} KS p={p:.1g}")
# 0.02 PSI=0.0005 KS p=4e-11    ← KS raises the alarm, PSI stays calm
# 0.1  PSI=0.0096 KS p=4e-234
# 0.5  PSI=0.2365 KS p=0

1. Bins from the reference — the cut points are the reference's quantiles and are frozen with it; an empty bin gets ε to avoid log(0).
2. Thresholds — the usual rule: PSI < 0.1 stable, 0.1–0.2 worth a look, > 0.2 alert. Calibrate it on your own features' history by comparing quiet weeks with each other.
3. What to monitor — the top features by importance, null and default rates, the score distribution; categorical features by category frequencies, with a separate bin for unseen values.
4. Against noise — a large enough window (an hour or a day), and alert only when the threshold is exceeded for several windows in a row; with strong seasonality use the same weekday as the reference.

⚠️ Common mistake: alerting on p-values at production volume — with a million events a shift of a few hundredths of σ becomes significant, and on-call quickly learns to ignore the alert.

Short answer: This is a dependency change, not a model-name string change. Record a baseline, run the old and new models on the same data with paired tests, check API compatibility separately, then go through shadow and canary with automatic rollback. Keep the old model reachable until the retirement date: after it, requests simply fail.

In depth:

1. The API breaks, not just quality — a new model or API version can reject what the old one accepted. Examples from the Claude API docs as of November 2026: prefilling the last message returns a 400 on models 4.6 and later; a forced tool_choice returns a 400 on several of the newest models; temperature, top_p and top_k return a 400 in API version 2026-10-21, even at their default values.
2. The bill changes too — prices and the hidden tool prompt differ between models: on the Claude API in November 2026 it is 497 tokens on Opus 4.6 and 286 on Opus 5.5. A KV cache is computed by a specific model's weights, so after the switch the prompt cache is cold: expect a spike in cost and latency, and raise the share gradually.
3. Prompts — what was tuned for one model often needs retuning. Do it through evals, not by blindly adding instructions. Version the prompt and the model together.
4. Timing — Anthropic's minimum as of November 2026 is 60 days' notice for public models, so you may get less than 90; keep the eval ready at all times.
5. Rollback — the old model stays the rollback target until the last day. After retirement there is nowhere to go back to, so finish the canary early.

⚠️ Common mistake: changing the model string in production after a couple of manual checks, or relying on a floating alias that changes behavior without you.

Short answer: Not on these numbers. On 300 prompts the standard error of accuracy is about 2.6 points, so a 3-point gap is comparable to noise. Compare the versions with a paired test on the same prompts: in the example of 210 → 219 out of 300, McNemar gives p = 0.23 and the 95% interval for the gain runs from −1.3 to +7.3 points. From there you either gather more prompts and run an online A/B, or apply a deliberate non-inferiority criterion.

In depth:

1. Pairing — only the disagreements matter, 27 versus 18. For exact McNemar this is a binomial test at p = 0.5. The paired standard error of the difference is 2.2 points and the unpaired one is 3.7; an unpaired z-test would give z = 0.81 and p = 0.42.
2. Sample size — n ≈ (z₀.₉₇₅ + z₀.₈)²·d/δ² = 2.8²·0.15/0.03² ≈ 1300 with a disagreement rate d = 15% and a true gain δ = 3 points. On 300 prompts power is only about 27% by this normal approximation and about 21% for exact McNemar, which is conservative with so few disagreements. Large effects need far smaller sets.
3. Continuous judge scores — a paired bootstrap or permutation test on per-prompt differences. At temperature > 0, first average the repeats within a prompt, then cluster by scenario.
4. Slices — language, intent, length. Correct for the number of metrics, and do not hunt for the one slice that went up. Check any LLM judge against humans and keep part of the set held out.
5. The decision — ship if the lower bound of the paired interval beats a margin fixed in advance and the guardrail slices did not drop. With a −2-point margin and a cheaper new version, this data is already enough.

import numpy as np
from scipy.stats import binomtest

only_b, only_a = 27, 18
print(binomtest(only_b, only_b + only_a, 0.5).pvalue)   # 0.2327

d = np.r_[np.ones(27), -np.ones(18), np.zeros(255)]     # B minus A
idx = np.random.default_rng(0).integers(0, 300, (10_000, 300))
print(np.percentile(d[idx].mean(axis=1), [2.5, 97.5]))  # -0.013, 0.073

⚠️ Common mistake: comparing two headline numbers ("73 > 70, ship it") with no interval, or using an unpaired test and concluding there is no signal when the paired design is markedly more precise.

Short answer: temperature = 0 is only an argmax over logits, and the logits themselves differ in the low bits under different load: batch size changes the reduction order in the kernels, and floating-point addition is not associative. Near-tied tokens swap places, and the answer diverges from that token on. The fix is batch-invariant kernels (in vLLM, VLLM_BATCH_INVARIANT=1, beta), or you accept nondeterminism and evaluate statistically.

In depth:

batch 1:   reduction A → logits …7f3a
batch 64:  reduction B → logits …7f3b
near-tied tokens: the argmax flips,
answers match up to that token and diverge after it

1. The cause — per Thinking Machines' analysis (September 2025), the forward pass is reproducible for a fixed batch. Load is the primary source: batch size varies from request to request, and RMSNorm, matmul and attention compute differently depending on it. The authors reject the popular story about atomic adds and concurrency.
2. The experiment — Qwen3-235B-A22B-Instruct-2507, temperature 0, 1,000 runs of one prompt: 80 distinct answers. The first 102 tokens matched, and at token 103 they split 992 versus 8. With batch-invariant kernels all 1,000 answers were identical.
3. The price — from the same post, 1,000 sequences on Qwen-3-8B: 26 s in default vLLM, 55 s in the unoptimized deterministic mode, 42 s with an improved attention kernel.
4. Limits — vLLM's mode is in beta and runs only on listed hardware; under tensor parallelism, all-reduces must be a multiple of 16 bytes. The result is not portable either: another GPU or PyTorch version can give different bits.
5. Why you would want it — stable regression tests and golden files, debugging, RL where the sampler and trainer must agree. Otherwise, repeated runs and paired tests are simpler.
6. Hosted APIs — batching is out of your hands. The Claude API has had no temperature parameter at all since API version 2026-10-21: a request with it returns a 400.

import numpy as np

x = np.random.default_rng(0).standard_normal(4096).astype(np.float32)
whole = x.sum()
by_16 = np.array([c.sum() for c in np.split(x, 16)]).sum()
print(whole, by_16, whole == by_16)   # -66.0646 -66.064606 False
print((0.1 + 0.2) + 0.3 == 0.1 + (0.2 + 0.3))   # False

⚠️ Common mistake: blaming the seed or "GPU randomness" and proposing to fix the seed: greedy decoding has no sampling to seed, and the main cause in this analysis is a reduction order that depends on batch size.