Data Scientist interview prep

Short answer: A Series is a one-dimensional labelled array (a single column); a DataFrame is a two-dimensional table of column-Series sharing a row index. You load data with pd.read_csv() / pd.read_sql() and inspect it with .head(), .info(), .describe(), .dtypes, .shape.

In depth:

1. Series vs DataFrame — df['col'] returns a Series; df[['col']] (double brackets) returns a one-column DataFrame. Both carry an .index.
2. Loading — read_csv (file/URL), read_sql (a SQL query plus a connection), read_parquet (fast columnar format).
3. Inspecting — .head()/.tail() peek at the edges, .info() shows dtypes and null counts, .describe() gives stats on numeric columns, .dtypes lists types, .shape gives size.

import pandas as pd

df = pd.read_csv("sales.csv", parse_dates=["order_date"])
df.shape          # (10000, 8) — rows, columns
df.info()         # column dtypes and non-null counts
df.describe()     # count/mean/std/min/quartiles/max for numerics
df["revenue"].head()   # Series — a single column

⚠️ Common mistake: forgetting parse_dates= on load — the date column stays object (a string), so date arithmetic and resample() won't work.

Short answer: The mean is the sum divided by the count and is sensitive to outliers. The median is the middle value of the ordered data and is robust to outliers. The mode is the most frequent value. For skewed data the median better reflects the "typical" observation.

In depth:

1. Mean — good for symmetric data and downstream math (variance, regression), but a single outlier shifts it.
2. Median — the 50th percentile; splits the data in half and barely reacts to extremes.
3. Mode — the only one of the three that works for categorical data ("most common plan").

⚠️ Common mistake: reporting the mean for income or latency. A long right tail drags the mean upward, so it no longer describes the typical user — use the median.

Short answer: An A/B test is a randomized experiment: users are split at random into control and treatment, shown different versions, and compared on a single pre-chosen primary metric under a fixed decision rule.

In depth:

1. Hypothesis — what you change, which metric it should move, and why.
2. Metric and MDE — one primary metric plus a minimum detectable effect (MDE).
3. Randomization — a random, stable split into control/treatment.
4. Sample size and duration — compute and fix them before launch.
5. Run to N — run until the sample is reached, without peeking at interim results.
6. Analysis — compare the metric, compute the p-value and the confidence interval of the lift.
7. Decision — ship or roll back per the pre-defined rule.

Hypothesis → Randomize → Run to N → Analyze → Decision
              │                                  │
       control / treatment              ship ✓ / rollback ✗

⚠️ Common mistake: changing the UI mid-test or stopping the moment it first looks 'significant' — both break the conclusions.

Short answer: A window function computes an aggregate over a "window" of rows but does not collapse them — every source row survives and gets its own value. GROUP BY instead folds each group into a single row.

In depth:

1. GROUP BY — N rows in a group → 1 result row. Detail is lost.
2. Window function — N rows stay N rows, with an aggregate (sum, rank, average, lag) added alongside.
3. Syntax — func() OVER (PARTITION BY ... ORDER BY ...). PARTITION BY defines the groups, ORDER BY sets order within the window (needed for running totals and ranks).

-- Running total of sales per user
SELECT
  user_id,
  order_date,
  amount,
  SUM(amount) OVER (
    PARTITION BY user_id
    ORDER BY order_date
  ) AS running_total
FROM orders;

⚠️ Common mistake: trying to filter on a window function's result in WHERE — windows are evaluated after WHERE/GROUP BY, so wrap the query in a CTE/subquery and filter on the outside.

Short answer: Overfitting means the model learned the noise of the training set instead of the pattern: low error on train, high error on new data. You detect it via the train/validation gap and fix it with regularization, a simpler model, early stopping, and more data.

In depth:

1. Detection — compare train vs validation error; learning curves: train error keeps falling while validation error stalls or rises → the model started memorizing noise.
2. Treatment — reduce capacity or add signal, don't just chase the metric.

⚠️ Common mistake: saying "cross-validation prevents overfitting." CV only detects it — it gives an honest quality estimate. Prevention comes from regularization, simpler models, and data.