Data Engineer interview prep
A spaced-repetition deck of 450+ Data Engineer interview questions — organised by topic and difficulty, and scheduled for review based on your ratings. Preview a few cards below, then choose access to study the whole track on an Anki-style SM-2 schedule.
Eligible new monthly or yearly subscribers get 7 days free. Checkout confirms eligibility and charges.
Before you start
You should be able to write basic Python and SQL and understand tables and files. Spark, streaming, and architecture build on those foundations; use a small pipeline for practical exercises.
First pass: Python → CS Fundamentals. Begin with the foundational questions in these modules, then follow the outline. Revisit advanced and senior questions as the job requires; all cards remain available from the start.
Shared topics can appear in several tracks and keep one review history. The outline labels shared foundations and role-specific modules; choose your depth for the job you are preparing for.
What's covered
Every topic in this track, grouped the way you'd study it.
Python
102 cardsShared foundations
CS Fundamentals
53 cardsShared foundations
Databases
82 cardsShared foundations
Data Modeling & Warehousing
12 cardsRole-specific module
ETL/ELT & Pipelines
18 cardsRole-specific module
Storage & File Formats
13 cardsRole-specific module
Spark & Distributed Processing
13 cardsRole-specific module
Streaming & Kafka
14 cardsRole-specific module
Kafka & Messaging
21 cardsShared foundations
SQL & Query Optimization
10 cardsRole-specific module
ClickHouse
8 cardsRole-specific module
Data Quality & Orchestration
10 cardsRole-specific module
DevOps & Infra
47 cardsShared foundations
Data Architecture & System Design
9 cardsRole-specific module
Behavioral
38 cardsShared foundations
Sample questions
A few cards from the deck — reveal each answer, then choose access to study the full set on a schedule.
What's the difference between mutable and immutable types?
What's the difference between mutable and immutable types?
Short answer: Mutable objects can change in place (list, dict, set); immutable objects cannot (int, str, tuple). Rebinding a variable is different from mutating its object.
items = [1, 2]
alias = items
items.append(3)
assert alias is items and alias == [1, 2, 3]
text = "hello"
original = text
text += " world"
assert original == "hello" # the original string did not change
In depth:
id()identifies an object and stays constant during its lifetime. Treating it as a memory address is a CPython implementation detail.- A tuple cannot replace its elements, but a list stored inside it can still change. Immutability is not automatically deep immutability or thread safety.
- Dictionary keys and set elements must be hashable: their hash stays stable, and equal objects have equal hashes. A tuple containing a list is unhashable. A user-defined mutable instance can be hashable by identity, so hashability is not synonymous with immutability.
- Functions receive references to objects. Mutating an argument can affect the caller; rebinding a local parameter does not rebind the caller's variable.
Sources: Python hashability, object identity.
What is the difference between concurrency and parallelism?
What is the difference between concurrency and parallelism?
Short answer: Concurrency is about the structure of a program (managing several tasks that make progress, but not necessarily at the same time). Parallelism is about physical simultaneous execution on multiple cores.
In depth:
- Concurrency — tasks switch and make progress "in turns," creating the illusion of simultaneity. One cook prepares 3 dishes, switching between them. It can happen on a single core.
- Parallelism — tasks actually execute in the same physical second. Three cooks, each preparing their own dish. Requires multiple cores/processors.
Concurrency (1 core): Parallelism (2 cores):
CPU: A B A B A B CPU0: A A A A
CPU1: B B B B
In Python:
asyncioandthreading(due to the GIL) give concurrency, but not real parallelism for CPU code.multiprocessinggives real parallelism.
💡 "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once." — Rob Pike.
⚠️ Gotcha: many people confuse the two and say "threads in Python run in parallel." For CPU-bound code that's not the case, because of the GIL.
Why do we need type hints if Python is dynamic?
Why do we need type hints if Python is dynamic?
Short answer: Type annotations are not checked by the interpreter at runtime — they exist for static analysis (mypy, pyright), IDE autocompletion, documentation, and catching errors early, before the code runs. Python stays dynamic; hints are a "contract" that external tools verify.
In depth:
Type hints (PEP 484) don't affect program execution. The interpreter ignores them — you can pass a string where an int is expected, and the code won't fail on its own.
def add(a: int, b: int) -> int:
return a + b
# Python does NOT check types — this runs without errors:
print(add("a", "b")) # "ab" — string concatenation!
# But mypy reports an error BEFORE running:
# error: Argument 1 to "add" has incompatible type "str"; expected "int"
So why are they useful:
- Early error detection —
mypy/pyrightcatch type mismatches before production. - Autocompletion and navigation in the IDE (PyCharm, VS Code).
- Documentation — the signature
def f(x: list[int]) -> dict[str, int]is clearer than a comment. - Refactoring — when a type changes, the tool shows every place it's used.
- Contracts in large teams — they reduce "implicit" agreements.
# Without hints — it's unclear what's expected:
def process(data, config): ...
# With hints — self-documenting:
def process(data: list[dict[str, int]], config: "Config") -> bool: ...
⚠️ Gotcha: Annotations are not validation: def f(x: int) won't prevent passing a string at runtime. To validate incoming data, use pydantic or explicit checks that raise — not assert, which is stripped when you run python -O.
What is Big O notation?
What is Big O notation?
Short answer: Big O describes how an algorithm's running time or memory usage grows as the input size n increases, dropping constants and lower-order terms. It's an upper bound on the growth rate.
In depth:
Big O answers the question "what happens when n becomes very large?". We're not interested in the exact number of operations, only the nature of the growth. That's why O(2n + 100) simplifies to O(n), and O(3n² + n) — to O(n²).
The main growth classes (from best to worst):
# O(1) — constant: doesn't depend on n
def first(arr):
return arr[0] if arr else None
# O(log n) — logarithmic: each step halves the problem (binary search)
def binary_search(arr, target):
lo, hi = 0, len(arr) - 1
while lo <= hi:
mid = (lo + hi) // 2
if arr[mid] == target:
return mid
elif arr[mid] < target:
lo = mid + 1
else:
hi = mid - 1
return -1
# O(n) — linear: a single pass
def total(arr):
s = 0
for x in arr: # n iterations
s += x
return s
# O(n log n) — efficient sorts (merge, quick on average, Timsort)
def sort_it(arr):
return sorted(arr)
# O(n^2) — quadratic: a nested loop
def has_dup_naive(arr):
for i in range(len(arr)):
for j in range(i + 1, len(arr)):
if arr[i] == arr[j]:
return True
return False
# O(2^n) — exponential: naive Fibonacci, enumerating subsets
def fib_naive(n):
if n < 2:
return n
return fib_naive(n - 1) + fib_naive(n - 2)
How to estimate code: sequential steps add up and the dominant term remains (O(n) + O(n log n) = O(n log n)); nested loops multiply (O(n·m), or O(n²) when m = n); discarding half the problem with constant work per step gives O(log n). With two inputs, keep both variables: O(n + m) can't be reduced to O(n) unless you know that m ≤ n.
⚠️ Gotcha: Big O is about asymptotics (behavior at large n), not about actual time. An O(n) algorithm can be slower than an O(n²) one on small data because of large constants. People also confuse: Big O (upper bound), Ω (lower bound), Θ (tight bound); in interviews "O" usually means Θ.
What does SELECT do, and in what order do the parts of a query execute?
What does SELECT do, and in what order do the parts of a query execute?
Short answer: SELECT retrieves rows from tables. The logical execution order does NOT match the written order: first FROM, then WHERE, GROUP BY, HAVING, SELECT, DISTINCT, ORDER BY, LIMIT.
In depth:
Written (syntactic) order:
SELECT DISTINCT col1, agg(col2)
FROM t
WHERE cond
GROUP BY col1
HAVING agg_cond
ORDER BY col1
LIMIT 10 OFFSET 20;
Logical execution order:
1. FROM / JOIN -- which tables, how to join
2. WHERE -- row filter BEFORE grouping
3. GROUP BY -- grouping
4. HAVING -- group filter AFTER aggregation
5. SELECT -- expressions, aliases, window functions
6. DISTINCT -- remove duplicates
7. ORDER BY -- sorting
8. LIMIT / OFFSET -- slice
⚠️ Gotcha: an alias from SELECT cannot be used in WHERE (since WHERE runs before SELECT), but it can be used in ORDER BY and often in GROUP BY (depends on the DBMS). Example of the error:
SELECT salary * 12 AS annual FROM emp WHERE annual > 100000; -- ERROR
SELECT salary * 12 AS annual FROM emp ORDER BY annual; -- OK
What is an index and why is it needed?
What is an index and why is it needed?
Short answer: An index is a separate data structure that stores the values of one or more columns in a sorted/organized form along with pointers to the physical location of rows (TID — tuple identifier), so the database can find rows without scanning the entire table.
In depth: Without an index, PostgreSQL is forced to read the whole table (Sequential Scan) and check every row. An index lets it find the needed TIDs in O(log n) (for a B-tree) and then fetch the rows themselves from the heap (the table's main storage area).
CREATE INDEX idx_users_email ON users (email);
-- Now the query searches the tree, not the whole table
SELECT * FROM users WHERE email = 'a@b.com';
Physically, a table in Postgres is stored as a heap — an unordered set of 8 KB pages. An index is a "table of contents" for this heap. A single table can have many indexes.
⚠️ Gotcha: An index speeds up reads but slows down writes (each INSERT/UPDATE/DELETE must keep the index current). Adding an index on every column "just in case" is an antipattern.
Ready to make it stick?
Explore a sample, choose a track, and build a review routine that fits your preparation.
Questions about this track
How should I prepare for a Data Engineer interview?
Study the concepts you'll be asked to explain, not just the ones you can code. RecallDeck's Data Engineer track gives you 450+ curated interview questions and resurfaces each one with an Anki-style SM-2 schedule based on your ratings — to practise recalling and explaining them. Combine reviews with coding and mock interviews.
What topics does the Data Engineer track cover?
The Data Engineer track is organised into the core areas Data Engineer interviews actually test, grouped by topic and by difficulty (Concept, Junior, Middle, Senior). You can preview the full outline and sample questions above before signing in.
Is spaced repetition effective for Data Engineer interview prep?
Yes. Active recall and spaced practice can help retain what you study. Grade yourself honestly and check your understanding with practical work. RecallDeck schedules each Data Engineer card using your ratings, so familiar cards return less often and difficult ones get more practice.
Can I try the Data Engineer track before paying?
Eligible new monthly or yearly subscribers can try the complete Data Engineer track and every feature for 7 days. Checkout confirms trial eligibility and the amount due; returning subscribers may be charged immediately. Cancel online before the trial ends to avoid its first charge. Lifetime access has no trial.