29 Backend Architecture and Scaling Interview Questions and Answers

Short answer: Vertical (scale up) — add resources to a single machine (CPU/RAM). Horizontal (scale out) — add more machines and distribute the load across them.

In detail:

• Vertical (scale up/down): get a beefier server. Simple (the code doesn't change), but there's a physical ceiling, it's expensive at the top end, and the machine itself remains a single point of failure. Upgrading often requires downtime.
• Horizontal (scale out/in): add nodes behind a load balancer. A practically unlimited ceiling, fault tolerance (one node dies — the rest keep running), linear cost. But it requires a stateless architecture, load balancing, distributed state, and it complicates consistency and debugging.
• In practice you first squeeze out vertical scaling (quick and cheap up to a certain point), then move to horizontal. Databases are the hardest to scale horizontally (hence sharding/replicas).

⚠️ Gotcha: "let's just add more servers" doesn't work if the service is stateful (stores sessions/files in memory/on the local disk). First make the service stateless, then scale.

Short answer: A stateless service doesn't store data locally between requests — any request can be handled by any instance. A stateful one keeps state (session, cache, files) inside itself, which pins a client to a specific instance.

In detail:

• Why stateless is easier to scale: instances are interchangeable. You can add/remove nodes, restart, do a rolling deploy without losing data. The load balancer can throw a request at any node. No sticky sessions needed.
• Where to store state: move it outside into shared storage:
  • sessions → Redis / Memcached / DB (or JWT entirely — state lives on the client);
  • files/uploads → object storage (S3), not the local disk;
  • cache → distributed Redis;
  • task queues → a broker (RabbitMQ/Kafka).
• The service itself becomes a "function": input → processing → output, with no internal memory between requests.

💡 Stateless doesn't mean "no state at all" — the state simply lives in external systems shared by all instances.

⚠️ Gotcha: hidden state — an in-memory cache, local temp files, in-process schedulers (a cron inside an instance will run N times across N instances). All of this breaks horizontal scaling.

Short answer: An L4 balancer operates at the transport layer (TCP/UDP, IP+port) and doesn't look at the content. L7 operates at the application layer (HTTP), sees the URL, headers, cookies, and can route based on them.

In detail:

• L4 (transport): fast, cheap on CPU, proxies packets/connections by IP and port. Doesn't understand HTTP. Examples: AWS NLB, IPVS, HAProxy in TCP mode.
• L7 (application): parses HTTP, can do content-based routing (/api → one pool, /static → another), terminates TLS, adds headers, does retry/circuit-breaking. More resource-intensive. Examples: nginx, AWS ALB, Envoy, HAProxy in HTTP mode.

Balancing algorithms:

• Round-robin — in a circle, evenly. Good when requests/servers are uniform.
• Weighted round-robin — with weights (a powerful server gets more).
• Least connections — to the node with the fewest active connections. Good for requests of varying duration.
• IP hash — the node is chosen by a hash of the client's IP → one client always goes to one node (primitive "stickiness").
• Least response time / random two choices — more advanced variants.

Health checks: the balancer periodically pings the nodes (active: HTTP /health; passive: watches errors on real traffic). An unhealthy node is removed from the pool so traffic doesn't go to a dead instance.

Sticky sessions (session affinity): the client is "glued" to one node (by cookie or IP hash) so a local session isn't lost. This is a crutch for stateful services — better to make them stateless + external session storage.

⚠️ Gotcha: sticky sessions break load uniformity and fault tolerance — if a node dies, all clients glued to it lose their session. It doesn't cure the problem, it just defers it.

Short answer: These are overlapping but distinct roles. A reverse proxy is an intermediary in front of the server(s): TLS, cache, compression, routing. A load balancer distributes load across several backends. One product (nginx) does both.

In detail:

• A reverse proxy hides the backend from the client and takes on cross-cutting functions: TLS termination, gzip/brotli, caching of static content and responses, security headers, rate limiting, serving static files, buffering. It can sit in front of a single server.
• A load balancer is a special case/function: it spreads requests across a pool, does health checks. It can be L4 (without knowing HTTP).
• Difference in focus: an LB answers the question "which of the N nodes," a reverse proxy answers "how to process/transform the request before the backend." nginx/Envoy/HAProxy combine both roles.

💡 In an interview: "nginx as a reverse proxy + L7 balancer in front of stateless application instances, with TLS terminated on it."

⚠️ Gotcha: don't confuse it with a forward proxy — that one sits on the client side and proxies the client's outbound traffic to the outside (e.g., a corporate proxy). A reverse proxy sits on the server side.

Short answer: There's a cache at every layer of the request path: browser → CDN → reverse proxy → application → DB. The closer to the client, the cheaper and faster, but the harder invalidation gets.

In detail (layers top to bottom):

1. Browser — HTTP cache via the Cache-Control, ETag, Last-Modified headers. The cheapest: the request doesn't go out at all.
2. CDN / edge — caches static content and cacheable responses geographically closer to the user.
3. Reverse proxy (nginx) — caches responses/pages on the data-center side, offloading the application.
4. Application — in-memory (local, fast, but not shared) and distributed (Redis/Memcached — shared by all instances). Caching of computed results, sessions, aggregates.
5. DB — buffer pool/page cache, query plan cache, materialized views.

Strategies: cache-aside (the application reads/writes the cache itself — the most common), write-through, write-behind, read-through. Key problems: invalidation, TTL, cache stampede (many misses at once → protect with locks/SETNX), cache consistency with the DB.

📎 Details on Redis (data structures, patterns, eviction, persistence) — see the separate redis file.

⚠️ Gotcha: "Phil Karlton: there are two hard things in CS — cache invalidation and naming things." Caching solves reads but adds the problem of stale data. Don't cache anything that must be strictly consistent without well-thought-out invalidation.

Short answer: Content Delivery Network — a geographically distributed network of edge servers that cache content closer to the user, to reduce latency and offload the origin.

In detail:

• What it caches: static content (images, CSS/JS, video), and with edge logic — dynamic/API responses too.
• Why: lower RTT (content is physically closer), offloading the origin server, DDoS protection, TLS termination at the edge, absorbing traffic spikes.
• Edge: points of presence (PoPs) around the world; a request goes to the nearest edge, on a miss it goes to the origin, and the response is cached. Edge computing — running lightweight code at the edge (Cloudflare Workers, Lambda@Edge).
• Invalidation — via purge/URL versioning (a hash in the filename → app.a1b2c3.js).

⚠️ Gotcha: don't put private/personal data into a public CDN cache without Cache-Control: private. Otherwise one user will see another's cached response.

Short answer: To make interaction asynchronous, decouple services, smooth out load spikes, and increase the reliability of getting work done.

In detail — four main motives:

1. Asynchrony: the request returns a response immediately, while the heavy work (sending an email, generating a report, processing video) runs in the background. The user doesn't wait.
2. Decoupling: the producer doesn't know about the consumer and doesn't depend on its availability/speed. You can deploy, scale, and change consumers independently.
3. Load leveling / buffering: a traffic spike fills the queue, and consumers work through it at their own pace. The DB/service doesn't get overwhelmed.
4. Reliability: the message is stored in the broker; if a consumer crashes — the message isn't lost and will be processed later (with ack/retry).

Bonus: scaling consumers (add workers — clear the backlog faster), a buffer between a fast and a slow component.

⚠️ Gotcha: a queue isn't free — it adds infrastructure, eventual consistency, debugging complexity, and a requirement for consumer idempotency. Don't insert a queue where you need a synchronous response here and now.

Short answer: RabbitMQ — a classic broker (smart routing, the message is deleted after processing). Kafka — a distributed event log (messages are retained, read by offset, great for streaming/large volumes). Redis (Streams/Pub-Sub) — simple and fast, in the same process as the cache. SQS — a managed queue in AWS, minimal operational overhead.

In detail — broker vs log (the key distinction):

• Broker (RabbitMQ, SQS): push/pull messages, the broker tracks delivery state; after ack the message is deleted. Rich routing (exchanges, routing keys, priority). A queue usually has one consumer per message (work queue).
• Log (Kafka): messages are written append-only into partitions and retained (by retention/size), even after being read. Consumers hold their own offset — they can reread history, and different groups read independently. Order is guaranteed within a partition. Scales to enormous throughput.

When to use which:

• RabbitMQ — tasks, RPC, complex routing, relatively moderate volume, when you need per-message guarantees and priorities.
• Kafka — event sourcing, analytics/streaming, change log, millions of messages/sec, when you need rereads and one event delivered to many consumers.
• Redis Streams — lightweight queues when Redis is already there; fewer durability guarantees, simpler.
• SQS — "I don't want to administer a broker," AWS stack; standard (at-least-once, no strict ordering) or FIFO (ordering + dedup).

⚠️ Gotcha: Kafka isn't "the best queue," it's a different tool. For a classic work queue with ack/retry/priorities, RabbitMQ is more convenient. Kafka is strong where you need a log, rereads, and streaming. Choose by the task, not by what's trendy.

Short answer: at-most-once — at most once (loss possible). at-least-once — at least once (duplicates possible). exactly-once — exactly once (expensive and conditional, usually achieved via at-least-once + idempotency/deduplication).

In detail:

• At-most-once: fire and forget, no retries. The message may be lost, but there are no duplicates. Suitable for metrics/telemetry, where loss isn't critical.
• At-least-once: ack + retries. If the ack doesn't arrive — retry. Duplicates are possible (processed, crashed before ack → re-sent). The most common default.
• Exactly-once: the "holy grail." In its pure form it's impossible in a distributed system because of network failures; in practice it's emulated: at-least-once delivery + an idempotent consumer or deduplication by message_id. Kafka provides exactly-once within its own ecosystem (transactions producer→topic→consumer-offset), but once you go outside (writing to a third-party DB) you again need idempotency.

Idempotent consumers: processing the same message N times has the same effect as processing it once. Implementation: a unique message_id → a store of processed IDs / UPSERT by a business key / conditional updates. This is the right way to "survive" duplicates.

Dead Letter Queue (DLQ): messages that couldn't be processed after N retries (or with an expired TTL, or invalid ones) go to a separate queue. There they're handled manually/via an alert, without losing them and without blocking the main queue with "poison" messages (poison messages).

⚠️ Gotcha: "we have exactly-once" almost always means "at-least-once + idempotency." If the consumer isn't idempotent, any retries (and there will be some) will lead to duplicated effects: two money debits, two emails.

Short answer: In a work queue a message is received by one consumer (competing workers share the work). In pub/sub a message is received by all subscribers (broadcast).

In detail:

• Work queue (point-to-point): N workers on one queue share the messages — each goes to exactly one. The goal is to parallelize processing. Example: a Celery task queue, order processing.
• Pub/Sub (publish-subscribe): the publisher sends to a topic, each subscriber gets a copy. The goal is to notify many independent consumers. Example: an "order created" event → billing, warehouse, analytics, notifications, each doing its own thing.
• In RabbitMQ this is configured via the exchange (fanout = pub/sub, direct/topic-routing). In Kafka each consumer group gets all messages (pub/sub between groups), while within a group the partitions are split between consumers (work queue).

⚠️ Gotcha: if you hang several semantically different handlers on one work queue, they'll start "stealing" messages from each other. For broadcast you need fanout/topic, not a shared queue.

Short answer: Celery is a framework for background tasks in Python. Tasks (@app.task) are placed into a broker (RabbitMQ/Redis), workers pick them up and execute them; the result is optionally written to a result backend.

In detail:

• Components:
  • Task: a function marked with @app.task, invoked via .delay() / .apply_async() — this publishes a message to the broker.
  • Broker: the message transport (RabbitMQ is preferred, Redis is popular for simplicity).
  • Worker: a process that listens to the queue and executes tasks; scales by the number of processes/threads and the number of workers.
  • Result backend: storage for results/statuses (Redis, DB) — optional, by default the result need not be stored.
• When to use: sending email/SMS, generating reports/PDFs, media processing, calls to slow external APIs, retries with backoff — anything that shouldn't block the HTTP response.
• Periodic tasks: Celery Beat — a scheduler (cron-like) that throws tasks into the queue on a schedule (one Beat instance per cluster, so as not to duplicate!).
• Features: retries (max_retries, retry_backoff), acks_late (ack after execution — at-least-once), routing by queues, chains/groups (canvas: chain, group, chord).

⚠️ Gotcha: Celery tasks must be idempotent — with acks_late and a worker crash the task will be re-executed. And don't run several Celery Beat instances — you'll get duplicated periodic tasks.

Short answer: Synchronous — the caller blocks and waits for the response (REST/gRPC request-response). Asynchronous — it sends a message/event and doesn't wait; the response (if needed) comes later or via a callback/event.

In detail:

• Synchronous: simple, clear, an immediate result and error. But it creates temporal coupling — both services must be alive at the same time; a crash/slowdown of the callee hits the caller; chains of synchronous calls multiply latency and the risk of cascading failures.
• Asynchronous (via a queue/event): decoupling in time, resilience to unavailability, smoothing of spikes. But eventual consistency, harder to trace the flow, and you need a handler for "so where's the result."
• Rule: synchronous — when you need an immediate response for the user (fetch a profile). Asynchronous — when the work can wait or decoupling matters (after placing an order: notifications, awarding bonus points).

⚠️ Gotcha: long synchronous chains A→B→C→D are an anti-pattern: total latency adds up, while availability multiplies (0.99⁴ ≈ 0.96). Some of it gets moved to asynchronous events.

Short answer: A monolith — one application/deployment. Microservices — many small, independently deployable services around business domains. Microservices give independence and scalability at the cost of huge operational and distributed complexity.

In detail:

• Monolith — pros: simplicity of development/deployment/debugging, transactions in a single DB, no network calls between modules, easier refactoring. Cons: scales only as a whole, a single point of failure at the process level, a large codebase, risky deploys, lock-in to one stack.
• Microservices — pros: independent deployment/scaling/stack choice per service, failure isolation, teams own services autonomously. Cons: distributed transactions and consistency, network failures and latency, complex observability/deployment/testing, you need mature infrastructure (CI/CD, monitoring, orchestration).
• When to use which: start with a (modular) monolith — it serves most needs for a long time. Microservices are justified with large teams, differing load on parts of the system, differing scaling/release requirements. A "modular monolith" is an excellent compromise.
• Service size: around a business capability / bounded context (DDD), its own DB, owned by one team. Not "per table" and not "nano-services."

💡 Distributed monolith (anti-pattern): services that look separate but are so coupled (synchronous chains, a shared DB, joint deployment) that they got the downsides of both worlds — the complexity of distribution without its benefits. Signs: you can't deploy one service without the others; a shared DB schema; one goes down — all go down.

⚠️ Gotcha: microservices are an organizational decision and an infrastructure tax, not "trendy/scalable by default." Without mature DevOps they'll slow the team down, not speed it up.

Short answer: Synchronous — REST or gRPC (request-response). Asynchronous — events through a broker. From the outside — an API Gateway. Network cross-cutting functions — optionally a service mesh.

In detail:

• REST (HTTP/JSON): universal, human-readable, cacheable, easy to debug. Slower, verbose.
• gRPC (HTTP/2 + protobuf): binary, fast, a strict contract (.proto), streaming, code generation. Worse for the browser/debugging. Good for high-throughput internal traffic.
• Asynchronous events: publishing events to a broker (Kafka/RabbitMQ) — maximum decoupling, event-driven architecture.
• API Gateway: a single entry point for clients: routing to services, authentication, rate limiting, response aggregation, versioning, TLS. Hides the internal topology.
• Service mesh (briefly): an infrastructure layer (Istio/Linkerd, the Envoy sidecar proxy) for service-to-service traffic: mTLS, retries, timeouts, circuit breaking, tracing, balancing — moved out of the code into the platform. Needed when there's a large number of services.

⚠️ Gotcha: an API Gateway (north-south, client↔system) ≠ a service mesh (east-west, service↔service). Different jobs, often used together.

Short answer: Without a shared DB you can't have a single ACID transaction across services. The options: 2PC (strong but fragile), Saga (a sequence of local transactions with compensations), eventual consistency, Outbox for reliable event publishing.

In detail:

• 2PC (two-phase commit): coordinator → prepare to all participants → if all ok, commit. Gives atomicity, but is blocking, the coordinator is a single point of failure, scales poorly, increases latency. In practice, avoided in microservices.
• Saga pattern: a business transaction = a chain of local transactions in different services; on a step failure, compensating actions are triggered (to undo the previous ones). Two styles:
  • Choreography: services react to each other's events (no center). Flexible, but hard to trace the overall flow.
  • Orchestration: a central orchestrator manages the steps and compensations. Clearer, but a coordinator appears.
• Eventual consistency: data temporarily diverges but converges over time. Sufficient for most business cases (counters, feeds, recommendations).
• Outbox pattern: the dual-write problem — you can't atomically write to the DB and publish an event. Solution: in one local transaction, write both the data and a record into an outbox table; a separate process (poller / CDC, e.g. Debezium) reads the outbox and publishes events to the broker. Guarantees "written → the event will be published" (at-least-once).

📎 CAP/DB consistency — see the nosql file.

⚠️ Gotcha: dual write (writing to the DB and immediately sending to Kafka as two operations) is a classic bug: if we crash between them, the data and events diverge. Cured by Outbox or transactional log tailing (CDC).

Short answer: During a network partition (Partition), a distributed system can preserve either Consistency or Availability, but not both. P is inevitable in a network, so the real choice is CP or AP.

In detail:

• C — all nodes see the same data (linearizability). A — every request gets a response. P — the system works when connectivity between nodes is lost.
• Since network failures happen, P is mandatory → you choose: during a partition either answer with possibly stale data (AP) or refuse for the sake of consistency (CP).
• An addition — PACELC: during a Partition you choose A/C, otherwise (Else) — a trade-off between Latency and Consistency.

📎 In detail with DB examples (Mongo, Cassandra, etc.) — see the nosql file.

⚠️ Gotcha: CAP isn't about "pick 2 of 3 forever." You can't "choose not to have" P — it's a property of the network. The choice is made only at the moment of a partition.

Short answer: An idempotent operation, when repeated, produces the same result as a single execution. Deduplication — discarding repeated messages/requests by a unique key.

In detail:

• Why: in a distributed system retries are inevitable (timeouts, queue re-sends, a user repeating). Without idempotency, a repeat = a duplicated effect (a double debit).
• Idempotency in HTTP: GET/PUT/DELETE are idempotent by semantics, POST is not. So for POST you use an Idempotency-Key (the client sends a unique key, the server remembers the result of the first call and returns it on repeats).
• Ways to achieve it:
  • UPSERT / conditional updates by a business key;
  • a table of processed IDs (processed_messages) — checked before processing;
  • natural idempotency: "set status = X" instead of "increment."
• Deduplication: at the broker level (SQS FIFO dedup window, Kafka producer idempotence) or the application level (by message_id in a store with TTL).

⚠️ Gotcha: an increment (balance += 100) isn't idempotent — a repeat doubles the effect. "Set a value" or "perform a transaction with a unique id" are idempotent. Design operations to be idempotent from the very start.

Short answer: Limiting the request rate to protect against overload/abuse. The main algorithms: token bucket, leaky bucket, fixed window, sliding window.

In detail — algorithms:

• Token bucket: a bucket of N tokens, refilled at a rate of r/sec; a request spends a token. Allows bursts up to the bucket size while holding the average rate. The most popular (flexible).
• Leaky bucket: requests are water, "leaking out" at a constant rate; overflow → rejection. Smooths traffic into an even flow, doesn't let bursts through.
• Fixed window: a counter over a fixed window (e.g., 100/min). Simple, but has the boundary problem — a double burst at the window seam (200 requests in a couple of seconds around the boundary).
• Sliding window log / counter: a sliding window (an exact log of timestamps or a weighted counter) — more accurate at the boundaries, slightly more expensive in memory/compute.

Where to apply: the API gateway, per user/IP/API key, on expensive endpoints, protecting downstream services and the DB. Often implemented in Redis (atomic counters/Lua). The response on exceeding the limit — HTTP 429 + Retry-After.

⚠️ Gotcha: fixed window allows a burst at the window boundary (up to 2× the limit). For strict limits use sliding window / token bucket. And remember distribution: the counter must be shared (Redis), not local on each instance.

Short answer: Sharding — horizontal partitioning of data across several DBs (scaling writes/volume). Replication — copies of data on other nodes (scaling reads and fault tolerance). Read/write splitting and CQRS complement this.

In detail:

• Sharding: data is sliced by a shard key into independent DBs:
  • by key/range (range): simple range lookups, but a risk of a hot shard;
  • by key hash (hash): even distribution, but range queries are expensive;
  • by directory (directory): flexible, but the lookup table is a point of failure.
  • Hot partition / hot shard: a bad key → one shard gets a disproportionate amount of traffic (e.g., sharding by country where 80% of users are). Choose a key with high cardinality and even distribution.
  • Rebalancing: when adding a shard, the data has to be redistributed. A naive hash % N breaks when N changes → use consistent hashing or virtual buckets to move a minimum of data.
• Replication: primary (writes) + read replicas (reads). Gives read scaling, backup, fault tolerance. Replicas lag (replication lag) → eventual consistency on reads.
• Read/write splitting: write to the primary, read from the replicas. Hit by lag — after a write, a read from a replica may return stale data (read-your-writes is solved by a "sticky read from the primary" for some time).
• CQRS (briefly): Command Query Responsibility Segregation — separating the write model (commands) and the read model (queries), often with separate stores optimized for their own patterns. Adds complexity, justified when read/write load differs greatly.

⚠️ Gotcha: sharding is an expensive and almost irreversible operation; cross-shard JOINs/transactions/aggregations are painful. First squeeze out vertical scaling, indexes, cache, and read replicas. Shard only when you've truly hit a wall on writes/volume, and choose the shard key carefully.

Short answer: A SPOF — a component whose failure brings down the whole system. Cured by redundancy (replication, multiple instances, multi-AZ) and failover — automatic switchover to a backup.

In detail:

• Finding SPOFs: a single server, one DB without replicas, one balancer, one availability zone, a single broker, a shared cache without replication.
• Eliminating them:
  • Replication: copies of data/services (DB primary+replicas, multiple application instances).
  • Redundancy: N+1 instances, a balancer distributes; the balancer itself is also duplicated.
  • Failover: on a primary failure a replica is automatically promoted (with a health-check and quorum to avoid split-brain).
  • Multi-AZ / multi-region for geographic resilience.
• Metrics: RTO (how long to recover) and RPO (how much data we can lose).

⚠️ Gotcha: "we have two replicas" is useless without auto-failover and regular testing of the switchover. An untested failover often doesn't kick in during a real incident. And beware of split-brain — you need a quorum/arbiter.

Short answer: Patterns for protecting against cascading failures. Timeout — don't wait forever. Retry with backoff+jitter — retry smartly. Circuit breaker — stop hammering a dead service. Bulkhead — isolate resources so a failure of one part doesn't sink everything.

In detail:

• Timeouts: every external call must have a timeout. Without one, a hung downstream holds the caller's threads/connections → pool exhaustion → crash. Timeouts should decrease down the chain.
• Retries + exponential backoff + jitter:
  • retry only idempotent/safe operations and only on retriable errors (5xx, timeout), not on 4xx;
  • exponential backoff: intervals grow (1s, 2s, 4s…) so as not to finish off the service;
  • jitter: a random addition to the interval so clients don't retry in sync (thundering herd / retry storm);
  • limit the number of attempts.
• Circuit breaker: counts errors; when a threshold is exceeded it "opens" (Open) — instantly fails requests without loading the dead service; after a pause it goes to Half-Open (a probe request) → Closed on success. Gives the downstream a chance to recover and quickly returns a fallback.
• Bulkhead: isolation of resources (separate thread/connection pools per dependency), like bulkheads in a ship. A failure/slowdown of one dependency doesn't exhaust the shared pool and doesn't sink the other functions.

⚠️ Gotcha: a retry without backoff+jitter and without a circuit breaker turns a small failure into a retry storm that finishes off an already wobbling service (a cascading failure). Retries without idempotency → duplicates. Combine: timeout + a limited retry with jitter + a circuit breaker.

Short answer: Graceful degradation — when part of the system fails, keep working with reduced functionality rather than crashing entirely. Backpressure — a mechanism by which an overloaded consumer signals the source to slow down.

In detail:

• Graceful degradation: if recommendations are unavailable — show a default list; if the cache is down — go to the DB more slowly; if the payment provider is down — accept the order in a pending status. Fallback values, feature flags, disabling secondary features under load (load shedding).
• Backpressure: the consumer can't keep up → it should slow down/stop the producer rather than buffer indefinitely (which leads to OOM). Mechanisms: bounded queues and buffers, blocking/rejecting when full, reactive streams (request(n)), HTTP 429/503, TCP flow control. In queues — a natural buffer + limits + DLQ.

⚠️ Gotcha: unbounded queues/buffers are an illusion of fault tolerance: instead of rejecting, the service accumulates memory and crashes with OOM (or inflates latency). Better an explicit rejection/degradation (fail fast / shed load) than silent accumulation.

Short answer: A set of 12 practices for building cloud-native applications: easy to deploy, scale horizontally, and operate.

In detail — the key factors (for an interview):

1. Codebase — one codebase in VCS, many deploys.
2. Dependencies — explicitly declared and isolated.
3. Config — in environment variables, not in the code.
4. Backing services — DB/cache/broker as attached resources (by URL/config).
5. Build, release, run — strictly separated stages.
6. Processes — the application as stateless processes (state — in external services). ← the basis of scaling.
7. Port binding — the service exports HTTP itself, self-contained.
8. Concurrency — scale via processes (horizontally).
9. Disposability — fast startup and graceful shutdown (important for rolling deploys/autoscaling).
10. Dev/prod parity — environments as similar as possible.
11. Logs — as a stream of events to stdout, aggregated externally.
12. Admin processes — one-off tasks (migrations) as one-off processes.

⚠️ Gotcha: the most frequently violated — config in the code (factor 3) and state in the process (factor 6). Those are exactly what hinder horizontal scaling and containerization.

Short answer: The three pillars of observability — logs (what happened), metrics (aggregated numbers over time), tracing (a request's path through services). A correlation/trace ID ties them together.

In detail:

• Logs: discrete events, preferably structured (JSON), aggregated (ELK/Loki). For the details of a specific event/error.
• Metrics: numeric series (RPS, latency p50/p95/p99, error rate, resource saturation). Cheap, aggregatable, for dashboards and alerts (Prometheus/Grafana). Methods: RED (Rate, Errors, Duration) and USE (Utilization, Saturation, Errors).
• Distributed tracing: one request passes through many services; a trace = a tree of spans with timings. You can see where the bottleneck is in the chain. OpenTelemetry, Jaeger, Zipkin.
• Correlation ID / trace ID: a unique identifier assigned to the request at the entry point (API gateway) and propagated through all services/logs/events. It lets you assemble the full picture of a single request in a distributed system.

💡 Logs and metrics tell you what broke; tracing — where in the chain.

⚠️ Gotcha: without a correlation id, the logs of a distributed system are useless — you can't tie together the events of a single request across services. Propagate the trace id from the very start, including async messages in queues.

Short answer: Strategies for safe rollouts. Blue-green — two full environments with an instant traffic switch. Canary — the new version first receives a small share of traffic that grows gradually.

In depth:

• Blue-green: blue (current) and green (new) run in parallel; once green is verified, the load balancer switches 100% of traffic to it. Instant rollback (switch back), zero downtime. Downside — double the resources and complications with DB migrations.
• Canary: the new version is rolled out to 1–5% of traffic, metrics are watched (errors, latency); on success it's gradually raised to 100%, otherwise rolled back. Minimizes the blast radius. Requires good monitoring and traffic routing by percentage.
• Rolling update: instances are updated one at a time (the default in k8s) — no double resources, but rollback is slower and versions temporarily coexist.

⚠️ Gotcha: with any strategy, DB migrations must be backward compatible — the old and new code versions work against the same schema simultaneously for a while. Use expand-and-contract (first add, then drop the column in a separate release).

Short answer: A direct call is synchronous and couples services in time; a queue gives you asynchrony, decoupling, peak buffering, and reliability.

In depth: with a direct call the caller waits, both must be alive at the same time, a traffic spike hits the downstream directly, and if the downstream is down — the operation is lost or fails. A queue: the response is instant, the downstream can be temporarily unavailable (the message will wait), peaks are smoothed by the buffer, delivery is reliable (ack/retry/DLQ), consumers scale independently. The price — eventual consistency and a requirement for idempotency. A queue is needed when the result doesn't have to come back immediately and resilience matters; a direct call — when you need a response here and now.

Short answer: Because interchangeable instances can be freely added, removed, and restarted, and any request can be handled on any node.

In depth: if an instance keeps state (session, cache, files) inside itself, you have to "pin" the client (sticky sessions), and a node failure loses data and breaks load balancing. Stateless moves state out into shared stores (Redis, S3, DB) → the load balancer can throw a request anywhere, rolling deploys and autoscaling happen without losses, and horizontal scaling becomes a trivial "add nodes." This is directly reflected in 12-factor (factor 6: processes).

Short answer: When there's no mature infrastructure/team, when the domain isn't understood, or when the benefits don't outweigh the distributed complexity — especially at a product's start.

In depth: microservices are a mistake if: a small team (the overhead eats the benefit); no CI/CD, monitoring, orchestration (operational hell); domain boundaries aren't clear yet (you'll have to constantly re-carve services — expensive over the network); the system requires cross-service transactions everywhere; the result is a distributed monolith (coupled services sharing a DB and deployed together — the downsides of both worlds). The right path: start with a modular monolith, extract services only when a real reason appears (different load/release cadence/teams).

Short answer: Move layer by layer: measure the bottleneck → make it stateless → scale horizontally behind a load balancer → cache → push heavy work to queues asynchronously → scale the data (replicas, then sharding) → add fault tolerance and observability.

In depth — structure of the answer:

1. Measure first: find the bottleneck (CPU, DB, IO, external API) via metrics — don't optimize blindly.
2. Stateless: move sessions/state/files outside (Redis/S3) so you can scale horizontally.
3. Horizontal + LB: N instances behind an L7 load balancer with health checks.
4. Caching: browser/CDN/reverse proxy/Redis — take read load off.
5. Asynchrony: heavy operations — into a queue to workers (Celery/broker), smooth out peaks.
6. Data: read replicas + read/write splitting; indexes; when hitting write/volume limits — sharding with a well-thought-out key.
7. Resilience: remove SPOFs, timeouts + retries (backoff+jitter) + circuit breaker + bulkhead, graceful degradation, autoscaling.
8. Observability: metrics/logs/tracing with a correlation id, so you can see the effect and new bottlenecks.

💡 Key idea: scaling is an iterative, metrics-driven process, not a one-time "cram in Kafka and Kubernetes."