System Design Cheat Sheet: A Complete Reference for Interviews and Production

System design is about making the right tradeoffs at scale. This cheat sheet covers every major concept you need — from load balancing to database sharding — with interview frameworks and real-world patterns.

Quick reference

Interview framework: how to approach any system design question

Use this step-by-step structure for every system design interview:

1. Clarify requirements (5 min)
   - Functional: what does the system DO?
   - Non-functional: scale, latency, availability, consistency

2. Estimate scale (2 min)
   - Users: DAU, requests/sec
   - Data: storage per day/year, bandwidth

3. High-level design (10 min)
   - Draw boxes: clients, servers, databases, caches, queues
   - Identify the core data flow

4. Deep dive (15 min)
   - Pick the hardest part and go deep
   - Databases, APIs, algorithms

5. Identify bottlenecks (5 min)
   - Single points of failure, hot spots, scaling limits
   - Propose solutions

Back-of-envelope estimation cheat sheet

Scalability

Horizontal vs vertical scaling

Vertical (scale up)          Horizontal (scale out)
────────────────────         ──────────────────────
Bigger machine               More machines
Simpler to implement         Requires coordination
Limited ceiling              Theoretically unlimited
Single point of failure      Fault tolerant
Expensive at high end        Commodity hardware

Stateless vs stateful services

Stateless — easier to scale:

• Any server can handle any request
• Session data lives in shared store (Redis, DB)
• Works with load balancers out of the box

Stateful — harder to scale:

• Client must reconnect to the same server (sticky sessions)
• Required for WebSockets, some game servers
• Use consistent hashing to route to correct node

Load balancing

Load balancing algorithms

Layer 4 vs Layer 7 load balancing

Layer 4 (Transport)          Layer 7 (Application)
────────────────────         ──────────────────────
Routes by IP + port          Routes by URL, headers, cookies
Faster (no parsing)          Slower (inspects content)
Can't route by path          Can do /api → servers A, /web → servers B
Lower cost                   Content-based routing, SSL termination

Health checks

# Nginx upstream with health check
upstream backend {
    least_conn;
    server 10.0.0.1:8080 weight=3;
    server 10.0.0.2:8080 weight=1;
    server 10.0.0.3:8080 backup;
}

Caching

Cache strategies

Cache-aside (lazy loading) — most common:

1. App checks cache → cache miss
2. App reads from DB
3. App writes result to cache
4. Next request → cache hit

Pros: Only caches what's needed
Cons: Cache miss on first request, stale data possible

Write-through — keeps cache in sync:

1. App writes to cache AND DB together

Pros: Data always fresh in cache
Cons: Write penalty, caches data never read

Write-behind (write-back) — fast writes:

1. App writes to cache only
2. Cache asynchronously flushes to DB

Pros: Very fast writes
Cons: Data loss risk if cache crashes before flush

Read-through — cache fetches from DB:

1. App only talks to cache
2. On miss: cache fetches from DB automatically

Pros: Simpler app code
Cons: Extra hop, cache tied to DB schema

Cache eviction policies

What to cache

✅ Good candidates:
- Session data
- User profiles (read-heavy)
- Computed results (expensive queries)
- HTML fragments
- API responses from slow third parties
- Database query results

❌ Bad candidates:
- Frequently updated data (payment info, inventory)
- User-specific secure data (without careful key design)
- Very large objects (wastes cache memory)

Redis key design

# Pattern: resource:id:field
user:42:profile
session:abc123
rate:user:42:2024-01-15

# SET with TTL
SET user:42:profile '{"name":"Alice"}' EX 3600

# Atomic increment for rate limiting
INCR rate:user:42:2024-01-15:13
EXPIRE rate:user:42:2024-01-15:13 60

Databases

SQL vs NoSQL

Database replication

Primary-replica (master-slave):
┌─────────┐   sync/async    ┌─────────┐
│ Primary │ ──────────────▶ │ Replica │ (reads)
│ (writes)│                 └─────────┘
└─────────┘ ──────────────▶ ┌─────────┐
                             │ Replica │ (reads)
                             └─────────┘

Sync replication: consistent, higher write latency
Async replication: faster writes, possible data loss on crash

Database sharding

Sharding strategies:

Range-based:   user_id 1-1M → shard 1, 1M-2M → shard 2
               Pros: simple. Cons: hot spots if range is popular.

Hash-based:    shard = hash(user_id) % num_shards
               Pros: even distribution. Cons: resharding is painful.

Directory:     lookup table maps key → shard
               Pros: flexible. Cons: extra lookup, single point of failure.

Geo-based:     EU users → EU shard, US users → US shard
               Pros: low latency. Cons: complex queries across shards.

Database indexing

-- B-tree index: range queries, equality
CREATE INDEX idx_users_email ON users(email);

-- Composite index (order matters — leftmost prefix rule)
CREATE INDEX idx_posts_user_date ON posts(user_id, created_at);
-- Uses: WHERE user_id = 1 ✅
-- Uses: WHERE user_id = 1 AND created_at > '2024-01-01' ✅
-- Skips: WHERE created_at > '2024-01-01' ❌ (no user_id prefix)

-- Covering index (all needed columns in index)
CREATE INDEX idx_covering ON posts(user_id, created_at, title);

-- Partial index (index a subset of rows)
CREATE INDEX idx_active ON users(email) WHERE active = true;

CAP theorem

Every distributed system can only guarantee 2 of 3:

        Consistency
           /\
          /  \
         /    \
        /  CP  \
       /        \
      /----AP----\
     /            \
Availability ── Partition
               Tolerance

C — Every read sees the most recent write
A — Every request gets a response (not guaranteed to be latest)
P — System works despite network partitions

Real-world picks:

PACELC extension:

• Even without partitions: choose Latency vs Consistency
• DynamoDB: PA/EL (Available during partition, Low latency else)
• BigTable: PC/EC (Consistent in both cases)

Message queues

When to use a message queue

✅ Use queues for:
- Decoupling services (email sending, notification delivery)
- Smoothing traffic spikes (order processing)
- Background jobs (image resizing, report generation)
- Broadcast events to multiple consumers (fan-out)
- Guaranteed delivery with retry

❌ Don't use for:
- Synchronous request/response (user waiting for result)
- Simple one-off background tasks (cron is simpler)

Queue comparison

Producer-consumer pattern

# Producer (Python + Redis)
import redis, json

r = redis.Redis()

def enqueue_email(to, subject, body):
    job = {"to": to, "subject": subject, "body": body}
    r.rpush("email_queue", json.dumps(job))

# Consumer
def process_emails():
    while True:
        _, data = r.blpop("email_queue")  # blocking pop
        job = json.loads(data)
        send_email(job["to"], job["subject"], job["body"])

API design

REST design principles

Resource-based URLs:
GET    /users          → list users
POST   /users          → create user
GET    /users/42       → get user 42
PUT    /users/42       → replace user 42
PATCH  /users/42       → partial update
DELETE /users/42       → delete user 42

Nested resources:
GET    /users/42/posts      → posts by user 42
POST   /users/42/posts      → create post for user 42

Filtering, sorting, pagination:
GET /posts?status=published&sort=-created_at&page=2&limit=20

Pagination strategies

-- Cursor (keyset) pagination — fast even at millions of rows
SELECT * FROM posts
WHERE (created_at, id) < (:last_ts, :last_id)
ORDER BY created_at DESC, id DESC
LIMIT 20;

Rate limiting algorithms

# Token bucket in Redis (Lua script for atomicity)
local key = KEYS[1]
local rate = tonumber(ARGV[1])       -- tokens per second
local capacity = tonumber(ARGV[2])   -- max burst
local now = tonumber(ARGV[3])        -- current timestamp ms

local last = tonumber(redis.call('hget', key, 'ts') or now)
local tokens = tonumber(redis.call('hget', key, 'tokens') or capacity)

-- Refill
tokens = math.min(capacity, tokens + rate * (now - last) / 1000)

if tokens >= 1 then
    tokens = tokens - 1
    redis.call('hset', key, 'ts', now, 'tokens', tokens)
    return 1  -- allowed
else
    return 0  -- rate limited
end

Reliability patterns

Circuit breaker

States:
  CLOSED → requests flow normally
  OPEN   → requests fail fast (no calls to downstream)
  HALF-OPEN → let one request through to test recovery

Transition:
  CLOSED  → too many failures → OPEN
  OPEN    → timeout expires → HALF-OPEN
  HALF-OPEN → success → CLOSED | failure → OPEN

Retry with exponential backoff

import time, random

def call_with_retry(fn, max_retries=3, base_delay=0.5, max_delay=10):
    for attempt in range(max_retries + 1):
        try:
            return fn()
        except TransientError as e:
            if attempt == max_retries:
                raise
            # Exponential backoff with jitter
            delay = min(base_delay * (2 ** attempt) + random.random(), max_delay)
            time.sleep(delay)

Idempotency

Make operations safe to retry:

# Idempotency key in header (client generates UUID per operation)
POST /orders
Idempotency-Key: 7d8f3e2a-1b4c-4d5e-9f0a-2b3c4d5e6f7a

# Server logic
def create_order(idempotency_key, data):
    existing = cache.get(f"idempotency:{idempotency_key}")
    if existing:
        return existing  # return cached response, don't process again
    
    result = process_order(data)
    cache.set(f"idempotency:{idempotency_key}", result, ex=86400)
    return result

Content delivery networks (CDNs)

What CDNs cache

✅ Cache with CDN:
- Images, videos, CSS, JS bundles
- HTML pages (for SSG sites)
- API responses (with Cache-Control headers)
- Fonts

❌ Don't cache:
- Personalised content (without Vary header)
- Real-time API endpoints
- Payment/checkout pages

Cache-Control headers

# Cache everywhere for 1 hour
Cache-Control: public, max-age=3600

# CDN caches, but browser doesn't
Cache-Control: public, s-maxage=86400, max-age=0

# Never cache (auth, real-time)
Cache-Control: no-store

# Revalidate before serving stale
Cache-Control: no-cache

# Stale-while-revalidate (serve stale, refresh in background)
Cache-Control: max-age=60, stale-while-revalidate=600

Consistent hashing

Used for distributing data across nodes with minimal reshuffling when nodes are added/removed:

Traditional hashing:    key → hash(key) % N
                        Adding a node changes almost all mappings!

Consistent hashing:     key → position on a ring
                        key maps to the next node clockwise on ring
                        Adding a node only affects its neighbours

With virtual nodes (vnodes):
- Each physical node gets multiple positions on the ring
- Better distribution, handles heterogeneous node sizes
- Used by: Cassandra, DynamoDB, Riak

Common system design problems

URL shortener (like bit.ly)

Core requirements:
- shorten(long_url) → short_code
- redirect(short_code) → long_url

Design decisions:
1. ID generation: auto-increment DB ID → base62 encode (a-z, A-Z, 0-9)
   1000000 → "4c92" (7 chars covers 62^7 ≈ 3.5 trillion URLs)

2. Storage: SQL (MySQL) — one row per URL
   (id BIGINT, long_url TEXT, created_at, user_id, clicks)

3. Redirect: 301 (permanent, browser caches) vs 302 (temporary, tracks clicks)
   Use 302 if you want accurate click analytics.

4. Cache: Redis — cache hot short codes (top 20% generate 80% of traffic)
   Key: short_code, Value: long_url, TTL: 24h

5. Scale: sharding by short_code prefix, read replicas for lookups

Rate limiter

Requirements: allow N requests per user per minute

Design:
1. Storage: Redis per user key with sliding window
2. Key: rate:{user_id}:{window}
3. Algorithm: token bucket or sliding window counter
4. Middleware: API gateway checks limit before routing
5. Response: 429 Too Many Requests + Retry-After header

Feed/timeline (like Twitter)

Fanout-on-write (push):
- When user posts, push to all followers' feeds
- Pros: fast read (pre-computed)
- Cons: slow write for users with millions of followers (celebrities)

Fanout-on-read (pull):
- When user opens feed, fetch from all followees
- Pros: fast write
- Cons: slow read (N queries)

Hybrid (used by Twitter):
- Regular users: fanout-on-write
- Celebrity accounts: fanout-on-read
- Merge at read time

Microservices vs monolith

Service communication

Synchronous (request/response):
- REST over HTTP — simple, widely supported
- gRPC — binary protocol, great for internal services (strongly typed, faster)

Asynchronous (events):
- Message queue (RabbitMQ, SQS) — point-to-point
- Event streaming (Kafka) — pub/sub with replay

Common mistakes

FAQ

What's the difference between latency and throughput? Latency is the time for one request to complete (ms). Throughput is how many requests the system handles per second (RPS). A fast server (low latency) can still have low throughput if it can only handle one request at a time. You typically improve throughput by parallelism.

When should I use a cache vs a database? Cache for frequently read, rarely changing data where stale results are tolerable. Use the database as the source of truth. Cache is a performance optimisation, not a storage solution.

How do I choose between SQL and NoSQL? Default to SQL (PostgreSQL). Choose NoSQL when: you need flexible/dynamic schema, horizontal write scaling at massive scale (Cassandra), document-oriented data (MongoDB), or your access pattern is purely key-value (DynamoDB/Redis).

What is eventual consistency? Eventual consistency means all replicas will converge to the same value eventually, but at any moment they may return different results. Acceptable for social likes, view counts, shopping cart (with conflict resolution). Not acceptable for bank balances, inventory that must not oversell.

How do I handle database migrations at scale? Zero-downtime migrations: 1) Add new column as nullable. 2) Deploy code that writes to both old and new columns. 3) Backfill old rows. 4) Deploy code that reads from new column. 5) Drop old column. Never rename a column in one step.

What does "sharding" solve that replication doesn't? Replication improves read throughput and availability. Sharding improves write throughput and allows datasets larger than one machine. Use replication first; add sharding only when writes become the bottleneck.