Microservices interviews test your understanding of distributed system design, inter-service communication, data consistency, resilience patterns, observability, and deployment strategies. This guide covers the 50 most common questions — with clear answers and real examples.
Quick reference
| Topic | Most asked questions |
|---|---|
| Architecture | Monolith vs microservices, bounded context, decomposition |
| Communication | REST vs gRPC vs messaging, sync vs async |
| Data management | Database per service, CQRS, event sourcing, saga |
| Resilience | Circuit breaker, retry, bulkhead, timeout |
| Security | JWT, OAuth 2.0, mTLS, API gateway auth |
| Observability | Distributed tracing, structured logging, metrics |
| Deployment | Docker, Kubernetes, CI/CD, blue-green, canary |
| Testing | Contract testing, consumer-driven contracts, chaos |
Core architecture
1. What are microservices and what problem do they solve?
Microservices is an architectural style where an application is composed of small, independently deployable services that each own a specific business capability and communicate over networks.
Problems they solve vs monolith:
- Independent deployments — change one service without redeploying everything
- Independent scaling — scale the checkout service more than the catalog service
- Technology diversity — choose the right tool per service
- Team autonomy — small teams own end-to-end services
- Fault isolation — a crash in recommendations doesn't take down checkout
Problems they introduce:
- Network latency and failures
- Distributed data management complexity
- Operational overhead (many services to monitor/deploy)
- Testing across service boundaries
2. Monolith vs microservices — when to choose which?
| Factor | Monolith | Microservices |
|---|---|---|
| Team size | Small (<10) | Multiple teams |
| Domain complexity | Simple | Complex, distinct subdomains |
| Deployment frequency | Low | High, per service |
| Scaling needs | Uniform | Different per component |
| Operational maturity | Low | High (K8s, observability) |
| Development speed (start) | Fast | Slow (infrastructure setup) |
| Development speed (scale) | Slow (big codebase) | Fast (small codebases) |
| Data consistency | Easy (single DB) | Hard (distributed) |
Rule of thumb: Start with a monolith. Migrate to microservices only when team/scale pain becomes real. Premature microservices is one of the most common architectural mistakes.
3. What is a Bounded Context?
A bounded context (from Domain-Driven Design) is an explicit boundary within which a particular domain model is defined and applicable. Different bounded contexts can have different models of the same concept.
Example: "User" means different things in:
- Auth service — credentials, sessions, roles
- Billing service — payment methods, invoices, subscription
- Shipping service — delivery addresses, preferences
Each service owns its model of "user" independently. No shared User entity across services.
4. How do you decompose a monolith into microservices?
Decomposition strategies:
| Strategy | Approach | Best for |
|---|---|---|
| By business capability | One service per business function (Orders, Inventory, Payments) | Most common |
| By subdomain (DDD) | Align with bounded contexts | Complex domains |
| By team/ownership | Conway's Law — services mirror team structure | Large orgs |
| By volatility | Extract frequently changing parts first | Legacy migration |
| Strangler fig | Route traffic gradually from monolith to new services | Safe migration |
Strangler fig pattern:
┌─────────────┐
Browser ──────────► │ API Gateway │
└──────┬──────┘
│ route by path
┌────────────┼────────────┐
▼ ▼ ▼
Old Monolith New Orders New Catalog
(shrinking) Service Service
5. What is the Single Responsibility Principle for services?
Each service should have one reason to change — own exactly one business capability with clear boundaries. A service that handles "user registration AND email sending AND billing" violates this.
Signs a service has grown too large:
- Multiple teams need to coordinate deployments
- A single service has hundreds of endpoints
- The codebase takes hours to understand
- Unrelated things break together
Communication patterns
6. Synchronous vs asynchronous communication — when to use each?
| Aspect | Synchronous (REST/gRPC) | Asynchronous (Kafka/RabbitMQ) |
|---|---|---|
| Coupling | Temporal coupling (caller waits) | Decoupled (producer/consumer independent) |
| Latency | Lower for single request | Higher (message propagation) |
| Availability | Caller depends on callee uptime | Resilient to callee downtime |
| Complexity | Simpler | Higher (message brokers, idempotency) |
| Use case | Query for immediate data, user-facing APIs | Background work, event notifications, fan-out |
Use sync for: "Is this product in stock?" — user is waiting.
Use async for: "Order placed" → send email + update inventory + notify warehouse.
7. REST vs gRPC vs GraphQL for inter-service communication?
| Feature | REST | gRPC | GraphQL |
|---|---|---|---|
| Protocol | HTTP/1.1 | HTTP/2 | HTTP |
| Format | JSON (verbose) | Protobuf (binary, compact) | JSON |
| Performance | Moderate | High (binary + multiplexing) | Moderate |
| Schema | OpenAPI (optional) | .proto (required) |
Schema (required) |
| Streaming | Limited (SSE) | Bidirectional streaming | Subscriptions |
| Browser support | Native | Needs proxy (grpc-web) | Native |
| Best for | External APIs, browser clients | Internal service-to-service | Client-driven APIs (BFF) |
gRPC example (.proto):
service OrderService {
rpc GetOrder (GetOrderRequest) returns (Order);
rpc CreateOrder (CreateOrderRequest) returns (Order);
rpc StreamOrders (Empty) returns (stream Order);
}
message Order {
string id = 1;
string user_id = 2;
double total = 3;
OrderStatus status = 4;
}
8. What is an API Gateway and what does it do?
An API Gateway is the single entry point for all client requests. It routes to the appropriate service and handles cross-cutting concerns.
Responsibilities:
| Function | Detail |
|---|---|
| Request routing | Route /api/orders → Order Service |
| Authentication/Authorization | Validate JWT before forwarding |
| Rate limiting | 100 req/min per API key |
| SSL termination | HTTPS externally, HTTP internally |
| Load balancing | Distribute across service instances |
| Request aggregation (BFF) | Combine multiple service responses |
| Protocol translation | HTTP externally, gRPC internally |
| Caching | Cache GET responses |
| Observability | Centralized logging, tracing injection |
Examples: Kong, AWS API Gateway, NGINX, Traefik, Envoy.
9. What is the Backend for Frontend (BFF) pattern?
A BFF is a dedicated API gateway per client type, optimized for that client's needs.
Mobile App ──► Mobile BFF ──► [Product Service]
──► [Inventory Service]
──► [Review Service]
Web App ────► Web BFF ──────► Same services, different aggregation
Why: Mobile needs fewer fields, smaller payloads. Web may need richer data with server-side aggregation. A single generic API forces each client to over-fetch or make multiple calls.
10. How do services discover each other?
| Mechanism | How | Example |
|---|---|---|
| Client-side discovery | Client queries registry, picks instance | Eureka + Ribbon |
| Server-side discovery | Client calls load balancer, which queries registry | AWS ALB + ECS |
| DNS-based | Service name resolves to IP(s) | Kubernetes Service DNS |
| Service mesh | Sidecar proxies handle discovery transparently | Istio/Envoy |
Kubernetes example: Services communicate via DNS names:
http://order-service.default.svc.cluster.local:8080/orders
# or within same namespace:
http://order-service:8080/orders
Data management
11. Database per service pattern — why and how?
Each service owns its data store exclusively. No other service queries it directly.
Why:
- Independent deployment — change the order DB schema without touching payment service
- Technology choice — use PostgreSQL for orders, Redis for sessions, MongoDB for product catalog
- Failure isolation — Payment DB going down doesn't affect Catalog service
How to share data:
- Services expose APIs (not DB queries)
- Eventual consistency via events
- Read replicas/materialized views for reporting
Order Service ──owns──► orders_db (PostgreSQL)
X (no direct access)
Payment Service ─owns──► payments_db (PostgreSQL)
Product Service ─owns──► products_db (MongoDB)
12. What is the Saga pattern and when do you need it?
A saga is a sequence of local transactions where each step publishes events that trigger the next step. If a step fails, compensating transactions undo previous steps.
Why: Distributed transactions (2PC) are impractical across microservices — they require all services to be available and create tight coupling.
Choreography saga (event-driven):
Order Service ──publishes──► OrderCreated
│
Payment Service ◄─subscribes─┘
Payment Service ──publishes──► PaymentCompleted
│
Inventory Service ◄─subscribes┘
Inventory Service ──publishes──► InventoryReserved
│
Shipping Service ◄─subscribes─┘
Orchestration saga (central coordinator):
Order Saga Orchestrator
│
┌─────────┼─────────┐
▼ ▼ ▼
Payment Inventory Shipping
Service Service Service
Compensating transaction example:
- Step 1: Create order ✓
- Step 2: Charge payment ✓
- Step 3: Reserve inventory ✗ (out of stock)
- Compensation: Refund payment → Cancel order
13. What is CQRS (Command Query Responsibility Segregation)?
CQRS separates write operations (Commands) from read operations (Queries) into different models.
Client
│
├── Write path ──► Command Handler ──► Write DB (normalized)
│ │
│ └──publishes events──► Read Model Builder
│
└── Read path ──► Query Handler ──► Read DB (denormalized, fast)
Why: Read and write workloads have different performance characteristics. Writes need consistency/validation; reads need speed and specific projections.
With Event Sourcing: The write side stores events (not state). Read models are built by replaying events.
14. What is the Outbox Pattern?
Ensures messages are published to a message broker atomically with a database write — prevents "dual write" problems.
Problem without outbox:
1. INSERT order INTO orders_db ✓
2. PUBLISH OrderCreated to Kafka ✗ (network failure)
→ Order created but event never published → inconsistent state
With outbox:
BEGIN;
INSERT INTO orders (id, ...) VALUES (...);
INSERT INTO outbox (event_type, payload) VALUES ('OrderCreated', '...');
COMMIT;
-- Separate process (CDC or poller):
SELECT * FROM outbox WHERE published = false;
PUBLISH to Kafka;
UPDATE outbox SET published = true;
Tools: Debezium (Change Data Capture), Transactional Outbox library.
15. Eventual consistency vs strong consistency — what's the trade-off?
| Aspect | Strong Consistency | Eventual Consistency |
|---|---|---|
| Guarantee | All reads see latest write | Reads may see stale data (temporarily) |
| Availability | Lower (requires coordination) | Higher |
| Latency | Higher | Lower |
| Feasibility across services | Very hard (2PC) | Natural with events |
| Use case | Bank balance, inventory count | Social feed, product ratings, analytics |
Microservices default: Eventual consistency. Design your UX to handle it (show "processing…", use optimistic UI).
Resilience patterns
16. What is the Circuit Breaker pattern?
Prevents cascading failures by stopping requests to a failing service. Like an electrical circuit breaker — opens to stop the flow when current is too high.
States:
failure threshold exceeded
CLOSED ────────────────────────────► OPEN
▲ │
│ success │ timeout
│ ▼
└─────────────────────────────── HALF-OPEN
probe request succeeds
| State | Behavior |
|---|---|
| Closed | Requests flow normally; failures counted |
| Open | Requests fail fast (no call to service); returns fallback |
| Half-Open | Allow one probe request; if success → Closed, if fail → Open |
Example with Resilience4j (Java):
CircuitBreaker cb = CircuitBreaker.ofDefaults("paymentService");
Supplier<Payment> decorated = CircuitBreaker.decorateSupplier(cb, () ->
paymentClient.charge(request)
);
Try.ofSupplier(decorated)
.recover(CallNotPermittedException.class, ex -> cachedPayment);
17. Retry, timeout, bulkhead — explain each.
Retry: Re-attempt failed requests with exponential backoff + jitter.
async function withRetry(fn, maxAttempts = 3) {
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
return await fn();
} catch (err) {
if (attempt === maxAttempts) throw err;
const delay = Math.min(1000 * 2 ** attempt + Math.random() * 1000, 30000);
await sleep(delay);
}
}
}
Timeout: Never wait indefinitely for a response.
const response = await Promise.race([
fetch('/api/orders'),
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 3000))
]);
Bulkhead: Isolate failures to a pool. If the recommendations pool is exhausted, other services still work.
Thread Pool: 200 total
├── Orders pool: 100 threads
├── Payments pool: 50 threads
└── Recommendations pool: 50 threads (can fail without affecting others)
18. What is rate limiting and how do you implement it in microservices?
Rate limiting prevents abuse and protects services from overload.
| Algorithm | Mechanism | Use case |
|---|---|---|
| Fixed window | N requests per window | Simple, burst-prone |
| Sliding window | N requests over rolling period | Smoother |
| Token bucket | Tokens refill at rate R; consume on request | Allows bursts up to bucket size |
| Leaky bucket | Requests drain at fixed rate | Smooth output rate |
Token bucket in Redis (Lua, atomic):
local key = KEYS[1]
local rate = tonumber(ARGV[1]) -- tokens per second
local capacity = tonumber(ARGV[2]) -- max burst
local now = tonumber(ARGV[3]) -- current timestamp (ms)
local requested = tonumber(ARGV[4]) -- tokens needed
local last = tonumber(redis.call('hget', key, 'last') or now)
local tokens = tonumber(redis.call('hget', key, 'tokens') or capacity)
tokens = math.min(capacity, tokens + rate * (now - last) / 1000)
if tokens >= requested then
redis.call('hmset', key, 'tokens', tokens - requested, 'last', now)
return 1 -- allowed
else
return 0 -- denied
end
19. How do you handle partial failures (graceful degradation)?
Return a degraded but functional response when a dependency fails:
async function getProductDetails(productId) {
const [product, reviews, recommendations] = await Promise.allSettled([
productService.get(productId), // critical
reviewService.getReviews(productId), // nice-to-have
recoService.getRecommendations(productId) // nice-to-have
]);
if (product.status === 'rejected') throw product.reason; // critical failure
return {
...product.value,
reviews: reviews.status === 'fulfilled' ? reviews.value : [],
recommendations: recommendations.status === 'fulfilled' ? recommendations.value : [],
};
}
Patterns:
- Return cached stale data
- Return empty/default values
- Show "recommendations unavailable" vs crash
- Feature flags to disable failing features
20. What is the Sidecar pattern?
A sidecar is a helper container deployed alongside the main service container, handling cross-cutting concerns.
Pod/VM:
┌─────────────────────────────────┐
│ Main Container (Order Service) │
│ │
│ Sidecar (Envoy Proxy) │
│ - mTLS │
│ - circuit breaking │
│ - distributed tracing │
│ - metrics collection │
└─────────────────────────────────┘
Service mesh (Istio, Linkerd) uses sidecars to provide these capabilities transparently without changing application code.
Security
21. How do you implement authentication in microservices?
Centralized auth with JWT (most common):
Client ──► API Gateway ──► validate JWT ──► attach user claims ──► Service
│
reads claims from header
(no DB call per request)
// API Gateway validates
const decoded = jwt.verify(token, JWT_PUBLIC_KEY);
// Passes to downstream service
req.headers['X-User-Id'] = decoded.sub;
req.headers['X-User-Roles'] = JSON.stringify(decoded.roles);
// Downstream service trusts gateway
const userId = req.headers['x-user-id'];
const roles = JSON.parse(req.headers['x-user-roles']);
Considerations:
- Gateway is the trust boundary — internal services trust forwarded headers
- Use HTTPS between gateway and services (or mTLS)
- Keep JWT payload small — roles/scopes, not full user object
22. What is mTLS and why use it between services?
Mutual TLS — both client and server authenticate with certificates.
Service A ──presents cert──► Service B
Service A ◄──presents cert── Service B
◄─────────────────►
encrypted
vs JWT auth:
| Aspect | JWT | mTLS |
|---|---|---|
| Identity | User identity | Service identity |
| Management | Short-lived tokens | Certificate rotation |
| Overhead | Token validation | TLS handshake |
| Automatic | Must pass token explicitly | Transparent with service mesh |
Service mesh (Istio) handles mTLS automatically — zero code changes needed in services.
23. How do you manage secrets in microservices?
Never hardcode secrets. Store in a secret manager:
| Tool | Use case |
|---|---|
| Kubernetes Secrets | Basic K8s deployments |
| HashiCorp Vault | Dynamic secrets, fine-grained policies |
| AWS Secrets Manager | AWS workloads, automatic rotation |
| Azure Key Vault | Azure workloads |
Dynamic secrets with Vault:
# Service requests a DB credential valid for 1 hour
vault read database/creds/my-role
# Returns: username=v-app-xyz, password=A1B2C3..., lease_duration=1h
# Auto-revoked after 1 hour
In Kubernetes:
env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: password
Observability
24. What are the three pillars of observability?
| Pillar | What | Tools |
|---|---|---|
| Logs | Timestamped event records | ELK, Loki, CloudWatch |
| Metrics | Numeric measurements over time | Prometheus, Datadog, CloudWatch |
| Traces | Request flow across services | Jaeger, Zipkin, AWS X-Ray |
Each answers different questions:
- Logs: "What happened at 14:32:07 in the payment service?"
- Metrics: "How many 5xx errors per minute over the last hour?"
- Traces: "Why did this checkout request take 3.2 seconds?"
25. What is distributed tracing?
Traces a request as it flows through multiple services, showing where time is spent.
Request ID: abc-123
┌─────────────────────────────────────────────────────┐
│ API Gateway 100ms total │
│ ├── Auth Service 5ms │
│ ├── Order Service 80ms │
│ ├── DB query 40ms ◄── slow! │
│ └── Inventory 35ms │
│ └── DB query 30ms │
└─────────────────────────────────────────────────────┘
Propagation via headers (W3C Traceparent standard):
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
version trace-id span-id flags
OpenTelemetry (language-agnostic SDK):
import { trace, context, propagation } from '@opentelemetry/api';
const tracer = trace.getTracer('order-service');
const span = tracer.startSpan('process-order');
// ... do work ...
span.end();
26. How do you implement structured logging in microservices?
Structured logs (JSON) enable searching and aggregation across services.
// Bad — unstructured
console.log(`Order ${orderId} failed for user ${userId}: ${error.message}`);
// Good — structured
logger.error('order.payment.failed', {
orderId,
userId,
errorCode: error.code,
errorMessage: error.message,
traceId: getTraceId(), // correlates with distributed trace
duration: Date.now() - startTime,
});
Correlation ID pattern: Pass a X-Request-Id header through all services. Include in every log entry to trace a request across services without full distributed tracing.
27. What is the difference between health check, liveness, and readiness?
| Probe | Question | Failure action |
|---|---|---|
| Health check | "Is the service responding?" | Alert, restart |
| Liveness | "Is the service in a good state (or deadlocked)?" | Restart the container |
| Readiness | "Is the service ready to receive traffic?" | Remove from load balancer (don't restart) |
Kubernetes:
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
Readiness check might verify: DB connection, cache connection, dependencies available.
Liveness check: Simple — if it responds, process isn't deadlocked.
Deployment and CI/CD
28. What deployment strategies work well for microservices?
| Strategy | How | Risk | Rollback speed |
|---|---|---|---|
| Recreate | Stop old, start new | High (downtime) | Fast |
| Rolling update | Replace instances gradually | Low | Slow |
| Blue-green | Run both, switch traffic | Low | Instant |
| Canary | Route small % to new version | Very low | Instant |
| A/B testing | Route by user segment | Low | Instant |
Canary with Kubernetes + Argo Rollouts:
spec:
strategy:
canary:
steps:
- setWeight: 10 # 10% to new version
- pause: {duration: 10m}
- setWeight: 50
- pause: {duration: 10m}
- setWeight: 100
29. How do you handle database schema migrations in microservices?
Expand-contract pattern (zero-downtime migrations):
Phase 1 — Expand: Add new column (nullable), deploy new service that writes both old + new column.
ALTER TABLE orders ADD COLUMN customer_ref VARCHAR(255);
Phase 2 — Migrate: Backfill data, new code reads new column.
UPDATE orders SET customer_ref = user_id::text WHERE customer_ref IS NULL;
Phase 3 — Contract: Drop old column after all instances run new code.
ALTER TABLE orders DROP COLUMN user_id;
Tools: Flyway, Liquibase, Alembic. Run migrations in CI/CD before deploying new service version.
30. What is GitOps and how does it apply to microservices?
GitOps uses Git as the single source of truth for infrastructure and deployments. A Git commit is the trigger for deployment.
Developer ──PR──► Git ──merge──► CI builds image ──► pushes to registry
│
ArgoCD watches git repo
│
Detects drift ──► syncs K8s cluster
Tools: ArgoCD, Flux.
Benefits for microservices:
- Audit trail — every deployment is a Git commit
- Easy rollback — revert the commit
- Environment parity — same manifests for staging/prod (different values)
Service mesh
31. What is a service mesh?
A service mesh is an infrastructure layer that handles service-to-service communication, providing:
| Feature | Without mesh | With mesh (e.g., Istio) |
|---|---|---|
| mTLS | Code in each service | Automatic via sidecar |
| Circuit breaking | Library per service | Config in control plane |
| Distributed tracing | SDK in each service | Automatic header injection |
| Traffic management | Manual | Declarative (VirtualService) |
| Observability | Per-service | Unified dashboard |
Architecture:
Data plane: Envoy sidecars (handle actual traffic)
Control plane: Istio Pilot (configure sidecars), Citadel (certs), Galley (config)
32. Traffic management with a service mesh — give examples.
# Canary: 90% to v1, 10% to v2
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
spec:
http:
- route:
- destination:
host: product-service
subset: v1
weight: 90
- destination:
host: product-service
subset: v2
weight: 10
# Timeout and retry
http:
- timeout: 3s
retries:
attempts: 3
perTryTimeout: 1s
retryOn: 5xx,gateway-error
Testing microservices
33. What is the testing pyramid for microservices?
┌───────────────┐
│ E2E Tests │ ← Few, slow, expensive
├───────────────┤
│ Integration │ ← Service + DB + queue
│ Tests │
├───────────────┤
│ Contract │ ← Inter-service boundaries
│ Tests │
├───────────────┤
│ Unit Tests │ ← Many, fast, cheap
└───────────────┘
Contract testing is the microservices-specific addition.
34. What is consumer-driven contract testing?
Contract testing verifies that services fulfill the contracts consumers expect, without needing to run all services together.
With Pact:
// Consumer (Order Service) defines expectations
const interaction = {
state: 'product 123 exists',
uponReceiving: 'a request for product 123',
withRequest: { method: 'GET', path: '/products/123' },
willRespondWith: {
status: 200,
body: { id: '123', name: 'Widget', price: 9.99 }
}
};
// Pact generates a contract file
// Provider (Product Service) verifies it can fulfill the contract
await provider.verifyProvider();
// Runs against real Product Service with mocked state
Why: Avoids needing all services running for integration tests. Fast feedback on breaking changes.
35. How do you test a service that depends on message queues?
// Option 1: Use a real queue (TestContainers)
const container = await new GenericContainer('confluentinc/cp-kafka')
.withExposedPorts(9092)
.start();
// Option 2: Mock the publisher/consumer
const mockPublisher = jest.fn();
const service = new OrderService({ publisher: mockPublisher });
await service.createOrder({ userId: '1', items: [...] });
expect(mockPublisher).toHaveBeenCalledWith('order.created', expect.objectContaining({
orderId: expect.any(String),
}));
Common design patterns
36. What is the Strangler Fig pattern?
Gradually migrate a monolith by routing traffic to new microservices alongside the old system.
Phase 1: All traffic → Monolith
Phase 2: /products → Product Microservice; rest → Monolith
Phase 3: /products + /orders → Microservices; rest → Monolith
Phase 4: All traffic → Microservices; Monolith retired
Use the API Gateway to control routing without changes to clients.
37. What is the Anti-Corruption Layer (ACL)?
A layer that translates between the models of two bounded contexts, preventing one service's model from corrupting another's.
// External payment provider uses their domain model
class PaymentProviderACL {
adapt(providerResponse) {
return {
// Our internal model
transactionId: providerResponse.txn_ref,
status: this.mapStatus(providerResponse.state),
amount: providerResponse.amount_cents / 100,
};
}
mapStatus(providerState) {
const map = { 'ACCEPTED': 'approved', 'DECLINED': 'rejected', 'PENDING': 'pending' };
return map[providerState] ?? 'unknown';
}
}
38. Explain event-driven vs request-driven microservices.
Request-driven:
Order Service ──POST /payments──► Payment Service (waits for response)
Simple but: Order Service is coupled to Payment Service availability.
Event-driven:
Order Service ──publishes──► kafka: order.created
Payment Service ◄──consumes──
Payment Service ──publishes──► kafka: payment.completed
Order Service ◄──consumes──
Decoupled but: harder to debug, eventual consistency, no immediate response.
Hybrid (CQRS + events): Writes via events, queries via synchronous API.
39. What is idempotency and why is it critical in distributed systems?
An operation is idempotent if calling it multiple times produces the same result as calling it once.
Why critical: In distributed systems, messages can be delivered more than once (at-least-once delivery). Without idempotency, duplicate events cause double-charges, duplicate orders, etc.
Implementing idempotency:
// Idempotency key approach
async function createOrder(request) {
const idempotencyKey = request.idempotencyKey; // client-provided UUID
// Check if we've already processed this key
const existing = await db.query(
'SELECT * FROM orders WHERE idempotency_key = $1', [idempotencyKey]
);
if (existing) return existing; // return same result
// Process and store with idempotency key
const order = await processNewOrder(request);
await db.query(
'INSERT INTO orders (..., idempotency_key) VALUES (..., $1)',
[..., idempotencyKey]
);
return order;
}
40. How do you implement the Inbox pattern (idempotent consumers)?
The inbox ensures each message is processed exactly once, even with at-least-once delivery.
-- Inbox table
CREATE TABLE inbox (
message_id VARCHAR PRIMARY KEY, -- Kafka offset or message UUID
processed_at TIMESTAMP
);
-- Processor
BEGIN;
INSERT INTO inbox (message_id) VALUES ($1)
ON CONFLICT DO NOTHING; -- idempotency guard
-- Only proceed if we inserted (not a duplicate)
IF rows_affected > 0 THEN
-- process business logic
INSERT INTO orders (...) VALUES (...);
END IF;
COMMIT;
Frequently asked
41. How many microservices is too many?
There's no fixed number. Signs you have too many:
- Services smaller than a function — no clear bounded context
- More time managing infrastructure than building features
- Constant cross-service coordination needed for every feature (chatty services)
Signs you have too few (services too large):
- Multiple teams contend on same codebase/deployment
- Services have 5+ unrelated responsibilities
Practical heuristic: A service should be maintainable by a small team (2-5 people) end-to-end.
42. How do you prevent cascading failures?
Defense in depth:
- Timeout every external call — never wait indefinitely
- Retry with backoff + jitter — don't retry all at once (thundering herd)
- Circuit breaker — stop calling a failing service
- Bulkhead — isolate thread pools / connection pools per service
- Graceful degradation — return cached/default data when dependency fails
- Rate limiting — protect services from overload upstream
43. How do you version microservices APIs?
| Strategy | Example | Trade-off |
|---|---|---|
| URL versioning | /api/v1/orders, /api/v2/orders |
Visible, easy to route |
| Header versioning | Accept: application/vnd.company.v2+json |
Cleaner URLs, harder to test in browser |
| Query param | /orders?version=2 |
Simple but pollutes query |
| Consumer-driven contracts | Pact contracts per consumer version | No explicit versioning |
Practical approach:
- Support N and N-1 simultaneously during transition
- Use the Expand-Contract pattern — add fields before removing
- Never remove fields from existing versions — add new endpoints
44. What is the two-phase commit (2PC) and why is it avoided in microservices?
2PC is a distributed protocol for atomic commits across multiple databases:
- Prepare phase: Coordinator asks all participants to lock resources and confirm ready
- Commit phase: If all say yes, coordinator tells everyone to commit; else abort
Why avoided:
- Requires all services to be available during the transaction
- Locks resources for the duration (low throughput)
- Coordinator becomes a single point of failure
- Doesn't work across services with independent databases
Alternative: Saga pattern with compensating transactions.
45. How do you handle service-to-service latency?
| Technique | How |
|---|---|
| Async over sync | Use events instead of blocking calls |
| Caching | Cache responses (Redis), avoid repeated downstream calls |
| Connection pooling | Reuse HTTP connections (HTTP keep-alive) |
| gRPC with HTTP/2 | Multiplexing, binary protocol, lower overhead than REST |
| Request collapsing | Batch multiple requests into one downstream call |
| Locality | Co-locate services that talk frequently (same AZ) |
| CDN/edge caching | Cache at edge for read-heavy APIs |
46. What is the Twelve-Factor App methodology?
The 12-Factor App defines principles for building cloud-native (microservices-ready) apps:
| Factor | Principle |
|---|---|
| Codebase | One codebase, multiple deploys |
| Dependencies | Explicitly declare, isolate dependencies |
| Config | Store config in environment |
| Backing services | Treat as attached resources |
| Build/release/run | Strict separation |
| Processes | Execute as stateless processes |
| Port binding | Export services via port binding |
| Concurrency | Scale out via process model |
| Disposability | Fast startup, graceful shutdown |
| Dev/prod parity | Keep environments similar |
| Logs | Treat as event streams |
| Admin processes | Run as one-off processes |
47. How do you handle configuration management across services?
| Approach | How |
|---|---|
| Environment variables | Per-deployment config (12-Factor) |
| Config service | Spring Cloud Config, Consul KV, etcd |
| Kubernetes ConfigMaps | Mount as files or env vars |
| Feature flags | LaunchDarkly, Unleash — runtime config without deploy |
Config service pattern:
Services ──startup──► Config Service ──reads──► Git repo / Vault
returns config
(watches for changes)
48. What happens when a message broker goes down?
Impact: All async communication fails. Services that only communicate via messages can't talk to each other.
Mitigation strategies:
- High availability brokers: Kafka with replication factor 3, RabbitMQ mirrored queues
- Fallback to sync: If Kafka is down, fall back to synchronous HTTP (with circuit breaker)
- Local buffering: Queue messages locally (SQLite/outbox) until broker recovers
- Health check + graceful degradation: If broker unreachable, return "order accepted, will process shortly"
49. How do you do load testing in a microservices environment?
# k6 load test targeting order service
k6 run --vus 100 --duration 5m - <<'EOF'
import http from 'k6/http';
import { check } from 'k6';
export default function() {
const res = http.post('https://api.example.com/orders', JSON.stringify({
userId: '123', items: [{ productId: 'abc', qty: 1 }]
}), { headers: { 'Content-Type': 'application/json' } });
check(res, {
'status 201': (r) => r.status === 201,
'response < 500ms': (r) => r.timings.duration < 500,
});
}
EOF
What to observe during load test:
- Which service saturates first (CPU, memory, DB connections)
- Error rates per service (distributed tracing)
- Queue depth (if using messaging)
- Circuit breaker state changes
50. Microservices vs SOA — what's the difference?
| Aspect | SOA | Microservices |
|---|---|---|
| Communication | ESB (Enterprise Service Bus) | Lightweight APIs, message brokers |
| Coupling | High (shared ESB, shared data) | Low (each service independent) |
| Granularity | Coarser services | Fine-grained services |
| Technology | Often homogeneous, vendor-specific | Polyglot |
| Deployment | Often together | Independently deployable |
| Focus | Integration of enterprise systems | Independent business capabilities |
| Governance | Centralized | Decentralized |
Common mistakes
| Mistake | Why it's wrong | Fix |
|---|---|---|
| Shared database | Creates tight coupling — any schema change breaks multiple services | Database per service |
| Chatty services (sync calls for everything) | Network latency compounds, cascading failures | Use async events for non-critical flows |
| Starting with microservices | Premature optimization — adds complexity before you understand the domain | Start with monolith, migrate as needed |
| No distributed tracing | Impossible to debug cross-service latency | Add OpenTelemetry from day one |
| No contract tests | Breaking changes discovered in production | Consumer-driven contracts with Pact |
| Microservices without CI/CD | Manual deployments across many services is chaos | Automate everything |
| Distributed monolith | Services that must deploy together or share data | Respect bounded contexts |
| No idempotency | Duplicate messages cause duplicate side effects | Idempotency keys + inbox pattern |
Microservices vs related architectures
| Architecture | Granularity | Deployment | Communication | Best for |
|---|---|---|---|---|
| Monolith | One unit | Together | In-process | Small teams, simple domains |
| Microservices | Fine-grained services | Independent | Network (REST/gRPC/events) | Large teams, complex domains |
| SOA | Coarser services | Together via ESB | ESB | Enterprise integration |
| Serverless | Functions | Auto-managed | Events/HTTP | Variable load, stateless functions |
| Modular monolith | Modules | Together | In-process | Intermediate complexity |
FAQ
Q: Should I use microservices from the start?
A: Almost never. Start with a modular monolith — clear module boundaries without network overhead. Migrate to microservices when you have multiple teams stepping on each other's toes, or distinct scaling needs. "Microservices first" is one of the most expensive mistakes in software architecture.
Q: How do you debug a request that touches 10 services?
A: Distributed tracing (Jaeger/Zipkin/AWS X-Ray). Every request carries a trace-id header. The trace shows the full call graph, timing for each service, and which service caused the slowdown. Without tracing, debugging microservices is guesswork.
Q: What's the difference between orchestration and choreography in Saga?
A: In choreography, services react to events independently (no central coordinator — services just subscribe to topics). In orchestration, a saga orchestrator drives the workflow by explicitly calling each service. Orchestration is easier to understand and debug; choreography scales better and has no single point of failure.
Q: How do you avoid a distributed monolith?
A: A distributed monolith has microservices that must be deployed together (tight runtime coupling) or share a database (tight data coupling). Avoid by: respecting bounded contexts, using database-per-service, accepting eventual consistency, and deploying services independently. If changing service A always requires changing service B, you have the wrong service boundaries.
Q: What's the hardest part of microservices?
A: Data management. In a monolith, you have ACID transactions. In microservices, data is distributed — you need sagas, eventual consistency, idempotency, and the outbox pattern. Most teams underestimate this. The second hardest is observability — debugging across services requires investment in tracing, structured logging, and metrics from day one.
Q: When should I split a microservice further?
A: When a single service violates single responsibility (has unrelated business capabilities), when two teams constantly need to coordinate changes to the same service, when parts of the service have wildly different scaling requirements, or when you want to use different technology for different parts.