Decision Rules
TL;DR
- This page is an index, not a textbook. Every row is a one-line rule plus a link to the page that owns the argument. If a rule surprises you, follow the link.
- Pick by the constraint that bites first: who the client is, what the call shape is, where the state lives, and what a failure costs.
- Most of these are tradeoffs, not winners. HTTP/3 does not retire HTTP/2, L7 does not retire L4, and a NAT gateway does not retire the free gateway endpoint.
- Defaults worth saying aloud: dual-stack for IPv6, SG as the application filter with a NACL as backstop, externalized session state over sticky sessions, versioned URLs over cache purges.
Cheat sheet
Transport and HTTP
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| TCP | In-order, reliable, flow- and congestion-controlled byte stream | Handshake; one lost segment stalls everything behind it; no message boundaries | You need every byte, in order, and can wait for retransmission | Transport overview |
| UDP | One sendto is one datagram; no setup; you own the semantics | No promises: loss, duplication and reordering are your problem; spoofable sources make it an amplification weapon | Late is worse than lost, or you are building your own transport (QUIC, WireGuard, DNS retries) | UDP |
| QUIC | Reliability, TLS 1.3 and independent streams in one user-space transport over UDP; 1 RTT handshake | Needs a UDP path, so browsers keep a TCP fallback | Lossy, mobile or high-RTT paths where HTTP/3 pays off | QUIC and HTTP/3 |
| HTTP/1.1 | Simple; persistent connections | One request in flight per connection (browsers open about six per host) | Simple or legacy clients | HTTP/1.1 and HTTP/2 |
| HTTP/2 | Many streams multiplexed on one connection; HPACK header compression | One lost TCP segment stalls every stream | Default over TLS when TCP is fine | HTTP/1.1 and HTTP/2 |
| HTTP/3 | HTTP over QUIC; a lost packet stalls only the streams it carried | Needs UDP; gains are small on a fast wired path | Mobile, lossy or high-RTT users | QUIC and HTTP/3 |
API style and real-time
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| REST | Plain HTTP; caches, proxies and browsers understand it | No built-in streaming | Public APIs, CRUD, anything that must work with caches and browsers | RPC and APIs |
| gRPC | Required .proto schema, binary protobuf, all four call shapes | Browsers need gRPC-Web or Connect; an L4 balancer pins one client to one backend | Service to service, streaming, many languages | RPC and APIs |
| GraphQL | Client names exactly the fields it wants | Not cacheable by default (queries are POST) | Many client types with different data needs | RPC and APIs |
| Webhook | Provider calls you when something happens | At-least-once: verify the signature, dedupe by event id | Being told that something happened | RPC and APIs |
| Polling | Ordinary requests, best proxy friendliness | Staleness up to the poll interval | Dashboards that tolerate staleness, job status | Real-time patterns |
| Long polling | Ordinary requests, about one one-way delay to see an event | Client re-issues each request; proxies may time out held ones | Fallback transport, restrictive networks | Real-time patterns |
| SSE | Server to client over plain HTTP; automatic reconnect with Last-Event-ID | One-way, UTF-8 text only; a buffering proxy can hold events back | Notifications, live feeds, LLM token streams | Real-time patterns |
| WebSocket | Full duplex, text or binary | Needs proxy Upgrade support; you write reconnect and heartbeats | Chat, games, collaborative editing | Real-time patterns |
Load balancing and traffic steering
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| L4 load balancer | Any TCP or UDP protocol, very high packet rates, can preserve the client IP | Sees IP and port only | Raw speed, non-HTTP protocols, fixed IPs | Load balancing |
| L7 load balancer | Routes on host, path, header or cookie; retries, rewrites, WAF | Terminates the client connection; more work per request | Routing depends on the request itself | Load balancing |
| DNS-based steering | Picks a region by changing the DNS answer | Cached and bounded by TTL; coarse | Sending users to a region | Global traffic steering |
| Round robin | Trivial | Ignores load | Requests cost about the same | Load balancing |
| Least connections or P2C | Keeps the busiest backend far less loaded | Needs connection counts | Request costs differ | Load balancing |
| Consistent hashing | Adding a node moves only about 1/(N+1) of keys, where hash % N moves N/(N+1) | Lumpy load without virtual nodes | Cache locality or sharded state | Consistent hashing |
| Sticky sessions | In-memory session state survives between requests | Uneven load; scale-in and failure lose the session | Only where affinity is genuinely wanted | Load balancing |
| Stateless backends | Any backend serves any request | State moves to a shared store or signed token | The default | Load balancing |
| Hash on source IP (L4) | Needs no cookie | Misfires behind NAT, where many users share one address | L4 affinity with no other handle | Load balancing |
| Cookie affinity (L7) | Pins per browser, not per address | L7 only | L7 affinity | Load balancing |
| Single-site unicast | One address, one place | One site is the failure domain | One region is enough | Global traffic steering |
| Anycast-fronted multi-site | One IP announced from every site; routing picks | Follows BGP policy, not geography or latency; a route change can move a user mid-connection | Multi-site with fast failover | Global traffic steering |
| Active-active | Small RTO | A hard data problem (writes, replication lag, fencing) | You can solve multi-writer data | Global traffic steering |
| Active-passive | Simplicity | A failover event | One writer region is acceptable | Global traffic steering |
TLS, proxies and crypto
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| Terminate TLS at the LB | One place for certificates; L7 can read and route; backends skip handshake CPU | The LB-to-backend leg is plaintext unless re-encrypted | Most web services | Load balancing |
| Terminate and re-encrypt | Same, plus an encrypted backend leg | A second encrypted hop to run | Compliance requires it | Load balancing |
| Passthrough | End-to-end encryption; routing only on SNI | LB cannot read requests; every backend holds the certificate | Encrypted bytes must stay opaque to the LB | Load balancing |
| Forward proxy | Acts for clients: egress control, filtering | Same shape as a reverse proxy, opposite side | Controlling outbound traffic | Proxies and gateways |
| Reverse proxy | Acts for servers: ingress, hides backends | Backend sees the proxy's IP unless passed on | Fronting servers | Proxies and gateways |
| Symmetric crypto | Fast bulk encryption (AEAD) | Needs a shared key first | Carrying the data | Symmetric crypto and hashing |
| Asymmetric crypto | Key agreement and signatures with no pre-shared secret | Slower; not used for bulk data | Establishing trust and a key | Key exchange and public-key crypto |
| Hybrid (what TLS does) | Public-key crypto bootstraps; a symmetric cipher carries the data | Both mechanisms to get right | Always, in practice | TLS |
Cloud networking
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| Security group | Stateful, allow-only, on the interface; replies return automatically | No deny rules | The application's filter | Security groups vs NACLs |
| NACL | Stateless, numbered, allow and deny, at the subnet edge | Rules for both directions, including ephemeral return ports | A coarse backstop, or the way to write an explicit deny, which security groups cannot | Security groups vs NACLs |
| VPC peering | Point-to-point route wiring | Non-transitive; ranges must not overlap; full mesh needs n(n-1)/2 peerings | A few VPCs | Private connectivity |
| Transit gateway | Hub: one attachment per VPC; transit works | Billed per attachment-hour and per GB processed | Many VPCs | Private connectivity |
| PrivateLink (interface endpoint) | Exposes one service, one way; works with overlapping CIDRs | Not a network; billed per hour and per GB | Overlapping CIDRs or one-service exposure | Private service endpoints |
| Site-to-site VPN | IPsec over the internet; up in minutes | Per-tunnel bandwidth ceiling | Fast start, or backup for a dedicated link | Private connectivity |
| Dedicated link (Direct Connect, Interconnect, ExpressRoute) | Steady latency, high bandwidth | Physical cross-connect, plan in weeks | Primary hybrid path | Private connectivity |
| NAT gateway | Managed many-to-one outbound IPv4 | Billed per hour and per GB; SNAT port limits | Private subnets reaching the internet | NAT gateways and egress |
| NAT instance | Same behaviour, you control the box | You patch, size and make it highly available; a single point of failure | Rarely; the managed gateway replaced it | NAT gateways and egress |
| Gateway endpoint (S3, DynamoDB) | A route-table entry; free | Only those two services | Reaching S3 or DynamoDB from private subnets | Private service endpoints |
| VPN (WireGuard or IPsec) | Encrypted path across an untrusted network | Costs MTU, so clamp the MSS; moves trust to the VPN operator | Joining sites or remote users | VPN, IPsec and WireGuard |
Caching, DNS and resilience
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| Straight to origin | Nothing to configure | Every user pays the full round trip; origin takes every request | Small, local audience | CDN |
| CDN in front | Shorter round trips and a shielded origin | Invalidation and cache-poisoning risks | Anything with distant users or load to absorb | CDN |
| Pull CDN | Fetches on a miss; change only DNS and headers | First user per PoP pays a miss | Most sites and APIs | CDN |
| Push CDN | No cold miss; origin off the path | Storage you pay for; upload and invalidation logic you own | Large, rarely changing, predictable files | CDN |
| Versioned URLs (content hash) | Long lifetimes with exact invalidation | Needs a build step | Assets | HTTP caching |
| Short or revalidated lifetime | Changes reach users quickly | More requests | The HTML that points to the assets | HTTP caching |
| Short DNS TTL | A changed record reaches clients quickly | More walks, more load on authoritative servers | Before a planned change | DNS resolution |
| Long DNS TTL | Cheaper and faster on average | A bad record stays wrong for up to the TTL | Stable records | DNS resolution |
| New connection per request | Simplest | TCP plus TLS 1.3 is 2 RTTs before the request is answered; ports pile up in TIME_WAIT | Rarely | TCP performance |
| Keep-alive and pooling | Pays the handshake once | Idle connections can be dropped by NATs and balancers | Almost always | TCP performance |
| Retry | Survives transient failure | Can duplicate work and sustain overload | Idempotent calls or calls with an idempotency key, with backoff, jitter and a budget | Timeouts, retries and backoff |
| Fail fast | Protects the caller | The user sees the error | Non-idempotent work, or a failing dependency (circuit breaker) | Timeouts, retries and backoff |
Layer 2, routing and addressing
| Option | Strength | Cost | Pick when | Owner |
|---|---|---|---|---|
| Hub | Repeats bits to every port | One shared collision domain | Legacy gear only | Switches and MAC learning |
| Switch | Forwards only toward the destination MAC; each port is its own collision domain | One broadcast domain unless split | Connecting hosts on a LAN | Switches and MAC learning |
| Router | Joins broadcast domains; cross-VLAN traffic goes through it | Per-hop forwarding decision | Joining networks | VLANs |
| VLAN | Splits a switch into separate broadcast domains | Traffic between VLANs must cross a router | Segmenting a LAN | VLANs |
| Subnet | The IP-layer prefix a router forwards on | Needs addressing and routing | Planning address space | Subnet design practice |
| OSPF | Link-state: every router holds the same map and runs Dijkstra; fast, consistent convergence | Scope is one AS | Routing inside one organization | OSPF |
| BGP | Exchanges reachability between ASes; path chosen by policy | Not shortest path: Local Preference is checked before AS-PATH length | Routing between organizations | BGP |
| Dual-stack | Works with IPv6 and IPv4-only peers | Two stacks to run and secure | The default | IPv6 |
| IPv6-only | No NAT needed | Needs NAT64/DNS64 to reach IPv4-only services | A deliberate choice, not a default | IPv6 |
Textbook vs real world
| The textbook says | Production does | Why |
|---|---|---|
| Use REST outside and gRPC inside | Pick by client and call shape | It is a habit, not a law (RPC and APIs) |
| HTTP/2 is strictly better than HTTP/1.1 | It can lose on a lossy link | One lost segment stalls every stream on the single connection (HTTP/1.1 and HTTP/2) |
| Anycast sends you to the nearest site | It sends you where BGP policy says | Policy, not geography or latency (Global traffic steering) |
| Use sticky sessions to keep users logged in | Externalize session state | Stickiness breaks on scale-in, deploys and failure, and skews load (Load balancing) |
| New design means IPv6 | Dual-stack, with IPv6-only as a deliberate choice | The real transition path is dual-stack plus Happy Eyeballs (IPv6) |
Gotchas
GOTCHAS3
Greenfield means IPv6 only
Unicast versus anycast is a tradeoff
Use UDP when occasional loss is acceptable
Interview questions
INTERVIEWREST versus gRPC: when would you pick each?mid
Show answer
Pick by who the client is and what the call looks like. REST fits public APIs and CRUD because caches, proxies and browsers understand plain HTTP. gRPC fits service to service and streaming: a required .proto schema, binary protobuf over HTTP/2, and all four call shapes. Browsers need gRPC-Web or Connect, and an L4 balancer pins one client to one backend, so use an L7 proxy or client-side balancing. Row detail: RPC and APIs.
Follow-ups:
- Where does GraphQL fit? Many client types with different data needs, at the cost of default HTTP caching.
- Does gRPC set a deadline? No, so you must set one.
INTERVIEWPolling, WebSocket or SSE for a live feed?mid
Show answer
Start with direction. If only the server pushes (notifications, live feeds, token streams), SSE is plain HTTP with built-in reconnect and Last-Event-ID resume. If both sides talk freely (chat, games), WebSocket gives full duplex but needs proxies to handle Upgrade and you write reconnect and heartbeats. Polling is for dashboards that tolerate staleness, and long polling is a fallback on restrictive networks. See Real-time patterns.
Follow-ups:
- What kills an idle connection? NATs and load balancers; AWS ALB defaults to 60 s, so send an application heartbeat well under that.
- How should clients reconnect? Exponential backoff plus jitter.
INTERVIEWL4 versus L7 load balancing: what are the tradeoffs?senior
Show answer
L4 sees IP and port only, so it is fast and protocol-agnostic and can preserve the client IP, but cannot route on content. L7 terminates the client connection and opens its own to the backend, so it can route on host, path, header or cookie, retry, rewrite, and terminate TLS, at more cost per request. With TLS passthrough the balancer is effectively L4 and can only route on SNI. See Load balancing.
Follow-ups:
- Why does gRPC need care? An L4 balancer pins all of one client's calls to one backend.
- How does the backend see the client IP behind L7?
X-Forwarded-Foror the PROXY protocol.
INTERVIEWHow do you choose between a push and a pull CDN?senior
Show answer
Pull is the default: the CDN fetches from the origin on a miss, so you change only DNS and headers, but the first user per PoP pays a miss. Push has you upload ahead of time, which suits large, rarely changing, predictable files where the origin should be off the path, at the cost of storage and invalidation logic you own. See CDN.
Follow-ups:
- What about purging? Prefer versioned URLs, then purge by tag; a CDN purge does not touch browsers (HTTP caching).
- How do you protect the origin? Accept traffic only from the CDN so the origin cannot be bypassed.
INTERVIEWPeering, Transit Gateway or PrivateLink for 50 VPCs?senior
Show answer
Peering is non-transitive and needs non-overlapping ranges, so a full mesh of 50 VPCs needs 50 x 49 / 2 = 1,225 peerings. A transit gateway is a hub with one attachment per VPC (50), and transit works, but it bills per attachment-hour and per GB processed. PrivateLink exposes one service one way and works with overlapping CIDRs, so it is the fallback for overlap or single-service sharing, not a network. See Private connectivity.
Follow-ups:
- How do costs differ? Peering has no hourly attachment charge; check the pricing pages.
- Hybrid to on-prem? A dedicated link as primary with a VPN as backup.