API Gateway

An API gateway is an edge entry point for routing and selected cross-cutting policies. It can centralize TLS termination, authentication integration, quotas, request shaping, and observability, but it should not silently become the owner of every business rule. Its capacity, configuration rollout, and failure behavior belong in the design.

The mechanism at a glance

Figure — Client → Gateway policy (TLS + identity); Gateway policy → Route table (bounded policy); Route table → Service owner (route); Identity context → Service owner (trusted principal); Service owner → Upstream budget (deadline/concurrency)

The numbered components identify responsibilities. Follow the labeled arrows rather than treating the numbers as a global execution order. The scenario later in this lesson shows one concrete sequence.

Step-by-step reasoning

1. 1 · Frame

Map external routes to service ownership and protocols. Specify which checks happen at the edge—coarse request size, token verification, route quota—and which must happen in the service—object authorization and business invariants. Define versioning, timeouts, retry policies, and how client identity propagates without trusting spoofable headers.

2. 2 · Model

Routes are configuration with version and rollout state. A request carries trace identity, authenticated principal, tenant, deadline, and bounded payload. Gateways should avoid logging credentials and sensitive bodies. Service discovery and route health have their own freshness. Streaming, large uploads, and WebSockets require explicit timeout and buffering behavior.

3. 3 · Scale

Use a fleet across failure zones and keep gateway CPU/memory limits known. Rate-limit by keys available at that layer but enforce critical global budgets at their authority. Circuit breaking can reduce pressure but needs a recovery policy. Deploy route changes gradually and retain rollback. Avoid synchronous gateway-to-many-service orchestration that adds latency to every request.

4. 4 · Recover

If the gateway is unavailable, can clients reach safe fallback endpoints? If it retries a POST after an upstream timeout, can it duplicate an effect? Use idempotency-aware retries and propagate an end-to-end deadline. A healthy gateway can still send traffic to an unhealthy backend; readiness and load shedding should consider upstream capacity.

Contracts and state

The following sketch makes the decision boundary concrete. Field names and capacity assumptions are illustrative; adapt them to the stated product contract.

Route(method, path, service, version, policy)
Context(principal, tenant, trace_id, deadline)
Gateway: authenticate/coarse limit -> route -> service
Service: object authorization -> invariant -> durable effect
Never log bearer credentials or trust client tenant headers

Worked example

A mobile client sends a large upload through the API. Proxying every byte through the gateway consumes connection and bandwidth capacity. The gateway can authenticate the upload-session request and issue scoped object-store authorization; the byte transfer takes a separate path. Finalization returns to the service for verification.

Failure walkthrough

A gateway retry policy repeats a POST when an upstream response times out. The upstream may already have committed. Restrict retries to safe operations or propagate a stable idempotency identity that the owning service enforces. Protect against retry amplification during partial backend failure and monitor route-specific timeouts, concurrency, and rejected requests.

Figure — Gateway sends POST → Service commits order → Response times out → Retry carries same identity → Service returns original result

Decisions and trade-offs

Check your understanding

Which authorization check belongs in a service even if the gateway already authenticated the user?

Primary documentation

Read the first-party engineering account or official technical reference. Company engineering posts describe the scope and date of that publication; the interview reconstruction and scenarios here are original teaching examples.

Continue the connection

Study Networking Essentials and explain which guarantee from this lesson carries into that topic.

Define the boundary it actually owns

A gateway can terminate supported protocols, authenticate tokens, route requests, apply quotas and collect telemetry. Authorization still needs resource context: a valid token does not prove that its owner can access a particular school or document. Decide which checks belong at the gateway and which require application data.

Keep gateway work bounded. Large transformations, long synchronous workflows or unbounded body buffering turn a shared ingress tier into a bottleneck. Propagate request identity and deadline to downstream services. If the gateway returns a timeout, the backend may still commit; clients need operation identity for safe retry.

Separate overload protection from customer quota enforcement. A WAF rule, a per-user rate limit and an internal concurrency cap solve different problems. Use route-specific limits because an inexpensive read and a large export request have different costs. Connection draining and versioned routing configuration are part of safe deployment.

Figure — A decision worksheet for API Gateway: read the mechanism and its guarantee together.

Operational sketch

edge -> gateway -> authentication -> route policy
application -> tenant/resource authorization
deadline and trace ID propagated
large job -> 202 + status resource

A tempting mistake

Putting a gateway in front of an API does not automatically make the origin unreachable directly. Restrict and authenticate origin access or clients may bypass the intended controls.

Transfer exercise

Which layer should check that a teacher can read one student record?