Learning pathsPricingA
GUIDED PRACTICE

Practice: Distributed Cache

Bound memory, TTL, routing, migration, and refill work for a disposable distributed cache.

Design a distributed disposable cache that serves a high-read product catalog. Keys are partitioned across nodes, entries have TTLs, and one shard failure can cause a large miss spike.

Scenario and constraints

  • Assume 100 million entries and state whether the byte estimate includes keys and metadata.
  • Peak request rate is one million/s. Compare a 1% and 5% miss rate against an origin safe limit of 12,000 reads/s.
  • Define GET, SET, DELETE, TTL, bounded MGET, and optional CAS.
  • State the topology authority, migration epoch, replica-promotion policy, and behavior of old owners.
  • Protect the origin with same-key coalescing, a global refill budget, and a policy for absent, unauthorized, and transient failures.

Your deliverables

  1. Draw the cache node's entry layout, allocator classes, TTL behavior, atomic operations, and eviction policy.
  2. Work through tokens 20, 60, and 90; route hash 45; then add token 50 and identify exactly which interval moves.
  3. Explain snapshot/copy/cutover, membership authority, fencing, and replica staleness.
  4. Calculate memory including replication and headroom, plus read amplification during a miss spike.
  5. Write the bounded refill and negative-cache behavior and define what callers receive when admission is exhausted.
  6. Specify the measurements and recovery drill for shard loss and a bad topology epoch.

Failure scenario

A request for key K reads source version 18 at second 0 with a 55-second freshness window. Version 19 commits at second 1, the invalidator is delayed, and the version-18 response arrives at second 60. Meanwhile, a promoted replica may not have the latest write. State what the cache may serve and how it protects the origin during recovery.

Self-review

Score each deliverable 0 when missing, 1 when it names a technology, and 2 when it names the authority, invariant, bound, and observable failure. A cache write acknowledgment is not a durable origin commit, and consistent hashing does not split a single hot key.

Clarify this prompt before choosing components

Ask the highest-impact questions first, agree on priorities, and record the interviewer’s answers. If a detail is unspecified, state an assumption and explain which decision depends on it. The numbers in this worked lesson are exercise assumptions, not company facts.

Ask the interviewerWhy the answer changes the design
Cache-aside or a durable database substitute?A reconstructible cache can make different loss guarantees.
TTL, eviction, hot keys and item-size distribution?Capacity and admission must use working set and skew, not average key count.
Consistency during membership change or partition?Declare stale-read and write-routing behavior; hashing alone does not preserve correctness.

Use the Interview Playbook for the shared 45-minute sequence and fresher, intermediate, senior and staff evidence. Start with one complete operation. Extend the answer with the failure/recovery traces below, then defend a workload-specific alternative and a measurable rollout. Scope and evidence matter more than a large component list.

Reveal the full worked solution

Save your attempt before continuing.

Distributed Cache: Understand the building blocks

Choose a mechanism because it satisfies a requirement. These explanations describe the exercise design, not undisclosed implementation choices at the named company.

Distributed Cache: Quick vocabulary

TermMeaning and role in this exercise
API: Application Programming InterfaceThe contract for requests, responses and errors between components. An API route does not itself enforce every business invariant.
CPU: Central Processing UnitGeneral-purpose compute whose time/concurrency must be bounded for judging, processing or encoding.
LRU: Least Recently UsedAn eviction policy favoring recently accessed entries. Exact shared ordering may add contention; approximation changes policy precision.
LFU: Least Frequently UsedAn eviction policy favoring frequently accessed entries. Counting/aging must be bounded and adapt to changing popularity.

Distributed Cache: Cache-aside, Redis and time to live (TTL)

What it means: A cache keeps reusable copies for faster reads. Cache-aside checks the cache, loads an authoritative source on a miss, then fills it. TTL is the declared entry lifetime.

Why this problem needs it: The service speeds reads while the catalog owns durable business facts.

How to use it: Key by the correct tenant/object/version, retain an absolute freshness deadline, bound entry size, and apply the stated invalidation or version-fence policy. Redis is one possible in-memory implementation.

Example in this design: An expired key is a miss even before physical cleanup.

When it fits: Useful when repeated eligible reads justify memory and the tolerated staleness is explicit.

Alternative and limits: Direct indexed database reads may suffice. A cache is not automatically durable authority; shorter TTL does not prove immediate revocation.

Distributed Cache: Consistent hashing and virtual nodes

What it means: Consistent hashing maps keys to owners around a stable hash ring, so membership changes move selected ranges. Virtual nodes assign multiple positions to each physical node.

Why this problem needs it: Membership changes should move a selected ownership range.

How to use it: Hash the key, choose the declared successor owner, and migrate the affected range under a versioned ownership authority.

Example in this design: Adding token 50 moves key hash 45 to the new successor owner.

When it fits: Useful when distributed key placement should avoid remapping nearly every key after a node change.

Alternative and limits: A range/partition directory may fit ordered access better. Balanced key count does not guarantee balanced bytes or traffic, and hashing is not a consensus protocol.

Distributed Cache: Lease, ownership epoch and fencing

What it means: A lease grants temporary work ownership. A fencing token or epoch identifies a newer owner so stale owners can be rejected at the mutation boundary.

Why this problem needs it: A partitioned former owner must not mutate after cutover.

How to use it: Claim a job under an atomic authority, increment its attempt/epoch, and require that value on result publication. The protected resource must enforce the check.

Example in this design: Owner epoch 11 fails after topology epoch 12 is active.

When it fits: Useful when workers can pause, restart or overlap after a timeout.

Alternative and limits: A non-expiring lock risks blocking forever. Lease expiry alone cannot stop an old process that resumes; external effects need their own supported idempotency/reconciliation.

Distributed Cache: Compare-and-set (CAS) and optimistic concurrency

What it means: A conditional write changes state only if an expected version or value still matches. The comparison and write share one atomic boundary.

Why this problem needs it: Content freshness is independent of placement correctness.

How to use it: Carry the version read by the caller; atomically compare, mutate and advance it. On conflict, re-read or reject under the business policy.

Example in this design: Reject source revision 7 after a known revision 8 even on a correctly routed request.

When it fits: Useful when concurrent requests must not silently overwrite the same owned state.

Alternative and limits: Row locks or a serial owner can simplify highly contended updates. A read followed by an unconditional write is not CAS.

Product contract and scope

A catalog API reads the same product description thousands of times. The cache keeps a bounded, regional copy near readers, expires it after a declared freshness interval, and falls back to the authoritative catalog when a value is absent. When one cache node fails, the system must not turn every miss into an unbounded database request.

This design is a disposable key/value cache. The catalog or other source of truth owns durable business state. A successful cache write is not a durable origin commit. We provide GET, SET with TTL, DELETE, bounded MGET, and optional compare-and-set. We explicitly choose what the caller may observe during a miss, a replica promotion, and an origin outage. Caching an authorization decision is a different problem and is excluded from shared-key negative caching.

RequirementContractEvidence
R1 RoutingEvery request names a topology epoch and routes to the owner under that epoch.V1–V2
R2 Entry lifecycleByte limits, allocation classes, TTL, eviction, and atomic key operations have stated behavior.V3–V5
R3 MigrationMembership has one authority; copies are version-checked; old owners are fenced after cutover.V6–V7
R4 FailoverReplica staleness policy is explicit; promotion cannot silently claim durability.V8
R5 RefillSame-key misses coalesce; global origin work is bounded; failures can shed load.V9–V10
R6 Negative entriesOnly short-lived absence is cached; permission failures and transient errors are not shared.V11

Access paths and data model

RecordIdentity and fieldsInvariant
Topologyepoch, token, owner node, replica set, membership stateCoordinator publishes monotonically increasing epochs. A request from an older epoch cannot mutate a shard after cutover.
CacheEntrykey, value, source_version, expires_at, allocation_class, last_accessOne entry per key at a shard. Logical expiry is checked on every GET even if physical reclamation is delayed.
VersionFencekey, highest observed source_version, retention policyAn older fill cannot replace a known newer source. Fence memory is bounded; authoritative origin validation remains necessary after fence eviction.
RefillBudgetregion, tenant/class, max concurrent reads, timeoutPer-key single-flight and a global budget both apply. A local process semaphore alone does not enforce a cluster-wide origin limit.
ReplicaAssignmenttoken, primary, replica, replication mode, freshness contractAsync replication can lose an acknowledged cache write. It does not promote a cache into a system of record.

Choose the partition and node layout from key count, value bytes, request heat, and failure reserve. A count-balanced hash ring does not imply byte balance: one key may hold a large value, and one viral key may account for most requests. Cap value bytes, MGET key count, request body size, and refill concurrency independently.

API contract

OperationBehavior
GET keyReturn value, source version, and remaining TTL if unexpired; otherwise miss. An expired value is a miss at its exact deadline.
SET key value TTL sourceVersion epochStore only if value fits and epoch is current. A lower source version is rejected; equal-version replay does not restart TTL.
CAS key expectedVersion newValue sourceVersion TTL epochOne atomic shard-owner comparison and write. A version mismatch returns conflict; it is not a distributed lock for a database transaction.
DELETE key sourceVersion TTL epochWrite a short versioned tombstone or invalidate with an origin fence so an older delayed fill cannot restore the value.
MGET keys epochBound the list length and response bytes. Preserve per-key hit/miss status rather than issuing an unbounded fan-out.

Capacity worksheet

Assume 100 million entries. If each averages 500 bytes of value, 50 bytes of key, and 100 bytes of metadata, logical memory is 650 bytes per entry, or 65 GB for one complete copy. Two copies use about 130 GB. Reserving 25% for allocator fragmentation, uneven shards, and failover headroom requires about 162.5 GB; three copies at the same headroom require about 243.75 GB. If 500 bytes already includes key and metadata, the corresponding one-copy baseline is 50 GB; state which size convention is used before capacity planning.

At one million requests/s, a 1% miss rate produces 10,000 origin reads/s. A 5% miss rate produces 50,000/s. If the source is safe for only 12,000 refill reads/s, the latter cannot be made safe by adding cache nodes alone: coalesce same-key misses, cap distinct-key refills, serve stale only where permitted, and reject excess requests explicitly. A local 512-request semaphore on each of 100 processes could admit 51,200 concurrent reads, so budgets must account for fleet size or use a shared admission mechanism.

One ten-minute TTL across a hot key can synchronize expiration. Apply bounded random jitter, for example ±10%, and coalesce one in-flight refresh per key. Jitter spreads expiry; it does not limit a cold-cluster refill storm. Watch both entry bytes and request volume, since a small viral key and many cold large values stress different resources.

Design step 1 — Define a cache node's memory and atomic operations

Each node stores a hash table entry with the key, bytes, source version, absolute expiration, and allocation metadata. A slab allocator places a 90-byte logical entry into the next 128-byte class; the unused 38 bytes are internal fragmentation. Report allocated bytes by class, not only payload bytes. Reject an item larger than a configured maximum before it can evict the whole working set.

For eviction, use approximate LRU or sampled LFU with bounded metadata work. Exact global LRU updates a shared ordering structure on every read and can create contention. Check TTL on GET and evict the expired entry immediately; a bounded background sweep can reclaim idle expired entries later. If memory pressure arrives first, evict expired entries encountered, then the policy's least valuable live entries. TTL controls logical validity; physical deletion is eventual.

CAS is atomic only at the current shard owner. If key k has source version 7, SET-if-version(expected=7,new=8) succeeds once; a competing expected=7 write sees version 8 and fails. This operation protects cache state. It cannot prove that an external business database committed the same version, so the origin remains authoritative.

A cache node accounts for value, key, metadata, size class, expiry, and atomic key operations.
Memory is charged by allocated class, while TTL validity is independent of physical reclamation.
Read the diagram

A cache node stores key and value bytes in a hash table, charges memory by allocator size class, and retains source version and expiry metadata. CAS is atomic at the shard owner. Every GET checks logical expiration; eviction and physical cleanup reclaim allocated bytes.

Open original
Cache node memory and key operations
A cache node accounts for value, key, metadata, size class, expiry, and atomic key operations.

Memory is charged by allocated class, while TTL validity is independent of physical reclamation.

Design step 2 — Route by ring ownership and migrate one interval

Use a stable hash of the key and choose the first token clockwise at or after that position, wrapping at the ring end. With tokens 20, 60, and 90, key hash 45 belongs to token 60. Add token 50: only keys in interval (20,50] move from owner 60 to the new owner. Key hash 45 moves; keys at 10 and 80 do not. Virtual nodes make many independently placed intervals, improving key-count balance, but do not divide one hot key.

Migration has an authority boundary. The membership coordinator first records a proposed epoch and the new owner. During copying, the old owner remains the only write authority; copy records with source versions and reject an older copy at the destination. A bounded change stream or a short write pause catches updates after the initial snapshot. At cutover, the coordinator publishes the new epoch and fences old-owner writes. Clients refresh the routing map and retry redirects only a bounded number of times. Since the cache is disposable, a missing migrated key can refill from the source; do not claim a copy is durable storage.

Before and after adding a ring token: one interval moves, and copied values retain source versions.
A new token moves only its predecessor interval; it does not rebalance every key.
Read the diagram

The ring has tokens A20, B60, and C90. Key hash 45 belongs to B60. Adding D50 moves only interval (20,50] to D, including key 45; other key ranges keep their owner. Virtual nodes improve placement balance but do not split one hot key.

Open original
Consistent hashing · one interval moves
Before and after adding a ring token: one interval moves, and copied values retain source versions.

A new token moves only its predecessor interval; it does not rebalance every key.

Design step 3 — Give membership and failover one authority

A consensus-backed coordinator or a single durable topology authority assigns increasing epochs. It grants one owner lease or fencing token per range. A partitioned client with epoch 7 cannot appoint itself owner after epoch 8 exists. The new owner rejects epoch-7 mutations even if an old router continues sending them. A lease without a fencing token is insufficient if a paused former owner can resume and write.

During rebalance, the topology can expose states such as stable, copying, and active. In copying, old owner accepts writes and streams changes. Before active, the coordinator fences the old lease and establishes a cutover watermark. The client either routes to the new epoch or receives a bounded retryable redirect. Simultaneous independent writes to old and new owners are not resolved by hashing; one write authority must win.

Membership epochs fence old owners before a new token becomes active.
Epoch authority and write fencing prevent two partitioned owners from claiming the same range.
Read the diagram

The topology coordinator keeps the old owner authoritative while a range snapshot and bounded change stream reach the new owner. It fences the old lease, publishes epoch 8, and routes new requests to Q. P rejects epoch 7 mutations after cutover.

Open original
Membership epoch and owner fencing
Membership epochs fence old owners before a new token becomes active.

Epoch authority and write fencing prevent two partitioned owners from claiming the same range.

An async replica can lag or omit a just-acknowledged cache write. On promotion, this design invalidates entries for the promoted range and refills from the source of truth. If an application chooses to serve replica data, it must state the maximum staleness and validate source versions where possible. Cache replication is a latency/availability choice, not durability.

Replica promotion advances the epoch, invalidates uncertain cached values, and fences the former primary.
Invalidate on promotion when the replica may have missed the last acknowledged cache update.
Read the diagram

An asynchronous replica may hold source version 17 when primary P acknowledged version 18 and then failed. The coordinator advances the topology epoch, fences P, and promotes R while invalidating uncertain range values. Misses refill from the authoritative source.

Open original
Replica promotion with invalidation
Replica promotion advances the epoch, invalidates uncertain cached values, and fences the former primary.

Invalidate on promotion when the replica may have missed the last acknowledged cache update.

Design step 4 — Protect the origin on cache misses

A cold miss checks the cache, joins a bounded single-flight for that key, checks again, then acquires a global refill permit. The origin call has a fixed timeout and bounded connection pool. A successful response carries a source version and an absolute freshness deadline derived from the read observation; a delayed response cannot restart its TTL when it finally arrives. The cache fills only if its source version is not older than the value or fence already known.

Refill pseudocode stepAction
1Read cache; return only a hit whose logical expiry and caller freshness rules pass.
2Join a bounded in-flight operation for this key, then check cache again after waiting.
3Acquire a global origin permit. If none is available, serve permitted stale data or return retryable 503.
4Read origin with a fixed timeout and capture source version plus observation time.
5For a value, fill until observation_time + positive_TTL. For not_found, fill a distinct marker until observation_time + shorter_negative_TTL. Reject a lower source version.
6For permission_denied or transient_error, return the result without caching it as absence.
7Release the origin permit and complete the shared future for waiters.

Negative caching is useful for repeated requests to a genuinely absent key, but it should have a short TTL and a distinct not-found marker. Do not cache permission denied under a shared key: one user's denial must not hide another user's allowed result. Do not cache timeouts or transient 5xx responses as absence. If a stale value is allowed during origin trouble, expose that policy and age; otherwise shed excess misses with a retryable error.

A hot-key miss creates one refill; duplicate callers wait and global admission bounds origin load.
Per-key coalescing reduces duplicate reads, while a fleet-wide budget caps distinct misses.
Read the diagram

Many callers miss the same hot key. Per-key single-flight joins their requests, a global origin permit bounds distinct misses, and one timed source read returns a versioned result. The cache conditionally fills and releases waiters; overload is shed under an explicit policy.

Open original
Hot-key refill coalescing and admission
A hot-key miss creates one refill; duplicate callers wait and global admission bounds origin load.

Per-key coalescing reduces duplicate reads, while a fleet-wide budget caps distinct misses.

When a cached value expires, its first reader becomes the refill leader; concurrent readers join it. If the origin budget is exhausted, return an allowed stale value only when the caller's freshness contract permits it, otherwise return 503 with retry guidance. Bound the number of in-flight distinct keys too, so an attacker cannot turn single-flight bookkeeping into an unbounded memory queue.

Failure verification trace

Source versions, tombstones, and absolute deadlines stop stale refills from restoring older data.
A late lower-version fill is rejected, and a delayed observation retains its original deadline.
Read the diagram

A cache read observes version 18 at second zero and receives an absolute freshness deadline at second 55. The source commits version 19 at second 1, but invalidation is delayed. When the old response arrives at second 60, its TTL cannot restart; reject it by version fence or re-read current origin state.

Open original
Late refill, version fence, and freshness deadline
Source versions, tombstones, and absolute deadlines stop stale refills from restoring older data.

A late lower-version fill is rejected, and a delayed observation retains its original deadline.

Start with a cache entry for key K at source version 18, observed at second 0, with TTL 55 seconds. A refill begun at second 0 returns at second 60. Its freshness deadline is still second 55, so the response must not be cached or served as a fresh hit at second 60. At second 1 the source commits version 19 and sends a tombstone/invalidation; a late version-18 fill is rejected by a version fence if that fence is present. If the fence was lost, the old absolute deadline still prevents extending the stale response beyond second 55. After that deadline, read current origin state or return retryable failure.

CaseExpected resultModel/test evidence
V1 Ring boundaryFirst token clockwise owns exact boundary and wraparound keys.test_ring_successor_boundaries_and_add_node_moves_only_interval
V2 Membership changeOnly the changed interval moves; old epoch reads/writes fail.test_old_topology_epoch_is_fenced_after_membership_change
V3 TTLExpiry is a miss at the exact deadline.test_ttl_expires_at_deadline_without_waiting_for_physical_cleanup
V4 MemorySize classes and eviction keep bytes under the node limit.test_lru_and_size_classes_bound_node_memory
V5 CASOne expected-version update succeeds; retries with old expected version conflict.test_cas_is_atomic_and_requires_expected_source_version
V6 MigrationOlder copies cannot replace newer source versions or extend expiry.test_migration_copy_never_replaces_newer_source_version_or_extends_ttl
V7 Epoch raceAn origin fill started under an old epoch cannot write after topology cutover.test_topology_change_during_origin_read_fences_old_epoch_fill
V8 PromotionStale replica values are invalidated and old primary writes are fenced.test_replica_promotion_invalidates_stale_copy_and_fences_old_writer
V9 Same-key stampedeConcurrent identical misses share one origin read.test_same_key_misses_share_one_origin_read
V10 Origin overloadDistinct misses beyond the global permit budget fail fast.test_global_origin_budget_sheds_distinct_misses
V11 Negative entryAbsence is short-lived; authorization denial is not cached.test_negative_cache_is_short_and_authorization_denials_are_not_cached

Final architecture and requirement trace

The client routes with an epoch to a single shard owner. The owner applies entry size, TTL, CAS, and eviction rules. Membership authority migrates ranges and advances epochs; replicas are optional and their staleness contract is explicit. Misses flow through per-key coalescing and global refill admission to the authoritative origin. A cache outage degrades read performance and can cause controlled misses, but it never changes business ownership.

RequirementDesign ownerObservable evidence
R1 RoutingClient ring and epoch coordinatorstale epoch rejects, route redirects, key movement per change
R2 LifecycleCache node allocator and entry policybytes by class, fragmentation, TTL misses, evictions, CAS conflicts
R3 MigrationCoordinator and range copiercopy lag, moved keys/bytes, old-owner fence rejects
R4 FailoverReplica policy and source validationpromotion epoch, invalidations, stale-version rejects
R5 RefillSingle-flight and global budgetin-flight distinct keys, coalesced callers, origin reads/s, 503 rate
R6 Negative cacheEntry kind and short TTLnegative-hit ratio, permission denial cache count (must be zero)

Operate and recover

Measure hit ratio by request and bytes, p50/p95/p99 hit latency, entry count and allocated bytes by slab class, internal fragmentation, evictions by reason, expired-on-read count, CAS conflicts, replication lag, promotion count, keys moved per epoch, route retries, single-flight waiters, refill concurrency, origin timeout rate, and stale-read age. Alert on origin fallback amplification and the oldest migration, not just cache CPU.

After a node restart, admit refills gradually, warm only measured hot keys, and keep origin concurrency bounded. During migration, pause cutover if copy lag or old-owner fencing is unhealthy. After a bad topology publish, advance to a new epoch; never reuse an old epoch. Since values are disposable, rebuild from the source rather than attempting to treat volatile cache snapshots as a durable database backup.

Practice and review

Prompt: A node owns 10% of keys but serves 60% of requests because key K is viral. At the same time, a shard loss raises miss rate from 1% to 5% at one million requests/s, while the origin is safe for 12,000 reads/s. Design the immediate response, then explain how ring migration, replica promotion, and TTL interact.

Failure follow-up: A source read for version 18 starts at second 0 with a 55-second freshness window, version 19 commits at second 1, and the old response arrives at second 60. Can a cache set its TTL to 55 seconds from arrival?

Review rubric: Full credit separates key balance from request heat; calculates 10,000 versus 50,000 origin reads/s; coalesces the viral key and bounds distinct fills; uses one epoch authority and fences the old owner; moves only the ring interval; compares source versions during migration/CAS; checks TTL logically at the deadline; and either invalidates a stale promoted replica or states a concrete bounded-staleness contract. The model tests these selected state and admission rules, not production benchmark throughput or consensus availability.

Operational contract

AreaDecision and signals
SLIsHit ratio; origin miss rate; stale-fill rejects; membership migration lag; replica promotion age.
Overload limitsBound per-key singleflight, global origin permits, item size/TTL, hot-key replicas, and migration backlog.
RecoveryFence writes by topology epoch; invalidate on promotion; refill from the authoritative origin and reconcile copied ranges.
Access controlsAuthenticate topology changes; isolate tenants/namespaces and protect cached sensitive values with explicit TTL/access policy.
Dominant costCache memory/replicas, origin reads during misses, and cross-node replication.

Version placement and cached content independently

At ring version 1, key 45 belongs to B. Adding token D=50 at ring version 2 moves the applicable interval, so key 45 belongs to D. During movement, a source read of content version 7 may return after version 8 invalidation. Ring version identifies placement; content version identifies freshness. Neither comparison substitutes for the other.

Contract / pseudocode
ring v1: key45 -> B; value v7
ring v2: key45 -> D
source/current value: v8
late fill carrying value v7: reject under version-aware protocol
ring v1: key45 -> B; value v7; ring v2: key45 -> D; source/current value: v8; late fill carrying value v7: reject under version-aware protocol
Version placement and cached content independently
Read the diagram

ring v1: key45 -> B; value v7; ring v2: key45 -> D; source/current value: v8; late fill carrying value v7: reject under version-aware protocol

Open original
Version placement and cached content independently
ring v1: key45 -> B; value v7; ring v2: key45 -> D; source/current value: v8; late fill carrying value v7: reject under version-aware protocol

Version placement and cached content independently

Declare whether data loss simply causes a miss or violates the service's durability promise. TTL, eviction and replicated read copies affect different conditions. A hot read can be replicated/coalesced; a hot write cannot be independently ordered by arbitrary replicas without a specified consistency protocol. Membership changes need bounded origin fallback and cleanup; otherwise a cache reconfiguration can overload the authoritative database.

Show answer and explanation

Answer: A valid ring-v2 route can still return stale value v7. A current value at a stale owner can still accept an invalid ownership write. The diagram and contract keep both versions visible.

Decision review: Distributed Cache

Distributed Cache: What it is

A distributed cache places reusable copies across nodes while defining freshness, eviction and behavior when placement changes.

Distributed Cache: Why it is needed

Hot reads may exceed origin capacity, but cache membership and source-content revisions are independent kinds of state.

Distributed Cache: When to use it

Use it when repeat access and acceptable freshness justify its cost; keep authoritative business rules outside disposable copies unless a stronger contract is explicitly implemented.

Distributed Cache: How to use it

Route through a versioned placement map, bound each node’s memory and protect the origin on misses. During movement, define which epochs may write and how a stale source fill is prevented from replacing a newer value.

Distributed Cache: Compare an alternative

An application-local cache avoids distributed routing but repeats storage per process and can increase invalidation work. A shared cache improves reuse while introducing another failure and coordination boundary.

Distributed Cache: Worked decision exercise

The following inputs and policies are original exercise assumptions; they are not measurements or undisclosed company behavior.

Placement epoch 12 assigns key k7 to B, while source revision 8 is its newest content. A client with epoch 11 targets A and a delayed fill carries revision 7. Accepting either as current without checking its relevant version mixes placement correctness with content freshness.

Distributed Cache: Placement epoch 12 owns key k7 at B; Source content revision 8 is current; Old route epoch 11 and late revision 7 differ; Check the relevant version; protect origin on miss
Distributed Cache: a concrete state and recovery trace under the exercise assumptions.
Read the diagram

Placement epoch 12 owns key k7 at B; Source content revision 8 is current; Old route epoch 11 and late revision 7 differ; Check the relevant version; protect origin on miss

Open original
Distributed Cache: worked example
Distributed Cache: Placement epoch 12 owns key k7 at B; Source content revision 8 is current; Old route epoch 11 and late revision 7 differ; Check the relevant version; protect origin on miss

Distributed Cache: a concrete state and recovery trace under the exercise assumptions.

Does adding more ring tokens fix a single key that produces half the traffic?

Show answer and explanation

Answer: No: distributing other keys does not split that hot key. Replicate its reads or change the representation if semantics permit, while bounding origin misses and respecting freshness. Rebalancing and content invalidation require separate protocols.

Distributed Cache: Check the design

Measure hot-key request volume, origin amplification during outage, eviction rate and stale-map rejection. Test a node loss while clients hold an old placement epoch.

30:00Self-guided practice timer
The timer resets when you leave this page. Save your design separately.
Your challenge

A cache node fails and its misses overwhelm the database. Which changes reduce the immediate blast radius?

Your design draft

Clarify assumptions, explain your approach, and test the difficult cases. Save your draft, then compare it with the study notes.

Open solution review

Self-review checklist

Self-guided practice. Automated AI feedback and code execution are not connected.