Redis

Redis offers in-memory data structures that can make shared counters, caches, rankings, and short-lived coordination convenient. Start by deciding whether the value is disposable or authoritative. A cache and a payment record have very different loss tolerances. Successful access to a fast data structure does not by itself establish a durable business guarantee.

Learning goals

Key/value and useful data structures; TTL; eviction; memory budget; atomic operations; persistence options; replication caveats; hot keys.

The mechanism at a glance

Figure — Match result → Durable record (commit); Durable record → Projection consumer (event); Projection consumer → Redis sorted set (score update); Top-K query → Redis sorted set (rank range); Durable record → Rebuild path (replay); Rebuild path → Redis sorted set (restore)

The numbered components identify responsibilities. Follow the labeled arrows rather than treating the numbers as a global execution order. The scenario later in this lesson shows one concrete sequence.

Step-by-step reasoning

1. Choose a data structure for an operation

Use strings for cached serialized values, hashes for bounded fields, sets for membership, and sorted sets for score-ordered members. A leaderboard updates a member’s score and retrieves a rank range without sorting every score in application code. Keep values and collection sizes bounded; a giant key can monopolize work or complicate eviction.

2. Make multi-step operations atomic when needed

A read followed by a write from the client can race with another client. Prefer an atomic command or a small server-side operation for the required transition. Atomic execution on one server is not the same as durable replication, and cross-shard operations have additional limitations. Keep scripts short so one operation does not block unrelated work.

3. Budget memory and eviction

Estimate keys, values, metadata, and replication overhead rather than counting payload bytes alone. Choose an eviction policy appropriate to the workload and set explicit expiration where useful. Eviction means a value may disappear under pressure. Do not mix critical non-evictable state with a disposable cache without carefully defined resource isolation.

4. Plan persistence and failure

Persistence and replication settings determine recovery and potential loss windows; verify them for the selected deployment. A cache outage should trigger bounded fallback to its source. A hot key remains hot even if many unrelated keys are well distributed. Consider application-local copies, sharded counters, or a different read model when the operation allows it.

Contracts and state

The following sketch makes the decision boundary concrete. Field names and capacity assumptions are illustrative; adapt them to the stated product contract.

Leaderboard sketch
ZADD leaderboard:season42 <score> <player_id>
ZREVRANGE leaderboard:season42 0 9 WITHSCORES

Authoritative match results -> durable store
Leaderboard -> rebuildable projection
Do not award irreversible prizes from an unverified cache alone.

Worked example

A game records match results durably and asynchronously updates a Redis sorted-set leaderboard. If Redis is lost, replay results or rebuild scores. Display a freshness marker during recovery and bound leaderboard queries. If a prize requires the exact final ranking, freeze the scoring window and verify the authoritative totals before awarding it. The same data structure can support a casual display and a stricter final decision through different contracts.

Failure walkthrough

Memory reaches the configured limit and hot entries begin to evict. The hit rate falls, causing more database reads, which slow fills and increase concurrent requests. Break this feedback loop with bounded fills and origin admission control. Watch memory pressure, eviction counts, latency, and key-size distribution; a high average hit ratio can hide a damaging single-key workload.

Figure — Memory pressure rises → Disposable key evicted → Reader misses cache → Bounded origin fetch → Repopulate with expiration

Decisions and trade-offs

Check your understanding

A leaderboard write succeeds immediately before the node fails. Can you promise the score survives?

Transfer to a new scenario

Build a rolling leaderboard with sorted sets and discuss where authoritative scores live.

Does a successful cache write automatically establish durable business state?

Primary documentation

Read the first-party engineering account or official technical reference. Company engineering posts describe the scope and date of that publication; the interview reconstruction and scenarios here are original teaching examples.

Match a data structure to an operation

Strings support counters and serialized values. Hashes group fields. Sets provide membership. Sorted sets attach scores for rankings and time-ordered work. Lists support simple ordered collections. Streams retain an append-style history with consumer-group state. The choice determines command cost and key locality; Redis is not a substitute for arbitrary relational queries.

SET profile:42 payload EX 60
INCR visits:2026-10-08
SADD followers:42 user:9
ZADD contest:7 1200 user:9
ZRANGE contest:7 0 9 REV WITHSCORES
XADD events:orders * kind order-created order_id 123

These examples illustrate operations; key names and retention policies must come from the workload. The * in XADD requests a generated entry ID.

Atomic commands are not an atomic workflow

A single command or supported server-side script can make a read-modify-write decision atomic on its execution boundary. Separate GET and SET calls cannot safely implement a concurrent token decrement. MULTI/EXEC groups commands, while WATCH can detect changes for optimistic retry; these mechanisms do not make calls to another database or payment provider part of the transaction.

For a rate limiter, store tokens and last-refill time together and calculate consumption atomically. Bound script runtime: long work delays other commands. Avoid commands that scan or return an unbounded dataset on a latency-sensitive instance.

Persistence and replication answer different questions

RDB snapshots and AOF logging provide different recovery windows and overhead. AOF fsync policy affects what a crash may lose. Asynchronous replication can leave a promoted replica behind the old primary. Neither the word “persistent” nor the existence of a replica proves that every acknowledged operation survives every failover.

For disposable cache entries, losing recent values may be acceptable. For inventory ownership or financial records, put the invariant in an appropriate authoritative store and specify its actual commit and failover guarantees. A distributed lock whose owner pauses past expiry also needs fencing at the resource it protects.

Cluster routing and key locality

Redis Cluster partitions keys into 16,384 hash slots. Clients route keys using slot ownership and respond to redirections when ownership changes. Multi-key operations generally require keys in the same slot. Hash tags can colocate related keys, but placing every tenant key in one tag may create a hot shard.

A cluster helps distribute many keys. It does not automatically split a single huge sorted set or one heavily accessed key. For a global leaderboard, consider per-contest keys or a bounded aggregation design and preserve an authoritative event source for rebuilding scores.

Pub/sub, streams and a queue contract

Pub/sub is useful for transient notifications to currently connected subscribers. It is not a durable replay log for disconnected consumers. Streams support retained entries and consumer processing state, but applications still handle pending messages, retries, poison work and idempotent side effects.

If a worker performs an external effect and crashes before acknowledgement, another worker may repeat it. Deduplication must protect the effect, not merely the message reception. Keep operational counters for pending age, retry count and terminal failures rather than using queue length alone.

Memory and failure exercise

Suppose Redis is configured to evict keys and holds both disposable profile caches and idempotency records for payments. What can go wrong?

Official references

Redis persistence, cluster specification and streams document the mechanisms and their limits.