Cassandra

Cassandra encourages query-first modeling: design tables around known access patterns and bounded partitions. It is useful for high-volume distributed workloads when those queries fit its model. Do not begin with a normalized relational schema and expect arbitrary joins to become cheap merely because the storage is distributed.

Learning goals

Query-first tables; partition and clustering keys; replication; consistency levels; hot partitions; time buckets; tombstones; limited transaction scope.

The mechanism at a glance

Figure — Conversation + bucket → Partition key hash (placement); Partition key hash → Replica set (replication); Replica set → Ordered message rows (clustering order); History reader → Ordered message rows (bounded range); Ordered message rows → Search projection (async indexing)

The numbered components identify responsibilities. Follow the labeled arrows rather than treating the numbers as a global execution order. The scenario later in this lesson shows one concrete sequence.

Step-by-step reasoning

1. Define the partition query

For conversation history, use conversation ID plus a time bucket as the partition key and time plus message ID as clustering columns. The partition key identifies where data lives; clustering order organizes rows within that partition. A request should normally know its partition and a bounded range.

2. Bound partition size and traffic

A global channel with all history in one partition grows without limit and can concentrate reads and writes. Time buckets bound historical size; an exceptionally busy current bucket may need an additional subdivision. Estimate both bytes per partition and operations per partition. More nodes do not fix an indivisible hot key.

3. Choose replication and consistency consciously

Replication places copies on several nodes. Read and write consistency levels determine how many acknowledgments an operation requires, with latency and availability implications. Do not infer every desired consistency guarantee from quorum arithmetic alone; concurrent updates, repair, timestamps, and the precise operation matter. Lightweight transactions serve particular conditional operations at added cost.

4. Account for deletion and repair

TTL and deletions create tombstone-related work rather than instantly removing every physical byte. Query patterns that scan many expired rows can become expensive. Keep retention aligned with table and bucket design, monitor tombstones and compaction, and operate repair according to the deployment’s requirements.

Contracts and state

The following sketch makes the decision boundary concrete. Field names and capacity assumptions are illustrative; adapt them to the stated product contract.

Illustrative table shape
PRIMARY KEY ((conversation_id, day_bucket), sent_at, message_id)
Query: latest messages in one conversation/day
Older history: walk earlier buckets with a bounded page size
Separate lookup table: message_id -> conversation/day if required

Worked example

A chat channel stores 100 messages/s at an illustrative 1 KB each. One day is roughly 8.64 GB before replication and overhead, so a daily bucket may still be too large for the operational target. Reduce the bucket duration or introduce a shard dimension while explaining how reads merge results. Bucket size is a capacity decision, not a universal daily convention.

Failure walkthrough

An application adds a new query to search messages across every conversation by a word. The existing partition key cannot answer that efficiently. Create an appropriate search projection rather than scanning all partitions on every request. The projection has its own freshness and deletion semantics, and the authoritative message record remains the final source for visibility checks.

Figure — Current bucket grows → Observe size and hot traffic → Create smaller future buckets → Reader traverses bucket boundaries → Expire old buckets deliberately

Decisions and trade-offs

Check your understanding

Why is a daily bucket not automatically small enough for every conversation?

Transfer to a new scenario

Store conversation messages by conversation and time bucket with a bounded partition size.

What can go wrong if every message in a huge channel shares an unbounded partition?

Primary documentation

Read the first-party engineering account or official technical reference. Company engineering posts describe the scope and date of that publication; the interview reconstruction and scenarios here are original teaching examples.

Design the table around a bounded query

A Cassandra table is designed for specific access patterns. For channel history, partition by channel and time bucket and cluster by message ID. A query for a different dimension generally needs another table or access system; arbitrary filtering does not become efficient merely because the cluster is distributed.

Writes pass through durable logging and memory structures before immutable disk tables and compaction. Reads may consult multiple structures, so compaction health and tombstone accumulation affect latency. A large or hot partition remains a problem even if total cluster capacity looks sufficient. Bucket size should reflect both bytes and query behavior.

Consistency settings control how many replicas participate, but the complete behavior also depends on replication topology, concurrent writes and repair. Lightweight transactions provide a conditional-update mechanism with different coordination cost from ordinary writes. Do not casually use timestamp-based last-write-wins as a substitute for business conflict resolution when clients have skewed clocks.

Figure — A decision worksheet for Cassandra: read the mechanism and its guarantee together.

Operational sketch

PRIMARY KEY ((channel_id,time_bucket), message_id)
query one bucket by message_id range
page into earlier buckets as needed
monitor partition bytes, tombstones and compaction lag

A tempting mistake

A TTL deletes data logically but cleanup and tombstones still create operational work. Reducing tombstone retention without understanding repair and failure windows risks resurrecting old data.

Transfer exercise

Why does adding nodes not necessarily fix one overloaded channel bucket?