Change Data Capture

Change data capture (CDC) publishes database changes for downstream projections and event consumers. Log-based CDC reads a database’s change log, often alongside an initial snapshot. The core design problem is not just “send every row”: define the snapshot boundary, ordering key, duplicate behavior, schema evolution, and what happens when a consumer is behind retention.

The mechanism at a glance

Figure — Source database → Snapshot at L (consistent boundary); Source database → CDC connector (transaction log); Snapshot at L → Projection consumer (bootstrap rows); CDC connector → Durable event log (ordered changes); Durable event log → Projection consumer (at-least-once delivery); Projection consumer → Read projection (idempotent state)

The numbered components identify responsibilities. Follow the labeled arrows rather than treating the numbers as a global execution order. The scenario later in this lesson shows one concrete sequence.

Step-by-step reasoning

1. 1 · Frame

Choose whether consumers need row changes, business events, or both. CDC captures committed row-level mutations; it may not explain the business intent behind them. Define tables, transaction boundaries, delete representation, ordering scope, and lag tolerance. Use an outbox when consumers need an explicit domain event committed with the write.

2. 2 · Model

A connector reads a consistent snapshot at log position L, then streams changes after L. Connector offsets track progress. Keys preserve entity identity; before/after images depend on source configuration. Consumers must checkpoint offsets and apply idempotent upserts or deduplicate event IDs. Schema changes require compatibility rules and a migration sequence.

3. 3 · Scale

Partition events by the entity key to preserve per-entity order. A single database transaction spanning entities may not be observed as one atomic downstream action unless the connector and consumer preserve transaction metadata. Snapshots and stream overlap must be reconciled so the same row may appear twice without changing final state.

4. 4 · Recover

If a consumer falls behind beyond log retention, recover from a fresh snapshot or an archived change log; do not silently skip the gap. Monitor source log retention, connector offset lag, snapshot progress, and schema errors. Tombstones and deletes need retention long enough for compacted topics or downstream replicas to observe removal.

Contracts and state

The following sketch makes the decision boundary concrete. Field names and capacity assumptions are illustrative; adapt them to the stated product contract.

Snapshot reads rows at boundary L
CDC event = {key, operation, source_position, schema_version, before?, after?}
Consumer applies idempotently; offset advances after durable sink write
Outbox event shares business transaction with source update

Worked example

A new search projection needs every existing product plus changes arriving during its build. Snapshot rows at boundary L, build the projection, and consume changes after L. If an overlapping row appears twice, an entity version makes the second application harmless. Once lag reaches zero and counts reconcile, route reads to the new projection.

Failure walkthrough

A consumer writes a row to its sink and crashes before committing its source offset. On restart it sees the event again. An idempotent upsert keyed by entity and source version prevents duplicate state; an external side effect needs its own idempotency key or outbox. If the offset advanced before the sink became durable, data could be lost.

Figure — Snapshot records boundary L → Rows load into projection → Changes after L are streamed → Crash replays one update → Entity version prevents double apply

Decisions and trade-offs

Check your understanding

Describe a snapshot-plus-stream bootstrap that tolerates a duplicate row and a consumer crash after writing but before offset commit.

Primary documentation

Read the first-party engineering account or official technical reference. Company engineering posts describe the scope and date of that publication; the interview reconstruction and scenarios here are original teaching examples.

Continue the connection

Study Kafka and explain which guarantee from this lesson carries into that topic.

Bridge snapshot and live changes

CDC exposes committed database changes for projections, search and integration. A snapshot provides existing rows, while a log position anchors the subsequent change stream. Taking an arbitrary snapshot and later starting a stream can miss changes in between; starting both without reconciliation can duplicate them. Use the connector’s supported snapshot protocol and retain source identity.

Apply downstream updates by source key and version or position. Deletes need tombstones or explicit delete events, and schema changes need compatibility handling. An at-least-once connector can replay after restart; the sink must not treat every delivery as a new business action. Capturing a row change also does not automatically explain the business intent behind it. An outbox can provide that semantic event.

Monitor source-log retention and connector lag. A stalled replication slot can retain logs and fill source storage; losing required history can force a new snapshot. Backfills and live updates need a version rule so old snapshot rows cannot overwrite newer streamed state. Protect the source from an aggressive recovery scan.

Figure — A decision worksheet for Change Data Capture: read the mechanism and its guarantee together.

Operational sketch

consistent snapshot + source position P
load rows into new projection generation
apply changes after P with version checks
validate counts and sampled content
switch readers; retain rollback generation

A tempting mistake

Database failover, schema evolution and log truncation are part of the design, not rare details to defer. A CDC consumer without deletion handling can permanently leak removed records.

Transfer exercise

Why might a business outbox be preferable to interpreting raw row changes?