Prerequisites and scoped previews
Events, identity and data contracts
Define keys, event/ingestion/availability times, schema evolution, dedupe and permitted use.
Labels, delayed outcomes and human annotation
Define label windows, maturity, censoring, annotation policy and disagreement.
Reproducibility and experiment tracking
Record data/code/config/environment/seeds/artifacts and distinguish replayability from bitwise determinism.
Scoped prerequisite preview. The complete chapter is planned; this preview does not count as authored coverage.
Registry, validation, rollout and rollback
Promote compatible artifact bundles through checks, shadow/canary and rollback gates.
Scoped prerequisite preview. The complete chapter is planned; this preview does not count as authored coverage.
What, when, why and how
Tenant and purpose-bound identity
- what
- A data key includes tenant/entity identity and a permitted use, with authorization established by trusted service context rather than caller-supplied labels.
- when
- Use this at ingestion, feature lookup, snapshots, caches, jobs and model consumption.
- why
- Tenant prefixes prevent some collisions but are not authorization. A shared feature store can still leak keys, values or resources through incorrect access or caching.
- how
- Authorize tenant/purpose before reading values, include scope in cache/registry/job keys and audit allowed/denied operations without exposing raw data. Validate multi-tenant isolation and noisy-neighbor limits independently.
Lineage and immutable evidence
- what
- Lineage maps sources, definitions, transformations, job runs, materialized revisions, snapshots and model consumption. Immutable references preserve which evidence produced an output.
- when
- Use this for incident diagnosis, schema changes, deletion impact and reproducible dataset generation.
- why
- A diagram of intended dependencies does not prove a particular run used them. A run marked complete does not establish quality or permission.
- how
- Record actual input/output versions, checkpoints, transform/config identity and run status, then verify lineage completeness. Separate design metadata from runtime observations and record reconstruction/retention limits.
Deletion barriers and lifecycle generations
- what
- A tombstone can revoke access and advance an entity generation so that delayed old-generation jobs cannot recreate deleted online values.
- when
- Use this when events/backfills can arrive after deletion, consent revocation or identity recreation.
- why
- Deleting one row may be undone by a retry; deleting current storage does not automatically remove snapshots, exports, caches or trained influence.
- how
- Check authoritative lifecycle generation on reads and writes, invalidate derived serving paths and track fan-out/rebuild acknowledgements. Define policy-specific retention/access and retraining/unlearning limits without promising universal erasure.
Fence a late backfill after deletion
Entity T1/U1 has generation 1 and value F2. A deletion request marks access revoked and advances the authoritative lifecycle generation to 2. An old generation-1 materializer/backfill completion is rejected even if its timestamp or local checkpoint looks newer.
The serving lookup returns deleted/no usable value while revocation is active. If the identity is later legitimately recreated under generation 2, old generation-1 values remain ineligible; new data must satisfy its new authority/contract. The fixture demonstrates generation fencing, not a distributed deletion service.
A historical snapshot from generation 1 does not become an authorization bypass. Access checks at export/read time still apply, and any retained minimal audit evidence must follow the declared policy rather than exposing deleted raw values.
Map access and deletion propagation
Track sources, historical feature ledger, online projections, cache entries, dataset examples/exports, lineage records and downstream model/index consumers. Assign each an owner, acknowledgement and limit. A deletion job is incomplete if one serving cache still returns revoked values.
Some immutable artifact formats require rewriting manifests/partitions or replacing accessible snapshot references rather than in-place row deletion. Some retained audit records require redaction or restricted access; define the actual process. This curriculum states engineering boundaries and no jurisdiction-specific legal deadline.
| Artifact | Action to define | Evidence / limit |
|---|---|---|
| Online values/cache | Deny/revoke and invalidate under lifecycle generation | No old-generation resurrection. |
| Offline/export snapshots | Locate, restrict/redact/rebuild as supported | Immutable references and completion acknowledgements. |
| Lineage/audit | Retain only permitted minimum or redact | Metadata itself can be sensitive. |
| Models/derived influence | Assess retraining/replacement process and scope | Row deletion is not proof of universal unlearning. |
Separate policy, platform and consumer responsibilities
Data/domain owners define semantics and permitted purpose; platform owns enforcement/lineage/materialization propagation; consumers own suitability of model/action use. Security/privacy/domain reviewers contribute independent checks where appropriate; an inventory count does not certify governance.
Use admission/quotas/resource isolation to protect tenants as well as access controls. An authorized backfill can still overload a shared store. Track per-tenant work/latency/cost while avoiding high-cardinality or sensitive labels in routine metrics.
Test what a lineage system cannot guarantee
A lineage catalog correctly lists every dataset, but an exporter omits authorization and serves an old snapshot after deletion. The metadata is useful for diagnosis but did not enforce the access boundary. Put the guard in the read/export path and validate it.
An old checkpoint resumes after generation change and attempts to publish a value. Require current generation and target-version validation at publication, not only at job creation. Record rejection and remediation without moving historical availability backward or silently rewriting evidence.
Diagrams and relationships
Current authority applies to old snapshots and queued work. The fixture rejects old-generation writes rather than claiming model unlearning.
Scroll horizontally on smaller screens, or enlarge the diagram.
Engineer, Senior and Staff follow-ups
These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.
Engineer: Why is tenant_id in a key insufficient?
It identifies scope but does not establish authorization. The trusted caller/purpose must be checked before reads, exports and writes, including caches and jobs.
Engineer: What happens to a generation-1 retry after the barrier moves to 2?
It is rejected at publication and cannot recreate the old serving value. Reads also enforce current access/generation; the tombstone is not just a background cleanup hint.
Senior: How do you know deletion propagation finished?
Track the declared artifact fan-out, owned acknowledgements, cache/serving checks and unresolved failures. Distinguish online access containment from slower offline rebuild and downstream influence limits.
Senior: Does lineage make a snapshot reproducible?
It supplies references and provenance, but source availability, permissions, transform/runtime compatibility and retained artifacts still matter. Record missing inputs and reproducibility limits rather than relying only on a graph.
Staff: How do you define a shared governance contract?
Standardize tenant/purpose identity, lifecycle generations, lineage, retention/access obligations and evidence reporting with domain owners. Permit domain-specific constraints and independent review; do not infer compliance from adopting a catalog or framework.
Staff: How do you handle expensive derived-artifact removal?
Contain access promptly under the declared policy, locate affected artifacts, plan supported rebuild/replacement with owners and expose unresolved limits. Negotiate costs and retention in advance; never equate deleting a source row with guaranteed removal of every learned effect.
Practice and self-review
Your practice draft
Trace a deletion through online values, backfills, historical export and model consumers. Explain how a delayed checkpoint is fenced and what remains unproven.
Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.
Progressive hints
Hint 1
Use a trusted current authority and generation, not only a timestamp.
Hint 2
List owned acknowledgements for every derived path.
Hint 3
Separate access containment, physical rewrite and model-influence claims.
Reveal worked answer guidance
Advance lifecycle authority, deny revoked reads, reject old-generation publications and invalidate serving/cache references. Use lineage to locate snapshots/exports/downstream consumers and track supported remediation with evidence. Retained metadata and model influence need explicit policies and limitations; the fixture proves only its bounded guards.
Evidence to check in your answer
- Trusted tenant/purpose authorization.
- Current-generation fencing at publish/read.
- Actual lineage and propagation ownership.
- Honest retention and influence limits.
For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.
Sources and verification scope
Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.
NIST AI Risk Management Framework ↗ · Risk vocabulary and governance; not a certification or blanket legal-compliance guarantee.
OpenLineage object model ↗ · Job/run/dataset and observed lineage vocabulary only. No OpenLineage SDK, authorization or deletion guarantee is implemented.
Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.