2. Prediction, label and dataset relationships

Prerequisites and scoped previews

Frame the decision and non-ML baseline

Define user action, prediction unit, target horizon, product outcome and guardrails.

What, when, why and how

Event identity and idempotency

what: An event is a fact with a stable producer identity. Retries can redeliver the same fact; an event ID identifies that fact, not its delivery attempt.
when: Use this at ingestion, stream processing, offline replay and label construction.
why: Counting retries as new behavior inflates aggregates, changes features and can create duplicate training examples.
how: Use a producer-scoped event key and validate payload consistency. The same key and payload is a retry; the same key with conflicting content is a contract violation. Quarantine or explicitly revise it rather than silently choosing the last delivery.

Event time versus availability

what: Event time is when a fact occurred. Ingestion time is when a consumer received it. Available time is when a validated value could be read by the decision path.
when: Use these for historical features, late events, backfills and training/serving comparisons.
why: A fact can occur before a prediction but arrive too late to be a valid historical input. An event-time-only join leaks future knowledge.
how: Persist relevant timestamps and versions. For this exercise, require both event_time <= decision_time and available_at <= decision_time. Apply a TTL/window and choose the latest eligible event with a deterministic tie-break.

Relationships and immutable revisions

what: A prediction references a decision entity; later labels reference an attributed prediction or outcome. An immutable revision records a correction without erasing earlier knowledge.
when: Use these when outcomes arrive late, policies evolve or datasets are rebuilt.
why: Joining mutable latest rows cannot explain which truth and features were used by a past training run.
how: Use tenant-qualified keys, explicit attribution, immutable dataset/run IDs and revision timestamps. Enforce same-tenant references and define deletion/retention propagation across derived artifacts.

Specify the logical schema

This is an educational logical schema, not a deployed database migration. UUID-like strings below are synthetic. Use composite keys where identity is tenant-local; a database FK alone does not enforce temporal availability.

Entity / primary key	Important fields	Relationship / invariant
Event (tenant_id, event_id)	entity_id, event_time, ingested_at, available_at, schema_version, payload_hash	Same event ID + different payload is rejected.
FeatureRevision (tenant_id, revision_id)	entity_id, feature_name, event_time, available_at, value, transform_version	One entity/feature has many immutable revisions; examples pin the selected revision IDs.
Prediction (tenant_id, prediction_id)	entity_id, decision_time, feature_snapshot_id, bundle_version, action	One entity can have many predictions; each prediction has one bundle.
LabelRevision (tenant_id, prediction_id, revision)	label, observed_at, available_at, policy_version, status	One prediction has zero or more label revisions; missing is not negative.
DatasetExample (snapshot_id, prediction_id)	feature_revision_ids, label_revision_id, split	One snapshot has many examples; reference the exact feature and label revisions.

Replay a decision with late data

At 10:00 UTC, prediction P1 scores account A. Feature F1 happened at 09:00 and was available at 09:01. F2 happened at 09:30 but was available only at 10:10. F3 happened at 10:05 and was available at 10:06. F1 is eligible. F2 is excluded despite its earlier event time. F3 is a future event.

At 10:15, a new prediction P2 can use F2 if its feature window/TTL permits. Rebuilding P1 tomorrow must still exclude F2 when the goal is serving-state replay. A corrected-history analytical dataset can intentionally include later revisions, but must be labeled differently and must not claim to reproduce the original serving state.

If F1 is delivered twice with the same identity and content, the aggregate counts it once. If the second payload changes its amount, fail validation and record a conflict. A legitimate correction needs an explicit revision and availability rule.

SELECT p.prediction_id, f.revision_id
FROM predictions p
LEFT JOIN feature_revisions f
  ON f.tenant_id = p.tenant_id
 AND f.entity_id = p.entity_id
 AND f.event_time <= p.decision_time
 AND f.available_at <= p.decision_time;
-- Apply the feature window, then select latest eligible revision
-- per prediction + feature with a stable tie-break.
-- This conceptual join alone can return multiple candidates.

Handle retries, backfills and permissions

A late batch cannot move available_at backward merely to make offline quality look better. Preserve when the online system could read the value. If the online materialization time was not recorded, disclose the replay limitation instead of asserting exact parity.

Entity IDs must not join across tenants. Treat an event timestamp with a missing timezone, non-finite numeric values or an unsupported schema as a contract violation. Quarantine, alert an owner and maintain a measurable rejected-event count; silently dropping rows hides denominator changes.

A deletion request affects raw events, feature snapshots, indexes and retained examples under the declared process. Deleting one row does not prove that all trained influence disappeared. State rebuild/retraining boundaries and retention obligations rather than promising model unlearning.

Choose storage by required reconstruction

A current-value key/value store can serve quickly but cannot by itself reconstruct older values. An append-only revision log supports replay at additional storage and query cost. Keep the serving store simple and derive a versioned historical store when the use case needs both.

A point-in-time feature retrieval tool can implement parts of this workflow; verify its timestamp/TTL semantics against your own availability contract. A library name is not evidence that late-data and revision behavior are correct.

Diagrams and relationships

2. Prediction, label and dataset relationships

Entity IDs are tenant-qualified. A prediction may have no mature label; dataset examples pin exact revisions.

Scroll horizontally on smaller screens, or enlarge the diagram.

Engineer, Senior and Staff follow-ups

These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.

Engineer: Which feature is valid for P1: earlier event time or earlier availability?

Both constraints apply. F2 occurred before P1 but was unavailable until 10:10, so only F1 can be used at 10:00 under this replay contract.

Engineer: Why is prediction_id different from event_id?

One event may trigger several predictions under different policies or times. Separate identities preserve which decision used which inputs and avoid treating every rescore as a new source event.

Senior: How do you handle an out-of-order correction?

Append the correction with a new revision identity and its true availability. Rebuild impacted derived artifacts using a declared snapshot cutoff; reject accidental replacement of an immutable dataset. Compare replay semantics with current-corrected analytics explicitly.

Senior: How would you detect training/serving mismatch?

Use a small set of prediction IDs with retained permitted feature snapshots and bundle versions. Compare eligible revision IDs and transform outputs before comparing scores. A score difference can come from input/transform/version mismatch rather than model drift.

Staff: Who owns schema evolution?

The producer owns the event semantics, consumers declare supported versions, and the platform enforces compatibility and quarantine. Roll out additive readers before writers, preserve old snapshot interpretation, and negotiate breaking changes with migration and rollback owners.

Staff: Can a common event schema solve all label attribution?

No. A purchase may follow several exposures; each product must declare attribution windows and ambiguity rules. A shared identity envelope helps traceability, while label meaning and evaluation ownership remain explicit domain contracts.

Practice and self-review

Your practice draft

Replay P1 at 10:00 with the three features above, a duplicate F1 and a conflicting F1 payload. Define output, rejected records and the dataset keys.

Explain your assumptions, calculation, alternatives and one failure.

Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.

Progressive hints

Hint 1

Check entity and tenant before time.

Hint 2

Apply event and availability cutoffs, not arrival ordering.

Hint 3

Separate a delivery retry from an explicit correction.

Reveal worked answer guidance

P1 selects F1 once. F2 and F3 are excluded for availability/future-time reasons. An identical duplicate is ignored; a changed payload under the same event key is quarantined or rejected. The dataset records snapshot_id, prediction_id, the exact eligible feature revision and an eligible label revision if one exists. Pending labels are retained as pending rather than silently converted to zero.

Evidence to check in your answer

Stable identity and conflict behavior.
No future/unavailable inputs.
Deterministic latest-eligible tie-break.
Explicit tenant and label revision boundaries.

For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.

Sources and verification scope

Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.

Feast point-in-time joins ↗ · Historical feature retrieval reference; actual availability/revision contracts still need explicit modeling.

Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.

Events, identity and data contracts