Prerequisites and scoped previews
Classification and rare-event evaluation
Define confusion counts, precision/recall, PR/ROC metrics, slices and threshold trade-offs.
Calibration, uncertainty and decision thresholds
Distinguish score quality from calibrated probabilities, uncertainty and action capacity.
What, when, why and how
Slice definitions and populations
- what
- A slice is a declared subgroup of eligible examples defined using permitted, meaningful attributes or conditions such as language, modality, new users or model-missingness.
- when
- Use slices for diagnosis, launch guardrails and identifying gaps in label or service coverage.
- why
- A strong aggregate can be dominated by a large easy group. Post-hoc slicing until a desired result appears also creates selection and uncertainty problems.
- how
- Predefine important slices and intersection checks with owners, record denominators and missing attributes, and separate exploratory discoveries from confirmatory evidence. Use only permitted attributes; do not invent sensitive inferences.
Error costs and incompatible targets
- what
- False restrictions, missed harmful items, review delay and inconsistent appeals can affect groups differently. Different fairness/quality goals can conflict under different populations.
- when
- Use an explicit harm review when selecting thresholds, collecting labels or allocating reviewer expertise.
- why
- Equal aggregate accuracy says little about false-positive/false-negative rates or who waits longest. One parity measure does not establish fair treatment in every context.
- how
- Define actions and affected users, inspect error types with sample sizes, audit label-policy consistency and negotiate guardrails/escalation. Explain the chosen trade-off and what evidence could reverse it.
Small samples and incomplete labels
- what
- Sparse slices, selected review and noisy annotation limit what error rates mean. An undefined metric is a missing denominator, not proof of perfection.
- when
- Use this for rare harms, new languages, intersectional groups and small appeal cohorts.
- why
- A zero-error sample can still be consistent with nontrivial unknown risk. Treating uncertain labels as certain can create apparent disparities or hide real ones.
- how
- Report raw counts, observation design, maturity, disagreement and uncertainty. Gather appropriate additional evidence, use specialist review where justified and keep unsupported conclusions explicit.
Expose an aggregate that hides a cohort
Synthetic slice A has 990 examples with 980 correct; B has ten with five correct. Aggregate accuracy=(980+5)/1,000=98.5%, yet slice B accuracy=50%. These numbers illustrate weighting and small support, not a fairness assessment or a real demographic dataset.
For consequential classification, obtain TP/FP/FN/TN within each slice and label-policy consistency. Accuracy alone cannot tell whether B suffers false restrictions or missed harmful content. A cohort with zero actual positives has undefined recall; do not assign perfect recall.
Inspect service coverage as well: missing OCR, unavailable language model, delayed review or higher fallback frequency can create unequal outcomes before the classifier’s measured score. Report how slice membership and missing values were determined.
Conduct a harm-aware decision review
Start with a concrete action: temporary restriction while awaiting review. Identify mistaken-restriction cost, exposure risk while allowed, expected delay and appeal access. Decide which measures are hard limits and who can approve exceptions.
A language slice with low sample support may need targeted expert annotation and an explicitly conservative action path. Do not promise that adding one reviewer or a new threshold solves the problem; measure label consistency, backlog age, appeal outcomes and user impact under a justified plan.
| Evidence | Question to resolve | Possible next action |
|---|---|---|
| Raw counts and uncertainty | Is apparent regression supported? | Collect appropriate additional audit evidence. |
| Policy/annotation consistency | Are labels comparable across slices? | Revise guidelines/adjudicate ambiguous cases. |
| Missing modality/default rates | Is the service path different? | Repair preprocessing or use declared fallback. |
| Review/appeal age and access | Who waits and who can correct an error? | Capacity/expertise and accessibility changes with owners. |
Compare interventions instead of declaring parity
A single shared threshold, slice-specific recalibration, additional data or broader human review have different consequences. Slice-specific actions require appropriate permission, policy justification, governance and evaluation; do not introduce them automatically from a table.
A simpler model with reliable coverage can outperform a larger model that fails for key inputs. Compare actual available paths and mature outcomes at feasible review budgets. The preferred intervention depends on observed causes, not a generic fairness slogan.
Keep diagnosis and certification separate
If a new label policy changes positives in B, compare compatible policy versions before attributing the difference to the model. If only appealed items are audited, document selection and collect a broader permitted sample when needed.
Record findings, assumptions, owners, acceptance/rollback criteria and unresolved harms. NIST provides a risk-management framework; linking it does not certify compliance or fairness. Independent domain review is separate from these toy calculations.
Diagrams and relationships
Metrics, label quality and service coverage jointly inform changes; no scalar automatically certifies fairness.
Scroll horizontally on smaller screens, or enlarge the diagram.
Engineer, Senior and Staff follow-ups
These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.
Engineer: Why can 98.5% aggregate accuracy coexist with 50% in B?
The aggregate weights examples, so 990 easy cases dominate ten difficult cases. Report both denominators and inspect error types; equal per-slice weighting would answer a different question.
Engineer: What is recall in a slice with no positives?
It is undefined under the declared count convention because TP+FN=0. Preserve the zero-positive count and explain that the data supplies no recall estimate for that slice.
Senior: What do you check before changing a slice threshold?
Verify population, label-policy and service-path consistency, uncertainty and root cause. Compare data/model/calibration/review interventions under harm and capacity constraints with appropriate governance; a threshold change may move rather than solve the harm.
Senior: How do you audit missing-modality harm?
Record OCR/image/language availability and fallback by cohort, then compare mature error/action outcomes and review delay. Avoid conditioning evaluation only on fully processed examples, which would hide the affected population.
Staff: Who owns unresolved slice trade-offs?
Domain/policy and product owners define acceptable actions with affected-user considerations; data/model/review owners supply evidence and remediation plans. Record accountability, escalation and revisit triggers rather than delegating judgment to a single aggregate.
Staff: How do you standardize harm review without imposing one fairness score?
Standardize population/label/metric provenance, slice counts, uncertainty, service coverage, ownership and review records. Permit task-specific guardrails and explain incompatible goals and limitations. Independent review is a distinct gate.
Practice and self-review
Your practice draft
Explain the 98.5%/50% example, list evidence needed before a policy change and specify an escalation for an under-observed language cohort.
Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.
Progressive hints
Hint 1
Keep the 990/10 denominators visible.
Hint 2
Accuracy does not reveal error type or missing service paths.
Hint 3
Choose a measured next action and owner, not a fairness certificate.
Reveal worked answer guidance
The aggregate is dominated by A and cannot certify B. Obtain per-slice confusion counts, policy/annotation and modality consistency, sampling/maturity/uncertainty and review/appeal access evidence. Assign an owner to gather targeted valid evidence and validate any action change with guardrails.
Evidence to check in your answer
- Explicit denominators and undefined metrics.
- Permitted slices and label comparability.
- Competing harms and service coverage.
- Owner, uncertainty and next measurement.
For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.
Sources and verification scope
Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.
NIST AI Risk Management Framework ↗ · Risk vocabulary and governance; not a certification or blanket legal-compliance guarantee.
Google ML evaluating for bias ↗ · General population/slice-evaluation context, not a fairness certificate or a specific moderation policy.
Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.