Prerequisites and scoped previews
Embeddings and representation contracts
Define embedding training objective, normalization, dimension and model/index compatibility.
What, when, why and how
Exact retrieval as an oracle
- what
- Exact retrieval scores every eligible candidate under a declared vector/metric and deterministic tie rule. ANN trades some search fidelity for resource efficiency.
- when
- Use exact search on small fixtures and sampled evaluation queries when validating an approximate index or tuning its resource budget.
- why
- An ANN result is not a quality oracle. A changed metric, filter or corpus can look like ANN error unless the reference has identical semantics.
- how
- Pin query/corpus/representation versions and eligibility, score all eligible items, sort by descending similarity then stable ID, and compare approximate returned IDs. Report search overlap separately from relevance Recall@K.
Eligibility before and after retrieval
- what
- Eligibility defines which items may be returned: publication, language, permissions, age policy, revocation and already-consumed rules may apply.
- when
- Use it before building the evaluation denominator and again before displaying results if eligibility can change.
- why
- Taking global top K and filtering afterward can underfill even when eligible items exist. An index’s stale snapshot cannot override an authoritative revocation.
- how
- Use supported prefilters or partitions where practical, oversample boundedly and refill from other eligible sources. Recheck live policy before output. Return fewer items or a declared fallback when the budget is exhausted.
Index operating point
- what
- An index operating point couples search effort, memory, build/update cost, eligible recall and tail latency under a fixed workload.
- when
- Use this when comparing flat exact search with approximate structures such as graph or partition-based indexes.
- why
- A recall/latency number from unrelated hardware or a different filter selectivity cannot size your service. Higher search effort can worsen tail latency or overload.
- how
- Benchmark representative query/filter/cohort distributions, count shortfalls, include memory overhead and rebuild capacity, then select a measured budget. This chapter’s ordered ID fixture simulates an ANN miss and does not benchmark an ANN implementation.
A filter changes which top K is correct
Query q=(1,0). Candidates X=(1,0), A=(0.9,0), B=(0.8,0), C=(0.7,0), D=(0.6,0), E=(0.5,0). X is ineligible and A–E are eligible. Exact eligible top-3 is A,B,C. A simulated approximate result A,C,D overlaps that oracle in two of three IDs: search overlap@3=2/3.
Suppose independently judged relevant eligible items are A,B,E. Relevance Recall@3 for A,C,D is 1/3, not 2/3. The first number measures approximation of the exact vector ordering; the second measures retrieval of judged relevant items. Both need their own ground truth and denominator.
A global top-3 X,A,B followed by filtering returns only A,B. Oversampling to four happens to include C in this toy corpus, but a fixed oversampling factor cannot guarantee enough eligible items for every filter. Validate limits and shortfall handling rather than promising a universal multiplier.
Bound work and plan the fallback
For a synthetic serving stage, retrieve at most 200 unique candidates from permitted sources and spend at most a declared retrieval deadline. Deduplicate IDs across followed, recent, popular and vector sources before expensive ranking. Preserve source provenance for diagnostics.
If an ANN service times out, use eligible popularity/followed candidates within the remaining deadline. If there are fewer than K eligible items, return fewer; never relax permissions or moderation to fill a slate. Signal degraded mode in logs and measure its quality and coverage separately.
Exact scoring costs O(Nd) arithmetic for N eligible d-dimensional vectors before top-K selection. A size-K heap can bound selection memory; the small fixture uses a full sort for transparency. Raw vector bytes are N*d*bytes_per_component, excluding index, metadata and allocator overhead.
Treat index rebuild as a release
Build a shadow snapshot from the declared catalog and representation versions. Record cutoff, coverage, invalid/zero vectors, deletion/revocation handling and compatibility metadata. Evaluate against exact retrieval under the same eligible corpus and inspect affected slices.
Promote an immutable index reference atomically with the matching query encoder. Keep the last compatible bundle warm or loadable. A partial rebuild that advertises a new representation while containing old vectors must fail validation. Backfills and retries need explicit item/version identity.
A library may support filter selectors or different update semantics across index types. Check the chosen implementation and version before writing code; this reference is library-neutral and uses no ANN dependency.
Avoid misleading retrieval success
Deduplicate IDs before counting a hit. A repeated A cannot count twice toward recall. Queries with no judged eligible positives have undefined relevance recall under this chapter’s convention and are reported with coverage counts. A filtered-out relevant item must be removed from the eligible-positive denominator, with the exclusion count retained.
Compare results on new items, rare languages, heavy users, selective filters and cold-start users rather than relying only on an average. Low ANN overlap can explain missing relevant items, but high ANN overlap with poor relevance suggests a representation/objective problem rather than an indexing problem.
Diagrams and relationships
The exact vector oracle and independent relevance labels answer different questions.
Scroll horizontally on smaller screens, or enlarge the diagram.
Engineer, Senior and Staff follow-ups
These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.
Engineer: Which denominator gives 2/3 in the toy fixture?
The three IDs in exact eligible vector top-3, A/B/C. The simulated ANN result A/C/D overlaps in A/C. Relevance uses a different judged set A/B/E and therefore gives 1/3.
Engineer: Can X be returned just to complete three results?
No. Eligibility is an invariant. Filter or reject X and use a bounded eligible refill; return a shorter slate if necessary. Count the shortfall rather than weakening policy.
Senior: What should you measure when filter selectivity worsens?
Shortfall rate, eligible recall, retrieval work and latency tails by filter cohort. Compare prefilter, partition and bounded oversampling options under their update/memory costs; do not reuse an unfiltered benchmark.
Senior: How do you investigate high ANN overlap but low user quality?
Verify relevance/label validity and the embedding objective before tuning search effort. Exact similarity ordering may itself poorly correlate with the product outcome. Inspect candidate-source contributions, cold start and downstream ranking.
Staff: Who owns revocation when indexes update slowly?
A shared catalog/policy authority defines the revocation contract and serving performs a final eligibility check. Index owners propagate removals and measure lag, while product owners define the safe underfill behavior. Slow refresh cannot authorize a revoked item.
Staff: How would you manage a costly index migration across regions?
Publish compatible bundle references, gate coverage and quality per region, reserve rebuild/warmup capacity and retain the prior bundle. Stage traffic, measure regional tails and revocation lag, and roll back references without mixing representation spaces.
Practice and self-review
Your practice draft
For the six candidates, compute exact eligible top-3, search overlap and relevance recall. Explain the timeout and eligibility-shortfall paths.
Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.
Progressive hints
Hint 1
Remove X before defining the oracle.
Hint 2
Use distinct denominators for vector overlap and judged relevance.
Hint 3
Refill has a work/deadline bound and must preserve eligibility.
Reveal worked answer guidance
Exact eligible top-3 is A/B/C. Approximate A/C/D gives search overlap 2/3 and relevance recall 1/3 for A/B/E. A timeout uses bounded eligible fallback; insufficient eligible items produce a shorter slate and a recorded shortfall. No ANN latency result can be inferred from these IDs.
Evidence to check in your answer
- Correct eligible corpus.
- Distinct search/relevance metrics.
- Duplicates and no-positive cases declared.
- Budgeted fallback and compatible migration.
For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.
Sources and verification scope
Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.
Faiss documentation ↗ · Similarity search/index trade-offs; pin versions and measure recall/latency under declared filters/data.
Faiss metric types and distances ↗ · Metric/normalization definitions only; the fixture executes exact scalar math and no Faiss/ANN benchmark.
Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.