Learning pathsPricingChecking sign-in…
← ML curriculum

AIF-006 · FOUNDATION

Hybrid retrieval, reranking and grounded evidence

Retrieval is a staged candidate-generation and evidence-selection problem. Dense search, lexical search, rank fusion, authorization, reranking and context packing solve different tasks; quality must be measured with judged queries and grounded-answer evidence rather than similarity scores alone.

Author self-review; independent technical review pending. Synthetic deterministic examples; no live embedding provider, vector database, benchmark or production deployment.

Prerequisites and scoped previews

Prompt contracts, structured outputs and tool calling

Separate trusted instructions from untrusted data, validate structured outputs and keep tool authority in application code.

Embedding spaces, similarity and model compatibility

Choose a task-appropriate embedding contract, understand vector similarity and migrate model/index generations without mixing incompatible spaces.

RAG ingestion, chunking and index lifecycle

Turn existing and changing source systems into provenance-preserving, permission-aware, versioned searchable evidence.

What, when, why and how

Dense and lexical retrieval have complementary failure modes

what
Dense retrieval uses semantic vector similarity while lexical retrieval matches terms and identifiers using an inverted index or sparse representation.
when
Use hybrid retrieval when user intent is semantic but exact names, codes, product IDs or rare terms also matter.
why
Dense search can miss exact tokens; lexical search can miss paraphrases and synonyms.
how
Run bounded searches in parallel, normalize and preserve result identities, fuse ranked lists and evaluate by query slice.

Fusion combines rankings, not necessarily raw scores

what
Reciprocal Rank Fusion combines result positions from multiple ranked lists without assuming their raw scores share a scale.
when
Use it to merge BM25 and vector candidates when score distributions are incomparable or difficult to calibrate.
why
Cosine scores and lexical scores often have different meanings and ranges; naive addition can let one dominate.
how
Compute a documented RRF score, deduplicate by stable chunk ID, retain per-channel ranks and tune using held-out relevance judgments.

Reranking spends more compute on fewer candidates

what
A reranker scores query-passage pairs or cross-attends over a bounded candidate set to improve ordering.
when
Use after broad retrieval when the first-stage top candidates contain enough recall but their ordering is noisy.
why
Reranking every corpus item is expensive; a reranker cannot recover a relevant document that retrieval never returned.
how
Retrieve a broad bounded pool, authorize before text hydration, rerank a smaller set, diversify evidence and pack within token budget.

1. Run dense and sparse retrieval as complementary channels

For query “invoice INV-2026-041 status”, dense retrieval may find passages about payment state and invoice lifecycle, while BM25 strongly matches the exact invoice identifier. Run both with the same tenant and metadata scope, bounded top-N and independent timeouts. Preserve each channel's rank and score for diagnosis.

Do not assume all queries need the same blend. Exact identifiers, error codes and legal clause numbers may favor lexical retrieval; paraphrased natural-language questions may benefit from dense retrieval. Query classification can be an optimization, but the fallback should remain measurable and deterministic.

VISUAL WALKTHROUGH

See the system in motion

Trace dense and lexical retrieval through rank fusion, authorization, reranking and evidence selection.

Process overviewAvailable even when interactive playback cannot load.
  1. Run dense and lexical retrieval in parallel

  2. Fuse rankings with a documented method

  3. Deduplicate by stable chunk identity

  4. Authorize before content hydration

  5. Rerank a bounded candidate pool

  6. Select evidence and preserve citations

Loading interactive walkthrough…

2. Fuse ranked lists without mixing incompatible scores

Reciprocal Rank Fusion uses RRF(d) = Σ 1 / (k + rank_i(d)). Synthetic example with k=60: document A ranks 1st in dense and 5th in lexical, so score = 1/61 + 1/65 ≈ 0.03178. Document B ranks 3rd in dense and 2nd in lexical, so score = 1/63 + 1/62 ≈ 0.03200. B narrowly wins under these assumptions; this is an illustrative calculation, not a universal k or production threshold.

Deduplicate by stable chunk ID, retain source/version and per-channel rank, and make tie-breaking deterministic. Rank fusion is robust to incompatible score scales, but it discards score margins; score fusion may work when channel scores are calibrated on representative judgments.

3. Rerank only a bounded, authorized candidate pool

A cross-encoder or hosted reranker evaluates query-passage pairs and often improves precision near the top. It adds model cost and latency, so use it on a bounded candidate pool after retrieval. Critically, enforce tenant and ACL policy before sending candidate text to an external reranker; candidate IDs alone are not permission to disclose content.

Measure retrieval recall before reranking and ranking quality after reranking. If a relevant passage is absent from the candidate pool, no reranker can restore it. Select diverse evidence across documents/sections, cap near-duplicates and preserve citation spans.

4. Evaluate evidence and answer support end to end

Build a versioned judged-query set with query intent, tenant/ACL scope, expected relevant source spans and acceptable abstention. Track Recall@K, MRR or NDCG for retrieval; citation precision/recall, claim support and answer correctness for generation; plus latency, cost and permission-denial correctness. Segment by language, source type, query class and tenant size.

A high similarity score is not a confidence guarantee. If evidence is missing, stale, contradictory or unauthorized, the system should abstain, ask a clarifying question or route to a human. Preserve enough trace metadata to replay a decision without logging sensitive raw content unnecessarily.

Diagrams and relationships

Hybrid retrieval to grounded evidence

Two candidate channels are fused, authorization is applied before text reaches the reranker, and evidence is selected for answer grounding.

Scroll horizontally on smaller screens, or enlarge the diagram.

Hybrid retrieval to grounded evidenceTwo candidate channels are fused, authorization is applied before text reaches the reranker, and evidence is selected for answer grounding. User query to Dense top-N: semantic. User query to Lexical top-N: exact terms. Dense top-N to RRF + dedupe: rank list. Lexical top-N to RRF + dedupe: rank list. RRF + dedupe to ACL + rerank: candidate IDs. ACL + rerank to Grounded top-K: authorized evidence.semanticexact termsrank listrank listcandidate IDsauthorized evidenceUser queryDense top-NLexical top-NRRF + dedupeACL + rerankGrounded top-K
Hybrid retrieval to grounded evidenceTwo candidate channels are fused, authorization is applied before text reaches the reranker, and evidence is selected for answer grounding. User query to Dense top-N: semantic. User query to Lexical top-N: exact terms. Dense top-N to RRF + dedupe: rank list. Lexical top-N to RRF + dedupe: rank list. RRF + dedupe to ACL + rerank: candidate IDs. ACL + rerank to Grounded top-K: authorized evidence.semanticexact termsrank listrank listcandidate IDsauthorized evidenceUser queryDense top-NLexical top-NRRF + dedupeACL + rerankGrounded top-K

Engineer, Senior and Staff follow-ups

These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.

Engineer: Why not add BM25 and cosine scores directly?

They may have different scales and distributions. Rank fusion such as RRF combines positions without pretending raw scores are calibrated together.

Engineer: Can reranking fix low retrieval recall?

No. A reranker only reorders candidates it receives. Improve first-stage recall or query coverage before increasing reranker complexity.

Senior: Where should authorization happen relative to reranking?

Apply tenant and ACL checks before candidate text is hydrated or sent to an external reranker, then recheck before context delivery when permissions can change.

Senior: Which metrics do you track at each stage?

Recall@K for candidate generation; MRR/NDCG for ranking; citation support and answer correctness for generation; plus latency, cost, freshness and ACL-denial correctness.

Staff: How do you choose a hybrid fusion strategy?

Compare RRF with calibrated score fusion on a held-out judged query set by query class and language. Preserve per-channel ranks/scores for attribution and select based on quality, latency and operational complexity.

Staff: How do you keep retrieval evaluation from being gamed?

Version judgments and corpus snapshots, prevent test-query leakage into tuning, report slice-level metrics and human-audit judge disagreements and unanswerable cases.

Practice and self-review

Your practice draft

Design retrieval for a multilingual enterprise assistant that must find exact policy IDs and paraphrased questions. Include dense and lexical channels, fusion, reranking, ACL enforcement, evaluation and abstention.

Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.

Progressive hints

Hint 1

Keep channel scores and ranks separate until a documented fusion step.

Hint 2

Never send unauthorized text to an external reranker or the model.

Hint 3

Separate retrieval recall, ranking quality and grounded answer support.

Reveal worked answer guidance

Normalize the query without removing identifiers. Resolve tenant and current permissions, then run dense and BM25 retrieval in parallel with bounded top-N, filters and deadlines. Merge candidates by stable chunk identity using RRF or a judged/calibrated score-fusion policy; preserve channel ranks and source provenance. Filter and recheck ACLs before hydrating text, then rerank a bounded authorized set. Diversify top-K evidence, preserve source spans and pack under a token budget. Evaluate Recall@K, MRR/NDCG, citation precision/recall, claim support, answer correctness and abstention across language, identifier, source and tenant slices. If no sufficiently supported authorized evidence exists, abstain or escalate instead of fabricating an answer.

Evidence to check in your answer

  • Complementary dense and lexical retrieval.
  • Correct rank fusion and deterministic deduplication.
  • Authorization before text disclosure to rerank/model.
  • Stage-specific and end-to-end evaluation with abstention.

For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.

Sources and verification scope

Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.

MongoDB Vector Search hybrid search overview ↗ · Rank fusion, score fusion and reranking concepts; feature/version availability is product-specific.

Create a vector query ↗ · Vector query and hybrid ranking concepts; defaults and score semantics are Azure AI Search-specific.

Relevance scoring in hybrid search using Reciprocal Rank Fusion ↗ · RRF combines ranked lists from parallel retrieval methods; rank constant and result semantics are Azure AI Search-specific.

Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.