Prerequisites and scoped previews
LLMs, transformers and token generation
Build a useful mental model of tokenization, transformer blocks, self-attention and autoregressive generation.
Context windows and context engineering
Construct bounded, relevant and permission-safe context from instructions, conversation state, retrieved evidence and tool results.
What, when, why and how
An embedding is a task-shaped representation
- what
- An embedding model maps an input into a fixed-length numeric vector whose geometry is learned from training objectives and data.
- when
- Use it for semantic retrieval, clustering, deduplication or candidate generation when lexical matching alone is insufficient.
- why
- The vector captures useful learned relationships but does not preserve every fact, exact identifier or logical relation in the source.
- how
- Choose a model for language, domain and modality; follow its query/document input contract; evaluate against representative labeled queries.
Similarity is meaningful only inside a compatible space
- what
- Cosine similarity, dot product and Euclidean distance compare vectors under explicit normalization and index assumptions.
- when
- Use a metric only after checking whether vectors are normalized and what objective the embedding model expects.
- why
- Raw scores from different models, dimensions or metric configurations are not directly comparable; a threshold is not portable by default.
- how
- Pin model and dimensions, validate norms and metric configuration, and calibrate retrieval using labeled relevance judgments.
Embedding changes are data migrations
- what
- A stored vector is derived data tied to a model/version, input transformation, dimensions and source revision.
- when
- Use a new index generation when changing model, dimensions, preprocessing or material embedding behavior.
- why
- Same dimensions do not prove two model spaces align; mixing old document vectors with new query vectors can silently damage recall.
- how
- Backfill a shadow generation, compare quality and operational metrics, switch query and index routing atomically, then retire the old generation after rollback expires.
1. What a vector can and cannot tell you
Illustrative vectors: query q = [0.8, 0.6], passage A = [0.9, 0.4], passage B = [0.2, 0.98]. Their dot products are q·A = 0.8×0.9 + 0.6×0.4 = 0.96 and q·B = 0.8×0.2 + 0.6×0.98 = 0.748. These tiny vectors are teaching values, not outputs from a real model.
A higher similarity means closer under the selected geometry, not that the passage is true, current, authorized or sufficient. Exact IDs, version numbers, legal clauses and rare names may need lexical retrieval alongside dense vectors.
2. Select the metric from the model contract
Cosine similarity divides the dot product by both vector norms. If all vectors are unit-normalized, cosine similarity and dot product produce the same ordering. Without normalization, dot product also reflects vector magnitude. Euclidean distance ranks by distance, where lower is closer; for unit vectors its ordering is equivalent to cosine similarity.
Never copy a score threshold from another model or corpus. Build a labeled query set, measure Recall@K and downstream answer support, then select thresholds by task and slice. Log model version, dimension, metric, normalization and index generation with each query.
| Choice | What it measures | Common caution |
|---|---|---|
| Cosine | Angle between vectors | Normalize/implement consistently |
| Dot product | Direction and magnitude | Magnitude can change ranking |
| Euclidean | Straight-line distance | Scale and normalization matter |
3. Version query and document encoders together
A retrieval collection should record embedding provider/model/version, dimensions, modality, query/document input mode, preprocessing version, normalization, metric, chunker version and source revision. A query encoder and document encoder may intentionally use different task prefixes or input types, but they must follow the same model's documented retrieval contract.
Do not mix generations because dimensions happen to match. A safe migration writes new vectors into a separate generation, dual-runs representative traffic, compares retrieval and answer quality, and atomically changes the active generation pointer. Preserve the old generation for a bounded rollback window.
| Generation | Document vectors | Query encoder | Serving state |
|---|---|---|---|
| g17 | embed-v1 | embed-v1 | active |
| g18 | embed-v2 | embed-v2 | shadow → canary |
| Rollback | embed-v1 | embed-v1 | retained until expiry |
VISUAL WALKTHROUGH
See the system in motion
Follow the pinned embedding model and index generation contract from source chunks to query-time similarity.
Pin model and tokenizer-independent embedding configuration
Embed source chunks with document input mode
Record dimensions, metric and normalization
Stage a separate index generation
Switch query encoder and index as a pair
Retain the old generation for rollback
Loading interactive walkthrough…
4. Make vector operations observable and recoverable
Batch embedding requests with bounded concurrency, explicit retry/backoff and per-item outcomes. Respect provider rate limits; retry transient failures but do not silently skip failed chunks. Persist a deterministic work key such as tenant + source ID + source revision + chunk ID + embedding version so a retry is idempotent.
Monitor embedding coverage, failed items, stale-source lag, index publication lag, vector dimension/norm anomalies, retrieval Recall@K, p95 latency and cost per indexed/query unit. A collection that answers quickly but silently omits failed documents is not healthy.
Diagrams and relationships
Query and document vectors meet only when their model contract and active index generation match.
Scroll horizontally on smaller screens, or enlarge the diagram.
Engineer, Senior and Staff follow-ups
These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.
Engineer: Why can equal dimensions still be incompatible?
Dimensions describe vector shape, not the learned coordinate system. Independently trained models can assign unrelated directions; query and document encoders must share a compatible retrieval contract.
Engineer: When are cosine and dot product equivalent?
When both compared vectors are unit-normalized, dot product equals cosine similarity. Verify normalization rather than assuming it.
Senior: How do you migrate without taking retrieval offline?
Build a separate generation, backfill idempotently, dual-run and compare labeled retrieval, canary a routing pointer that switches query encoder and index together, and retain the old generation for rollback.
Senior: What do you monitor beyond vector-store latency?
Coverage and failed embeddings, source freshness, Recall@K, answer support, dimension/norm anomalies, tail latency, rate-limit retries and cost per indexed/query unit.
Staff: How do you choose a new embedding model?
Use a representative, versioned query set and compare retrieval quality by language/domain/tenant slice, operational latency, throughput, dimensions, modality, context limits, cost and migration risk.
Staff: When should a team not use dense embeddings?
When exact identifiers, deterministic filters or lexical terms dominate, a lexical or structured lookup may be more reliable. Hybrid retrieval is a candidate when semantic recall and exact matching both matter.
Practice and self-review
Your practice draft
A production knowledge base has 12 million chunks encoded with model v1 at 1,536 dimensions. A proposed v2 also returns 1,536 dimensions. Design a no-downtime migration and define the evidence required before switching.
Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.
Progressive hints
Hint 1
Treat vectors as versioned derived data; equal dimensions do not establish space compatibility.
Hint 2
Backfill a separate index generation and make work idempotent by source revision and chunk identity.
Hint 3
Compare recall and downstream answer support, then switch query encoder and index pointer together.
Reveal worked answer guidance
Create generation g2 with the v2 model contract and a new vector field/index. Backfill a consistent source snapshot in bounded batches, then consume the change log to catch updates and deletions. Track per-chunk success and publish only after coverage and index readiness checks. Shadow a fixed, labeled query set against g1 and g2, compare Recall@K, MRR/NDCG where relevant, answer citation support, p95 latency and cost by slice. Canary routing by tenant or traffic cohort; the router selects matching query encoder and index generation as one versioned pair. Keep g1 queryable until rollback expiry, then delete under retention policy.
Evidence to check in your answer
- Model/index compatibility is explicit.
- Backfill and change capture avoid gaps and duplicate writes.
- Quality is compared on representative labeled slices.
- Cutover and rollback switch a compatible pair atomically.
For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.
Sources and verification scope
Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.
Embeddings guide ↗ · Embedding request concepts and model-specific output behavior; verify currently deployed model limits and dimensions.
Migrate to a new embedding model ↗ · Embedding migration requires a compatible query/document model pair and corpus re-embedding; details are Voyage/MongoDB-specific.
Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.