Learning pathsPricingChecking sign-in…
← ML curriculum

AIF-004 · FOUNDATION

Embedding spaces, similarity and model compatibility

Embeddings map text into a model-specific vector space for a chosen task. Reliable retrieval depends on compatible query/document encoding, measured similarity behavior, versioned vector metadata and a safe migration path—not merely matching vector dimensions.

Author self-review; independent technical review pending. Synthetic deterministic examples; no live embedding provider, vector database, benchmark or production deployment.

Prerequisites and scoped previews

LLMs, transformers and token generation

Build a useful mental model of tokenization, transformer blocks, self-attention and autoregressive generation.

Context windows and context engineering

Construct bounded, relevant and permission-safe context from instructions, conversation state, retrieved evidence and tool results.

What, when, why and how

An embedding is a task-shaped representation

what
An embedding model maps an input into a fixed-length numeric vector whose geometry is learned from training objectives and data.
when
Use it for semantic retrieval, clustering, deduplication or candidate generation when lexical matching alone is insufficient.
why
The vector captures useful learned relationships but does not preserve every fact, exact identifier or logical relation in the source.
how
Choose a model for language, domain and modality; follow its query/document input contract; evaluate against representative labeled queries.

Similarity is meaningful only inside a compatible space

what
Cosine similarity, dot product and Euclidean distance compare vectors under explicit normalization and index assumptions.
when
Use a metric only after checking whether vectors are normalized and what objective the embedding model expects.
why
Raw scores from different models, dimensions or metric configurations are not directly comparable; a threshold is not portable by default.
how
Pin model and dimensions, validate norms and metric configuration, and calibrate retrieval using labeled relevance judgments.

Embedding changes are data migrations

what
A stored vector is derived data tied to a model/version, input transformation, dimensions and source revision.
when
Use a new index generation when changing model, dimensions, preprocessing or material embedding behavior.
why
Same dimensions do not prove two model spaces align; mixing old document vectors with new query vectors can silently damage recall.
how
Backfill a shadow generation, compare quality and operational metrics, switch query and index routing atomically, then retire the old generation after rollback expires.

1. What a vector can and cannot tell you

Illustrative vectors: query q = [0.8, 0.6], passage A = [0.9, 0.4], passage B = [0.2, 0.98]. Their dot products are q·A = 0.8×0.9 + 0.6×0.4 = 0.96 and q·B = 0.8×0.2 + 0.6×0.98 = 0.748. These tiny vectors are teaching values, not outputs from a real model.

A higher similarity means closer under the selected geometry, not that the passage is true, current, authorized or sufficient. Exact IDs, version numbers, legal clauses and rare names may need lexical retrieval alongside dense vectors.

2. Select the metric from the model contract

Cosine similarity divides the dot product by both vector norms. If all vectors are unit-normalized, cosine similarity and dot product produce the same ordering. Without normalization, dot product also reflects vector magnitude. Euclidean distance ranks by distance, where lower is closer; for unit vectors its ordering is equivalent to cosine similarity.

Never copy a score threshold from another model or corpus. Build a labeled query set, measure Recall@K and downstream answer support, then select thresholds by task and slice. Log model version, dimension, metric, normalization and index generation with each query.

ChoiceWhat it measuresCommon caution
CosineAngle between vectorsNormalize/implement consistently
Dot productDirection and magnitudeMagnitude can change ranking
EuclideanStraight-line distanceScale and normalization matter

3. Version query and document encoders together

A retrieval collection should record embedding provider/model/version, dimensions, modality, query/document input mode, preprocessing version, normalization, metric, chunker version and source revision. A query encoder and document encoder may intentionally use different task prefixes or input types, but they must follow the same model's documented retrieval contract.

Do not mix generations because dimensions happen to match. A safe migration writes new vectors into a separate generation, dual-runs representative traffic, compares retrieval and answer quality, and atomically changes the active generation pointer. Preserve the old generation for a bounded rollback window.

GenerationDocument vectorsQuery encoderServing state
g17embed-v1embed-v1active
g18embed-v2embed-v2shadow → canary
Rollbackembed-v1embed-v1retained until expiry

VISUAL WALKTHROUGH

See the system in motion

Follow the pinned embedding model and index generation contract from source chunks to query-time similarity.

Process overviewAvailable even when interactive playback cannot load.
  1. Pin model and tokenizer-independent embedding configuration

  2. Embed source chunks with document input mode

  3. Record dimensions, metric and normalization

  4. Stage a separate index generation

  5. Switch query encoder and index as a pair

  6. Retain the old generation for rollback

Loading interactive walkthrough…

4. Make vector operations observable and recoverable

Batch embedding requests with bounded concurrency, explicit retry/backoff and per-item outcomes. Respect provider rate limits; retry transient failures but do not silently skip failed chunks. Persist a deterministic work key such as tenant + source ID + source revision + chunk ID + embedding version so a retry is idempotent.

Monitor embedding coverage, failed items, stale-source lag, index publication lag, vector dimension/norm anomalies, retrieval Recall@K, p95 latency and cost per indexed/query unit. A collection that answers quickly but silently omits failed documents is not healthy.

Diagrams and relationships

Versioned embedding retrieval

Query and document vectors meet only when their model contract and active index generation match.

Scroll horizontally on smaller screens, or enlarge the diagram.

Versioned embedding retrievalQuery and document vectors meet only when their model contract and active index generation match. Source text to Document encoder v2: embed. Document encoder v2 to Index generation g18: write vectors. Query encoder v2 to Similarity search: query vector. Index generation g18 to Similarity search: same space. Similarity search to Authorized candidates: candidate IDs.embedwrite vectorsquery vectorsame spacecandidate IDsSource textDocument encoder v2Index generation g18Query encoder v2Similarity searchAuthorized candidates
Versioned embedding retrievalQuery and document vectors meet only when their model contract and active index generation match. Source text to Document encoder v2: embed. Document encoder v2 to Index generation g18: write vectors. Query encoder v2 to Similarity search: query vector. Index generation g18 to Similarity search: same space. Similarity search to Authorized candidates: candidate IDs.embedwrite vectorsquery vectorsame spacecandidate IDsSource textDocument encoder v2Index generation g18Query encoder v2Similarity searchAuthorized candidates

Engineer, Senior and Staff follow-ups

These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.

Engineer: Why can equal dimensions still be incompatible?

Dimensions describe vector shape, not the learned coordinate system. Independently trained models can assign unrelated directions; query and document encoders must share a compatible retrieval contract.

Engineer: When are cosine and dot product equivalent?

When both compared vectors are unit-normalized, dot product equals cosine similarity. Verify normalization rather than assuming it.

Senior: How do you migrate without taking retrieval offline?

Build a separate generation, backfill idempotently, dual-run and compare labeled retrieval, canary a routing pointer that switches query encoder and index together, and retain the old generation for rollback.

Senior: What do you monitor beyond vector-store latency?

Coverage and failed embeddings, source freshness, Recall@K, answer support, dimension/norm anomalies, tail latency, rate-limit retries and cost per indexed/query unit.

Staff: How do you choose a new embedding model?

Use a representative, versioned query set and compare retrieval quality by language/domain/tenant slice, operational latency, throughput, dimensions, modality, context limits, cost and migration risk.

Staff: When should a team not use dense embeddings?

When exact identifiers, deterministic filters or lexical terms dominate, a lexical or structured lookup may be more reliable. Hybrid retrieval is a candidate when semantic recall and exact matching both matter.

Practice and self-review

Your practice draft

A production knowledge base has 12 million chunks encoded with model v1 at 1,536 dimensions. A proposed v2 also returns 1,536 dimensions. Design a no-downtime migration and define the evidence required before switching.

Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.

Progressive hints

Hint 1

Treat vectors as versioned derived data; equal dimensions do not establish space compatibility.

Hint 2

Backfill a separate index generation and make work idempotent by source revision and chunk identity.

Hint 3

Compare recall and downstream answer support, then switch query encoder and index pointer together.

Reveal worked answer guidance

Create generation g2 with the v2 model contract and a new vector field/index. Backfill a consistent source snapshot in bounded batches, then consume the change log to catch updates and deletions. Track per-chunk success and publish only after coverage and index readiness checks. Shadow a fixed, labeled query set against g1 and g2, compare Recall@K, MRR/NDCG where relevant, answer citation support, p95 latency and cost by slice. Canary routing by tenant or traffic cohort; the router selects matching query encoder and index generation as one versioned pair. Keep g1 queryable until rollback expiry, then delete under retention policy.

Evidence to check in your answer

  • Model/index compatibility is explicit.
  • Backfill and change capture avoid gaps and duplicate writes.
  • Quality is compared on representative labeled slices.
  • Cutover and rollback switch a compatible pair atomically.

For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.

Sources and verification scope

Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.

Embeddings guide ↗ · Embedding request concepts and model-specific output behavior; verify currently deployed model limits and dimensions.

Migrate to a new embedding model ↗ · Embedding migration requires a compatible query/document model pair and corpus re-embedding; details are Voyage/MongoDB-specific.

Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.