Modern Embedding Approaches

Modern Embedding Approaches

Instruction-Tuned Embeddings

Instructor Model: Prefix text with task instruction: "Represent this document for retrieval: [text]"

Different instructions → different embeddings.

Matryoshka Embeddings

Embeddings that work at multiple dimensions:

OpenAI text-embedding-3 supports this.

Late Interaction

ColBERT:

Multimodal Embeddings

CLIP:

ImageBind:

Sparse-Dense Hybrid

Combine:

Best of both worlds for search.