Embedding Models & Dimensions
An embedding model is the trained network that converts text (or images) into vectors — and choosing the right one, at the right number of dimensions, largely decides how good your search and RAG results will be.
Explain like I'm new to AI
Embeddings don't appear by magic — they're produced by an embedding model. It's a neural network trained specifically to place similar meanings close together in vector space.
Different models produce vectors of different dimensions (how many numbers per vector) and different quality (how well "close in vectors" matches "close in meaning"). Picking a good model is one of the highest-leverage decisions in a search or RAG system.
Mental model
Think of an embedding model as a cartographer drawing the meaning-map. A great cartographer places every town (idea) exactly where it belongs, so nearby towns really are related. A poor one scatters them, and your "nearest neighbor" search returns nonsense. Same map idea — very different usefulness.
How it works — what to compare
- Dimensions: the vector length (e.g. 384, 768, 1536, 3072). More dimensions can capture more nuance but cost more storage, memory, and compute per comparison.
- Quality / accuracy: how reliably similar meanings map close together — usually judged on retrieval benchmarks.
- Max input length: how much text the model can embed at once (matters for chunk size).
- Speed & cost: local open models vs. hosted API models; latency and price per million tokens.
- Domain fit & languages: general-purpose vs. code/legal/medical; multilingual support.
Real-world example
For a support-docs search you might pick a compact 384-dimension open model that runs cheaply on your own hardware and is plenty accurate. For a nuanced legal-research tool, you might pay for a larger, higher-quality hosted model. The right choice is the smallest, cheapest model that hits your accuracy bar — bigger isn't automatically better.
Technical explanation
Modern text embedding models are transformer encoders trained with contrastive learning on huge sets of related/unrelated pairs, often with instruction prefixes (e.g. "represent this query for retrieval:") that tune vectors for a task. Compare candidates on public leaderboards like MTEB (Massive Text Embedding Benchmark) — but always re-test on your own data, since benchmark rank rarely transfers perfectly. Practical rules:
- The same model must embed both your documents and your queries.
- Re-embedding everything is required if you switch models (vectors aren't cross-compatible).
- Many current models are Matryoshka-trained, letting you shorten dimensions to save space with little quality loss.
Common mistakes
Assuming more dimensions always means better results. Extra dimensions add cost and can add noise; a well-trained smaller model often beats a bigger one — measure on your data.
- Embedding queries with a different model than the documents.
- Trusting a leaderboard without testing on your own content and query style.
When to use it
- Any semantic search / RAG system — choosing and evaluating the embedding model is step one.
When NOT to use it
- Not applicable; if you use embeddings at all, you're choosing an embedding model.
Alternatives
- Managed embedding APIs vs. self-hosted open models — pick based on cost, privacy, latency, and accuracy.
Quick quiz
Related concepts
- Similarity: Cosine, Dot Product & Euclidean — Similarity metrics measure how close two embedding vectors are; cosine similarity compares direction, dot product adds magnitude, and Euclidean measures distance.
- What are Embeddings? — Embeddings turn information into numerical vectors where similar meanings sit close together, so software can compare meaning with math.
Last reviewed: 2026-09-01 · Written by ByHeart AI · Reviewed by ByHeart AI