Advanced7 min read

What are Matryoshka Embeddings?

Matryoshka embeddings are trained so the most important information sits in the first dimensions — so you can chop the vector shorter to save storage and speed while keeping most of the quality.

Explain like I'm new to AI

Normally an embedding is all-or-nothing: a 1536-number vector is only useful if you keep all 1536 numbers. Matryoshka embeddings change that.

They're trained so the beginning of the vector carries the most important meaning, and later numbers add finer detail. That means you can truncate — keep just the first 256 or 512 numbers — and still get good results, using a fraction of the space.

Drag the slider to keep fewer dimensions and watch size shrink while quality barely moves:

Keep only the first 16 of 16 dimensions — the vector still works.

dim 1most important first →dim 16

Dimensions kept16

Storage / speed

100%

of full size

Quality retained

≈ 100%

of full accuracy

One model, many sizes: truncate a Matryoshka embedding for big savings with only a small quality drop.

Mental model

Like the Russian nesting dolls they're named after, a Matryoshka embedding contains smaller complete embeddings inside it. The full 1536-dim vector holds a good 768-dim vector, which holds a decent 256-dim vector — each usable on its own.

How it works

The model is trained with a special objective (Matryoshka Representation Learning) so early dimensions are the most informative.
To use a shorter vector, you simply keep the first k numbers (and re-normalize).
Shorter vectors mean less storage, less memory, and faster similarity search — with only a small accuracy drop.

Real-world example

A common pattern is two-stage ("adaptive") retrieval:

Shortlist with truncated 256-dim vectors — fast and cheap over millions of items.
Re-rank the top candidates with the full-length vectors for accuracy.

You get most of the speed of small vectors and most of the quality of large ones. Several leading embedding APIs now expose a dimensions parameter precisely because their models are Matryoshka-trained.

Technical explanation

MRL adds loss terms at multiple nested dimensionalities during training, so each prefix of the vector is itself a valid embedding. Truncation is a plain slice followed by L2 re-normalization; no re-embedding needed. Trade-offs: quality degrades gracefully as you shrink (e.g. keeping half the dimensions often costs only a couple of percent of accuracy), letting teams tune the storage/quality/latency balance per use case. It pairs naturally with quantization (lower-precision numbers) for even smaller indexes.

Common mistakes

Common mistake

Truncating a non-Matryoshka embedding. Ordinary embeddings spread information across all dimensions, so chopping them destroys quality. Truncation only works for models trained with MRL.

Forgetting to re-normalize after truncating.
Mixing truncated and full-length vectors in the same index/comparison.

When to use it

Large-scale search where storage and speed matter, or adaptive shortlist-then-rerank retrieval.

When NOT to use it

When your model isn't Matryoshka-trained, or the dataset is small enough that full vectors are cheap.

Alternatives

Quantization and dimensionality reduction (PCA) also shrink vectors, but MRL is built-in and lossless to slice.

Quick quiz

Related concepts

Embedding Models & Dimensions — An embedding model is the trained network that turns input into vectors; its dimensions, training, and quality determine how good your search and RAG will be.
Dense vs Sparse Embeddings — Dense embeddings capture meaning in a few hundred numbers; sparse vectors capture exact keywords across a huge vocabulary. Hybrid search combines both.