
Embedding models turn text into numerical vectors that a search system can compare. For a retrieval application, the useful question is whether those comparisons bring the right passages back for the people using it. A high benchmark score or a larger vector does not settle that question.
Choose a model by matching it to your queries, documents, languages, and operating constraints, then testing the shortlist on judged examples. This guide focuses on pretrained text embedding models for retrieval. If you are building a network layer that maps token IDs to learned vectors, see the separate PyTorch nn.Embedding guide.
Start with a short workload description: who searches, what they ask, which documents they may access, and what a useful result looks like. A support agent looking for an answer to a customer question has a different task from someone finding near-duplicate questions.
Sentence Transformers distinguishes symmetric and asymmetric search. Similar-length question pairs are a symmetric example; a short query retrieving a longer passage is asymmetric. Use a model and encoding procedure intended for the task. Queries and documents need compatible representations, even when their instructions differ.
For example, an English/French support library could require a French question to find an English troubleshooting passage. Add those cross-language cases deliberately. A multilingual label alone does not establish that the model handles your product vocabulary, abbreviations, or uncommon language pairs well.
Write down the expected chunk count after document splitting, typical and long-tail input lengths, daily changes, query concurrency, and acceptable latency. Also record data-handling requirements for any hosted endpoint. These details eliminate unsuitable candidates before a larger experiment.
A model name is only part of the configuration. Record its revision, tokenizer, input-length limit, pooling or output procedure, query and document instructions, normalization, and supported similarity measure. Keep the deployment library version alongside those choices.
The examples below illustrate different constraints. They are candidates to investigate, not a ranking or a claim that they outperform newer models. Model-card details were checked on November 16, 2026.
| Example model | Dense vector dimensions | Input and usage detail | Listed license |
|---|---|---|---|
| all-MiniLM-L6-v2 | 384 | English sentence and short-paragraph encoder; default truncation beyond 256 word pieces. | Apache 2.0 |
| multilingual-e5-base | 768 | Multilingual; truncates at 512 tokens. Retrieval uses query: and passage: prefixes, including for non-English text. | MIT |
| BGE-M3 | 1,024 | Multilingual; documented sequence length of 8,192 tokens. Also supports sparse and multi-vector modes; this comparison concerns its dense output. | MIT |
Token limits are not word counts. Inspect the tokenized inputs that your application actually sends, including required prefixes. A longer supported sequence does not prove that one vector will preserve every relevant detail in a long document. Compare sensible chunking choices, and record truncation rather than letting it silently remove the answer.
Review the actual license files and any separate hosting terms before adoption. For an API, also verify the exact model version, output dimensions, request limits, data retention, and how version changes are communicated. Do not assume that an API with a familiar format has identical embedding behavior.
MTEB evaluates several kinds of embedding tasks. Its results can help form a shortlist, but a broad aggregate score is not a substitute for the language, domain, and retrieval task you care about.
Build a judged set of realistic queries with relevant passage IDs. Include common requests, domain-specific wording, multilingual cases, exact identifiers, and questions with no answer in the corpus. Keep a separate set for final comparison so repeated tuning does not teach the experiment to pass its own examples.
Compare candidates on the same corpus snapshot and relevance labels. Start with consistent document splitting, result count, and access filters while applying each model's required preprocessing. If you later tune chunking for a model, report that as a separate configuration change. Keep keyword retrieval as a baseline, especially for product codes and exact names.
Choose metrics that describe the failure you want to reduce. Recall@k concerns how much of the known relevant material is retrieved; MRR rewards placing the first relevant result early. The Sentence Transformers retrieval evaluator supports these and other measures. A simpler review can count queries with at least one useful result in the first five, but label that measure explicitly rather than calling it recall.
Inspect failures by query group, not just an overall average. Treat no-answer questions separately, and do not transfer a similarity threshold unchanged between models. A high similarity score is not evidence that a passage answers the question or that the user has permission to read it.
For a single dense float32 vector per chunk, the raw vector payload is the number of chunks multiplied by the number of dimensions multiplied by four bytes. The table assumes one million chunks. It is arithmetic, not a measurement of a database or a model's runtime memory.
| Dimensions | Calculation | Raw vector payload |
|---|---|---|
| 384 | 1,000,000 × 384 × 4 bytes | 1.536 GB, about 1.43 GiB |
| 768 | 1,000,000 × 768 × 4 bytes | 3.072 GB, about 2.86 GiB |
| 1,024 | 1,000,000 × 1,024 × 4 bytes | 4.096 GB, about 3.81 GiB |
GB uses one billion bytes; GiB uses 1,073,741,824 bytes. Add the index structure, stored text, metadata, replicas, backups, and temporary rebuild space to your capacity plan. Model weights and inference working memory are separate again. A million source documents can produce many more than a million chunks.
Reducing dimensions or changing the stored precision can change storage needs and retrieval quality. Use only configurations supported by the model and index, then repeat the relevance checks. Cutting an arbitrary vector to a shorter length is not a general substitute for choosing a suitable model.
There are three workloads to measure: the initial corpus build, ongoing document updates, and live query encoding. The first may benefit from large batches. The last must meet the application's latency target under concurrent requests. One throughput number cannot describe all three.
For each candidate, record hardware, input-length distribution, batch size, precision, memory use, and whether startup is included. Measure chunks or tokens per second for indexing, along with latency percentiles for query encoding. Report document loading, transfer, and index writes separately so a slow ingestion path is not mistaken for a slow model.
Estimate a rebuild window from the measured workload, then allow for retries and shared capacity. A CPU deployment may be adequate for a modest workload; a GPU can be worth testing when encoding demand justifies it. Choose from measured capacity and complete operating cost, including idle time, rather than assuming every embedding service needs a GPU.
Keep encoding measurements separate from retrieval, reranking, and answer generation. The RAG performance guide covers the broader pipeline, where an embedding change is only one possible source of delay.
Two models can produce the same number of dimensions while using incompatible vector spaces. Unless compatibility is explicitly established, re-embed the corpus when changing the embedding model. Route queries through the encoder that matches the index being searched.
Build the replacement index alongside the current one. Compare retrieval on the held-out queries, measure capacity, reconcile document changes and deletions, and confirm access filters before switching traffic. Keep a rollback path that restores the matching encoder and index together.
Store the model revision and preprocessing configuration with the index version. If a new tokenizer, prompt, chunking rule, or normalization setting changes the representation, assess that change explicitly instead of treating it as an invisible deployment update.
If your team operates the encoder and retrieval service, Compute with Hivenet provides CPU and GPU infrastructure to assess against the measured workload. Plan the service, updates, data access, and recovery responsibilities alongside the instance choice.
If model selection and deployment need to be designed together, Private AI with Hivenet offers a scoped discussion around the model, data, infrastructure, region, and operating requirements. Bring the following evidence so the discussion starts with your workload:
Agree on the exact model, supported endpoint behavior, and operating responsibilities before adopting a service. Hosting capacity alone does not establish retrieval quality; the decision should connect the quality evidence to a deployment your team can maintain.
Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.