
Hybrid search combines keyword retrieval with vector-based semantic retrieval, then merges their results into one ranked list. It can help when people search the same knowledge base using both precise identifiers and descriptions that share few words with the answer. Whether it improves your search depends on your documents, queries, and ranking choices.
Consider a support library. One person searches for an exact error code. Another describes the same problem as “I cannot sign in after changing my device.” A useful search system should handle both without losing product versions, access restrictions, or the specific recovery steps. Start by testing those needs separately, then decide whether combining retrieval methods earns its extra complexity.
Keyword retrieval ranks documents using the words in the query and the indexed text. BM25 is one common scoring method. Vector retrieval compares numerical representations of queries and documents, usually created by an embedding model. It can retrieve related wording even when the query and document use different terms, but its results depend on the model and how the material is indexed.
Microsoft’s Azure AI Search documentation describes a hybrid implementation that runs text and vector queries in parallel and merges their rankings. Other systems expose different controls. The shared idea is to use more than one retrieval signal, rather than assume one signal suits every query.
| Query in a support library | What to preserve | What to test |
|---|---|---|
| “M42” or a product identifier | The exact code, product, and version | Whether the correct reference appears before general troubleshooting |
| “Cannot sign in after changing my device” | The meaning of a description | Whether retrieval finds recovery instructions with different wording |
| “M42 after changing my device” | Both the identifier and the situation | Whether fusion improves the useful results without burying the precise fix |
| A question about another team’s account | The requester’s access permissions | Whether every retrieval route excludes unauthorized material |
Keyword search does not automatically guarantee an exact match. Text analysis may lowercase words, remove punctuation, or split a code into tokens. Elasticsearch’s term-query documentation distinguishes matching an exact indexed term from searching analyzed text. If an identifier must match literally, choose the field mapping and query deliberately, and test real identifier formats. Vector similarity is not a substitute for that requirement.
First, each retriever produces a candidate list. A keyword retriever might return the top 50 matches while a vector retriever returns another 50. Those numbers are design choices, not recommended defaults. Small windows can miss useful documents; larger ones require more work and may introduce more weak candidates.
Next, fusion combines the lists. A rank-based method uses where a document appears, while score-based methods need a defensible way to combine the retrievers’ different score scales. Adding a raw BM25 score to a vector similarity score gives the numbers an arbitrary relationship unless you account for their meaning and range.
A reranker is a separate, optional stage that reassesses the surviving query-document pairs. For example, the Sentence Transformers retrieve-and-rerank guide describes using a cross-encoder after initial retrieval. This adds processing work and needs evaluation against your latency and quality goals. A reranker cannot select a useful document that never reached it.
Answer generation comes later, if the product uses a language model to write a response. Hybrid search can supply evidence for retrieval-augmented generation, or RAG, but retrieval quality and answer quality need separate checks. A relevant passage does not establish that the generated answer uses it correctly. For the broader latency question, see our guide to RAG performance at scale.
Reciprocal rank fusion, or RRF, combines rankings without requiring the underlying scores to share a scale. In the equal-weight form described in Elasticsearch’s RRF documentation, each list contributes 1 / (c + rank) for a document it contains. Ranks start at 1; a document absent from a list receives no contribution from that list. The constant c controls how strongly differences in position affect each contribution.
Use the fictional query “M42 when signing in after changing device.” Imagine these four documents: A is the specific M42 recovery guide, B is general account recovery that also mentions M42, C is an M42 event reference, and D covers expired sign-in links. We assign the following rankings to demonstrate the calculation. They are invented inputs, not results from a search engine or embedding model.
The keyword list is A, B, C. The vector list is B, D, A. Set c = 60 for this illustration. Elasticsearch uses 60 as its default rank constant; that does not make it the right setting for every workload.
| Fused order | Document | Keyword rank | Vector rank | RRF calculation | RRF score |
|---|---|---|---|---|---|
| 1 | B: General account recovery | 2 | 1 | 1/62 + 1/61 | 0.032522 |
| 2 | A: Specific M42 recovery | 1 | 3 | 1/61 + 1/63 | 0.032266 |
| 3 | D: Expired sign-in links | Absent | 2 | 1/62 | 0.016129 |
| 4 | C: M42 event reference | 3 | Absent | 1/63 | 0.015873 |
B ranks first because it appears near the top of both lists. A follows, despite being the first keyword result. If a support specialist judges A to contain the direct fix and B to be too general, fusion has not put the best answer first. The arithmetic is correct; the relevance decision still needs evidence.
The scores above are ranking values, not probabilities that a document is relevant or correct. Also notice that fusion can only use the candidate lists it receives. If A were missing from both lists, changing the fusion constant would not recover it. Fix candidate retrieval, document preparation, or query handling before trying to tune away that problem.
Both retrieval paths should operate on the intended document version and access scope. If one index contains an obsolete recovery guide and the other contains its replacement, the combined result can be misleading. Track document IDs and versions across indexing jobs, and decide how updates and deletions reach each index.
Define the unit being ranked. Is one result a whole document or a passage? If keyword search returns documents while vector search returns passages, specify how they map to each other before fusion. Otherwise, repeated passages from one source may crowd out other useful evidence. Choose a deduplication and grouping rule that preserves the context readers need.
Apply authorization to every retrieval route and check it again where results are assembled or served. Do not retrieve restricted text into a reranker, model context, log, or cache and rely on the final answer to hide it. Tenant boundaries and document permissions are constraints on the system, not relevance signals. Query filters must have consistent meaning across the keyword and vector paths.
Build a small, representative set of actual search tasks with human relevance judgments. Include exact codes, paraphrases, mixed queries, supported languages, stale documents, and questions with no suitable answer. Use separate cases to tune retrieval settings and to check the final choice, so repeated tuning does not become your only evidence.
Compare keyword-only, vector-only, and hybrid retrieval against the same corpus snapshot and access rules. Keep the result budget comparable and document any differences. Where a reranker is involved, report its contribution separately rather than attribute the entire improvement to hybrid search.
Choose acceptance criteria before reading the final comparison. For example, you might require better paraphrase retrieval without regressing exact-code queries, while staying within your response-time budget. Those are workload decisions; there is no universal score or latency threshold that proves a hybrid system is ready.
Hybrid retrieval adds components to operate: indexes, an embedding process, application logic, and sometimes a reranking or generation model. Estimate their requirements from corpus size, update rate, query concurrency, and the models you select. Do not assume every stage needs a GPU, or that a small index proves the full application will fit on a small server.
Compute with Hivenet provides CPU and GPU infrastructure for workloads your team operates. You remain responsible for choosing, configuring, and validating the search software. For an internal search or assistant project, Private AI with Hivenet is a route to discuss the data, model, region, and deployment requirements.
Before that discussion, prepare sample queries, relevance labels, corpus size and formats, access rules, update frequency, and a latency budget. Bring the keyword-only baseline as well as the hybrid result. Discuss the workload with Hivenet once you can show where retrieval succeeds, where it fails, and what the system needs to handle.
Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.