Hybrid Search: Combining Keyword & Vector

A developer searches an internal knowledge base for "CUDA_OUT_OF_MEMORY error code 0x3F" and gets back generic GPU troubleshooting articles, not the exact page documenting that exact error code. Semantic search found things that are conceptually related to GPU memory problems, but it missed the one document that is an exact, literal match, because an embedding model reasons about meaning, not about matching specific tokens precisely.

An embedding model represents "0x3F" and "error code" as points in meaning-space alongside thousands of other technical terms, and a specific hexadecimal code is exactly the kind of token an embedding model has the least reason to represent distinctively, it looks, semantically, like any other short alphanumeric string. Keyword search (classically, BM25, a refinement of simple term-frequency matching) has the opposite strength and weakness: it excels at exact and near-exact token matches, but has no concept of meaning at all, it would never connect "puppy" and "dog."

Combining keyword and semantic search is harder than it sounds, because their raw scores are not comparable: a BM25 score and a cosine similarity live on completely different scales, and naively averaging them is meaningless. Reciprocal Rank Fusion (RRF) sidesteps the problem entirely by ignoring raw scores and working with rank position instead. Each retrieval method produces its own ranked list, and RRF combines them by rewarding a document for showing up near the top of either list.

Running this shows D1, the exact-match document that pure semantic search had buried at rank 4, essentially tied for first place once its dominant keyword rank gets folded in. Neither signal alone was enough, keyword search alone would miss the query entirely if it were phrased even slightly differently, and semantic search alone buried the one document that mattered, but combined, the right answer rises back to the top.