Design an AI Search Engine
An AI search engine ranks sources for a query — hybrid retrieve, rerank, then optional grounded snippets. If you skip retrieval eval and only generate paragraphs, you built a chatbot, not search.
Explain like I'm new to AI
Search success is the right documents at the top. A generated blurb is optional sugar. Teams collapse the two and ship fluent wrong links.
A cross-encoder or listwise reranker on the top N. This is where quality usually moves, not a bigger generator.
Job: for this query, show the best allowed hits (and maybe a short grounded answer). SLO: nDCG / recall@k, then snippet faithfulness, then latency.
Shape: workflow. Query rewrite can be an LLM step; the control flow is still fixed.
Mental model
A library catalog (ranking) that may also print a two-sentence card. The card is not allowed to invent a book.
How it works
- Query: spell, expand, maybe split. Keep lexical (BM25) for SKUs and names (dense vs sparse, hybrid).
- Retrieve: hybrid over your index. ACL if needed (enterprise lesson).
- Rerank: cross-encoder on top N — usually the biggest quality lever.
- Present: ranked list always. Generated snippet only from top hits; still cite.
- Eval: labeled qrels for ranking; RAG-style faithfulness if you generate. Don't average them away.
- Serve: cache popular queries (exact), not semantic-cache of "close" SKUs.
Public web search adds crawling, spam, freshness — same pipeline, harder ingest. Don't pretend a single chat model replaced information retrieval.
Real-world example
"Search" that only embeds the query and asks a 70B to write an answer with fake links. nDCG was never measured. They added hybrid+rerank, generated less, and users found files again.
Technical explanation
This is the RAG architecture with the generator optional and small. Many products should stop at reranked hits.
Multimodal search (CLIP, frames) is the multimodal RAG lesson — same design loop, extra embedding space.
Common mistakes
Replacing the ranking stack with a chatbot because demos look smarter.
- Pure vectors on catalog IDs.
- No ACL on an internal search box.
When to use it
- Users need to find things (files, SKUs, docs, web). Generation is a helper, not the product.
When NOT to use it
- Don't use this when the user wants a ticket resolved (support) or a patch (coding).
- Don't skip ranking metrics because the snippet "reads well."
Alternatives
- Classic search without an LLM. RAG chatbot when the primary job is an answer, not a list. Research agent when they want a memo.
Quick quiz
Related concepts
- What is Hybrid Search? — Hybrid search combines dense (semantic) and sparse (keyword) retrieval and fuses their rankings, getting meaning-based recall plus exact-term precision.
- Reranking & Cross-Encoders — A reranker re-scores the shortlist of retrieved chunks by reading each query-document pair closely, pushing the most relevant results to the top.
- Design a RAG Chatbot — A RAG chatbot retrieves an approved corpus, reranks, and answers with real citations — not an agent, not a fine-tuned wiki.
Last reviewed: 2026-09-04 · Written by ByHeart AI · Reviewed by ByHeart AI