Knowledge Base
Knowledge Retrieval
How vector search retrieves grounded context passages at runtime, injects citations, and prevents hallucinations.
Overview
Knowledge Retrieval is the real-time execution step that occurs every time a visitor submits a question. The system queries vector indices, re-ranks matching passages, and builds the context window for LLM generation.Runtime Retrieval Step-by-Step
- User Query Vectorization: The incoming user prompt is embedded into a 1536-dimensional vector in < 25ms.
- Approximate Nearest Neighbor (ANN) Search: Queries the project's HNSW vector graph across millions of vectors.
- Re-ranking: Cross-encoder models score the top 20 candidate chunks for precise semantic relevance to the query.
- Context Injection: The top 3–5 highest-scoring chunks are injected into the system prompt with explicit delimiter tags:
<context_source id="src_1" title="Pricing Tier Guide">The Pro plan starts at $49/month and includes 10,000 monthly assistant messages...
</context_source>
Re-ranking & Context Pruning
Re-ranking ensures that even if a chunk has slight keyword mismatches, its true relevance is recognized before sending to the model, boosting answer accuracy by over 35%.
Citation Payload Structure
Every retrieved chunk includes metadata enabling the frontend to render interactive citation chips with tooltips.