Assistant Responses

Overview

When a visitor submits a question, the Asiyst inference engine orchestrates dense vector retrieval, system prompt formatting, model streaming, and action dispatch in under 200 milliseconds.

The Generation Pipeline

[User Message] ──▶ [Embedding Generation] ──▶ [Vector RAG Search]
                                                       │
                                                       ▼
[Client Response] ◀── [Token Streaming] ◀── [LLM Inference Router]

1. Embedding Generation: The user's query is converted into a 1536-dimensional dense vector using state-of-the-art embedding models.

1. Vector RAG Search: The query vector is compared against your project's indexed knowledge base using cosine similarity search.

1. Context Assembly: The top matching document passages, recent conversation turns, and system instructions are assembled into a structured prompt.

1. LLM Inference: The prompt is submitted to your selected AI Provider (Claude 3.5 Sonnet, GPT-4o, or Llama 3.1).

1. Streaming & Action Extraction: The model streams tokens back over the WebSocket connection while parsing structured action tags (e.g. navigation triggers).

Grounding & Vector Retrieval

To prevent hallucinations, the assistant uses strict confidence scoring:

• High Confidence (> 0.82 similarity): The assistant answers definitively and attaches citation badges to the response.

• Medium Confidence (0.65 – 0.82): The assistant answers and notes related documentation links.

• Low Confidence (< 0.65): The assistant politely responds that verified information is unavailable and offers a link to contact human support.

Model Cascading & Fallbacks

Asiyst includes built-in high availability:

• If your primary model endpoint experiences rate limits (HTTP 429) or high latency (> 2500ms), the system automatically retries with your secondary backup provider.

• All model endpoints are monitored with automated health checks every 10 seconds.

Response Formatting & Action Triggers

Assistant responses support standard GitHub-flavored Markdown, syntax-highlighted code blocks, tables, and actionable link chips that navigate the user to relevant pages on your website.