Average Response Time

Overview

In conversational AI experiences, latency directly impacts user perception of intelligence and responsiveness. Asiyst measures response speed at every phase of the request lifecycle, ensuring your assistant responds within milliseconds.

Latency Metrics Taxonomy: TTFT vs Total Time

[Visitor Sends Query] 
        │
        ├──▶ (40ms)  Network Edge Transit
        ├──▶ (75ms)  Embedding & Vector Search
        ├──▶ (120ms) LLM First Token Generated ──▶ [Time to First Token (TTFT): 235ms]
        │
        └──▶ (850ms) Full Stream Concluded     ──▶ [Total Response Time: 1085ms]

1. Time to First Token (TTFT): The duration from when the visitor presses Enter until the first word appears and the avatar begins speaking.

1. Total Response Time: The duration required to stream the complete answer, render markdown tables, and deliver citation links.

1. Median (p50), p90, and p99 Latency: Standard percentiles tracking typical performance and outlier spikes.

Performance Benchmarks by AI Provider

How Asiyst Minimizes Response Latency

• Edge Vector Indexing: Document embeddings are distributed to edge regions (North America, Europe, Asia) to execute semantic search in under 30ms.

• Speculative Handshakes: The SDK maintains a persistent WebSocket connection, eliminating TLS handshake overhead on subsequent messages.

• Parallel Retrieval: User embeddings and intent categorization run in parallel rather than sequentially.

Monitoring Latency SLAs & Alerts

In Project Settings > Alerts, configure automated notifications if p90 TTFT exceeds 1,500ms over a 5-minute rolling window, ensuring proactive incident management before visitors notice performance degradation.