Configuration
Models
Deep dive into model architectures, token costs, context window sizes, speed benchmarks, and selection guidelines.
Overview
Selecting the right model balance determines response speed, accuracy, and operational cost.
Comprehensive Model Comparison
| Model | Context Window | Median TTFT | Quality Rating | Cost / 1k Tokens |
|---|---|---|---|---|
| Claude 3.5 Sonnet | 200,000 tokens | 320ms | 9.8 / 10 | $0.003 input / $0.015 output |
| GPT-4o | 128,000 tokens | 380ms | 9.7 / 10 | $0.0025 input / $0.010 output |
| GPT-4o-mini | 128,000 tokens | 210ms | 9.0 / 10 | $0.00015 input / $0.0006 output |
| Claude 3 Haiku | 200,000 tokens | 190ms | 8.8 / 10 | $0.00025 input / $0.00125 output |
| Llama 3.1 70B (Groq) | 128,000 tokens | 140ms | 9.1 / 10 | $0.00059 input / $0.00079 output |
Model Selection Decision Matrix
- High-Touch Enterprise Sales & Support: Claude 3.5 Sonnet — Best for explaining complex pricing tiers, technical APIs, and enterprise compliance.
- High-Volume B2C Help Desk: GPT-4o-mini or Llama 3.1 70B — Delivers blazing-fast answers at near-zero marginal cost.
- Global Multi-Lingual Sites: GPT-4o — Best translation fidelity across Japanese, Arabic, German, and Mandarin.
Switching Models at Runtime
You can switch models instantly without code redeployment in the dashboard. The update propagates across all edge workers in < 10 seconds.
Context Window Management
Asiyst automatically trims conversation memory to keep prompt sizes compact, ensuring the assistant operates far below model limits while preserving key conversational facts.