Knowledge Base
Documents
Upload and index PDF, Markdown, DOCX, text, and HTML documents with automatic OCR and semantic chunking.
Overview
The Documents module enables teams to upload static files—including technical whitepapers, product spec sheets, employee handbooks, and compliance policies—directly into the vector knowledge base.
Supported File Types & Specifications
| Format | File Extensions | Max File Size | Highlights |
|---|---|---|---|
| Portable Document Format | .pdf | 50 MB | Multi-column parsing, table extraction, OCR support |
| Markdown / MDX | .md, .mdx | 20 MB | Header hierarchy preservation, fenced code blocks |
| Microsoft Word | .docx | 30 MB | Heading styles, tables, bulleted lists |
| Plain Text | .txt, .csv, .tsv | 10 MB | Raw text, tabular delimiter indexing |
| HTML Documents | .html, .htm | 15 MB | DOM tree semantic extraction |
Document Parsing & OCR Pipeline
When a document is uploaded:
- Layout Analysis: Detects multi-column layouts, sidebars, headers, and footers, filtering out page numbers and repeated copyright notices.
- Table Reconstruction: Markdown and HTML tables are preserved as structured tabular blocks so the AI model understands rows and columns.
- Code Snippet Extraction: Programming code blocks are tagged with language identifiers to provide syntax-aware technical support.
- Optical Character Recognition (OCR): Scanned PDFs without digital text layers are processed with high-accuracy OCR.
Uploading via Dashboard & REST API
Via Dashboard:
Drag and drop your files into Knowledge > Documents in the dashboard.
Via REST API (Multipart Form):
curl -X POST https://asiyst.com/api/v1/projects/proj_9f8a2b3c4d/knowledge/sources/upload \
-H "Authorization: Bearer $ASIYST_API_TOKEN" \
-F "file=@./enterprise-security-whitepaper.pdf" \
-F "collectionId=col_security_docs" \
-F "title=Security & Compliance Whitepaper 2026"
Chunking Strategies for Long Documents
In Knowledge > Source Settings, adjust how documents are segmented:
- Hierarchical Markdown (Default): Splits by
# H1,## H2, and### H3headers, keeping related paragraphs together.
- Fixed-Size Sliding Window: 512 tokens per chunk with 64-token overlap. Ideal for uniform text documents without clear headers.