Knowledge Base Setup
Complete guide to setting up your AI Knowledge Base with website crawling, file uploads, and text entry
Knowledge Base Setup Guide
What is the Knowledge Base?
The Knowledge Base is the foundation of your AI Customer Support system. It's a centralized repository where you store information about your business, products, services, policies, and FAQs. This content powers your AI agent's ability to provide accurate, contextual responses.
How It Works
When an email or chatbot message arrives, the AI:
- Analyzes the query against your Knowledge Base
- Determines if it's customer support-related based on your KB content
- Retrieves relevant information using advanced vector search (RAG)
- Generates a response using the context from your Knowledge Base
- Applies your response style (friendly, professional, concise, or custom)
Key Point: Every customer support email and chatbot message is evaluated and answered using your Knowledge Base. The more comprehensive your KB, the better your AI performs.
Why Knowledge Base Matters
Email Classification Accuracy
The Knowledge Base helps the AI understand what qualifies as a customer support inquiry for your business. Without it, the AI can't distinguish between legitimate customer questions, spam, internal communications, or marketing messages.
Response Quality
Your KB directly impacts response accuracy. The AI can only provide information that exists in your Knowledge Base. Missing information = incomplete or escalated responses.
Confidence Scoring
The AI assigns a confidence score to each response based on how well your KB content matches the customer's question. Better KB content = higher confidence = more automated responses.
Three Ways to Add Content
You can populate your Knowledge Base using three flexible methods. Choose the one that works best for your content.
Method 1: Website Crawling (Professional+ Plans)
Best for: Businesses with comprehensive websites, help centers, or documentation sites.
How It Works
- Enter Your Website URL - Paste your website or help center URL
- AI Crawls Your Site - The system discovers and crawls up to 50 pages (default)
- Content Extraction - Extracts clean text from each page
- Intelligent Chunking - Breaks content into 1000-character chunks with 200-character overlap
- Vector Embeddings - Creates AI-readable embeddings stored in Pinecone
- Real-time Progress - Track crawling status with live updates
Plan Limits
| Plan | Max Crawls/Month |
|---|---|
| Starter | Not available |
| Professional | 10 crawls |
| Enterprise | 100 crawls |
Crawl limits reset monthly on your billing anniversary date.
Best Practices
✅ Do:
- Use your main help center or documentation URL
- Ensure pages contain relevant customer support information
- Keep pages up-to-date (re-crawl when content changes)
❌ Avoid:
- Crawling pages with login requirements
- Including marketing-only pages without support content
- Crawling sites with dynamic JavaScript-only content
Method 2: File Upload
Best for: Businesses with documents, PDFs, or structured content files.
Supported File Types
- PDF - Product manuals, guides, policies (max 10MB per file)
- TXT - Plain text documents, FAQs, scripts
- DOCX - Microsoft Word documents, formatted guides
Plan Limits
| Plan | Max Files | Max Storage |
|---|---|---|
| Free Trial | 50 | 5 GB |
| Starter | 10 | 500 MB |
| Professional | 50 | 5 GB |
| Enterprise | 200 | 200 GB |
Best Practices
✅ Do:
- Use descriptive file names (e.g., "Product_FAQ_2024.pdf")
- Keep files focused on specific topics
- Update files when information changes
- Use text-based PDFs (not scanned images)
❌ Avoid:
- Uploading files with mostly images
- Scanned PDFs without OCR
- Password-protected or encrypted files
- Files with outdated information
Method 3: Direct Text Entry
Best for: Quick setup, FAQs, key information snippets, or testing.
How It Works
- Type or Paste Content - Enter text directly into the textarea
- Auto-Conversion - System converts text to .txt file on save
- Automatic Upload - File uploaded with datestamp
- Processing - Text chunked and vectorized like uploaded files
- Editable - Update text anytime and save again
Example Format
== Product Information ==
What is [Product Name]?
[Product Name] is [description]. It helps customers [benefit].
== Pricing ==
How much does [Product Name] cost?
Our pricing starts at $X/month for the Basic plan...
== Common Questions ==
How do I reset my password?
To reset your password, click...
Storage Management
Your Knowledge Base storage is tracked in real-time across all three methods (website crawls, file uploads, and text entries).
Storage Display
- Progress Bar: Visual indicator of storage usage
- Percentage: Current usage vs. plan limit
- Warning at 80%: Alert when approaching limit
- Detailed Breakdown: See individual file sizes
What Counts Toward Storage?
✅ Counted:
- Uploaded file sizes (PDF, TXT, DOCX)
- Website crawl content size (extracted text)
- Text entries converted to files
❌ Not Counted:
- Response layout templates (separate 20MB limit)
- Vector embeddings (stored separately)
Best Practices
Comprehensive Coverage
Include information about:
- All products and services you offer
- Common customer questions and issues
- Pricing and billing information
- Company policies (refunds, shipping, returns)
- Technical specifications or requirements
- Account management procedures
- Contact information and hours
Keep Content Updated
- Review KB monthly or after major product changes
- Delete outdated files or crawls
- Re-crawl websites after significant updates
- Update text entries when policies change
- Version your content (include dates in file names)
Structure for AI Understanding
- Use clear headings and sections
- Write in Q&A format when possible
- Include keywords customers actually use
- Avoid ambiguous language
- Define acronyms and technical terms
Test and Iterate
- Start with core information
- Monitor escalated emails (questions AI couldn't answer)
- Add missing information to KB
- Test changes by asking sample questions
- Use confidence scores as feedback
Combine Methods
You don't have to choose just one method:
- Website crawl for your help center
- File upload for product manuals
- Text entry for quick FAQs or new information
Troubleshooting
Website Crawling Issues
Problem: Crawl shows "failed" status
- Check if the website is publicly accessible
- Verify the website has proper HTML content
- Try crawling a different page or use file upload instead
Problem: Very few pages crawled
- Check if pages are linked from the homepage
- Enable "Include subdomains" if needed
- Try increasing max pages or crawl specific sections
File Upload Issues
Problem: "File too large" error
- Maximum file size is 10MB per file
- Split large files or compress PDFs
- Remove unnecessary images from PDFs
Problem: "Plan limit reached" error
- Check storage usage in the dashboard
- Delete old files or upgrade plan
- Use text entry as an alternative (minimal storage impact)
Problem: File stuck at "processing"
- Ensure PDFs have extractable text (not scanned images)
- Try saving DOCX files as plain TXT instead
- Wait 1-2 minutes for processing to complete
- Delete and try uploading again if still stuck
General Issues
Problem: AI not using Knowledge Base content
- Check that content is uploaded and processing status = "completed"
- Test with a question directly related to KB content
- Review confidence threshold (very high threshold = more escalations)
- Ensure sufficient information exists in KB for the query
Getting Started Checklist
- Choose your primary KB method (website, files, or text)
- Gather existing support content (docs, FAQs, policies)
- Upload/crawl initial content (start with top 10 customer questions)
- Review processing status (wait for "completed")
- Test with sample customer questions
- Review escalated emails after first week
- Add missing information to KB
- Set regular KB review schedule (monthly)
- Monitor storage usage
- Update content when products/policies change
Next Steps
Need help? Contact Support
Need more help?
Our support team is here to assist you with any questions.