# Dashboard User Guide

Learn how to use the CiteFlow dashboard to monitor AI bot activity and referral traffic.

## Dashboard Overview

The CiteFlow dashboard provides real-time insights into how AI bots interact with your website. It's designed for **AI Bots + AI Referrals + AI Traffic Intelligence**.

## Key Concepts

### Three-Layer Architecture
CiteFlow separates data into three layers for optimal performance:

| Layer | Purpose | Data Source |
|-------|---------|-------------|
| **Raw** | Debugging raw crawler requests | CrawlerLog (30 days) |
| **Analytics** | Business facts for reporting | BotVisit, Referral |
| **Aggregation** | Aggregated statistics | Page, Session |

**Important:** The dashboard queries only Analytics and Aggregation tables for fast performance.

## Pages

### Overview

The main dashboard shows key metrics at a glance:

| Metric | Description |
|--------|-------------|
| AI Crawls | Total bot crawl events in last 24 hours |
| AI Referrals | Visitors from AI platforms |
| New Pages Discovered | Pages first crawled by AI bots |
| Active Bots | Number of different AI bots seen |
| Top AI Sources | Which AI platforms drive the most traffic |

**Activity Feed**
Real-time feed showing:
- Bot crawl events with category (AI_CRAWLER, AI_SEARCH_CRAWLER, etc.)
- Page discoveries
- AI referral visits with source
- New bot detections

### Pages

Track which pages AI bots discover and monitor.

**Page Status**

| Status | Description |
|--------|-------------|
| Hot | Frequently crawled in last 7 days |
| Cooling | Previously active, now declining |
| Stale | Not crawled in 30+ days |
| Never Seen | Never discovered by AI bots |

**Features**
- Search and filter pages by URL
- View crawl count and referral count per page
- See which bots crawl each page
- Track last crawl time and indexed status
- Export page data as CSV

### Bots

Monitor AI crawler activity with detailed insights.

**Supported Bots**
| Bot | Vendor | Category |
|-----|--------|----------|
| GPTBot | OpenAI | AI_CRAWLER |
| ClaudeBot | Anthropic | AI_CRAWLER |
| PerplexityBot | Perplexity | AI_SEARCH_CRAWLER |
| Google-Extended | Google | AI_CRAWLER |
| Gemini | Google | AI_CRAWLER |
| Meta-ExternalAgent | Meta | AI_CRAWLER |
| Bytespider | ByteDance | AI_CRAWLER |

**Metrics**
- Total crawls per bot
- Last visit time
- Trend (increasing/decreasing)
- Recently crawled pages
- Category breakdown

### Referrals

Track visitors coming from AI platforms.

**Metrics**
- Total AI referrals
- Top AI source
- Traffic by source (ChatGPT, Claude, Gemini, etc.)
- Daily referral trends

**Sources Tracked**
- ChatGPT (chat.openai.com, chatgpt.com)
- Claude (claude.ai)
- Perplexity (perplexity.ai)
- Gemini (gemini.google.com)
- Copilot (copilot.microsoft.com)

**Referral Quality**
- Direct referrals vs. search referrals
- Engagement metrics per source
- Conversion tracking (Pro plan+)

### Alerts

Configure notifications for important events.

**Alert Types**

| Alert | Trigger |
|-------|---------|
| New Bot Visit | An AI bot visits your site for the first time |
| New Page Discovered | AI bot finds a page it hasn't seen before |
| Crawl Spike | Crawl activity increases 300%+ in one day |
| Crawl Drop | No crawls for 30 consecutive days |
| Referral Spike | AI referral traffic increases 200%+ |
| New AI Source | Traffic from a new AI platform detected |

**Notification Channels**
- Email (Pro and above)
- Webhook (Agency)
- In-app notifications

### Settings

Manage your account and sites.

**Profile Settings**
- Name and email
- Password
- API keys (Agency)

**Site Management**
- Add new sites
- View site IDs and tracker code
- Delete sites
- Manage site-specific settings

**Notifications**
- Enable/disable alert types
- Set notification preferences
- Configure webhook endpoints (Agency)

**Subscription**
- View current plan
- Upgrade/downgrade
- Billing history
- Plan limits overview

## Insights Menu

### Analytics

Deep dive into AI traffic patterns.

**AI Classification**
- Breakdown by category (Crawler, Search, User Fetch, Referral)
- Vendor comparison (OpenAI vs Anthropic vs Google)
- Confidence levels of detections

**Page Performance**
- Most crawled pages
- Pages with most AI referrals
- Indexed vs non-indexed pages
- Crawl frequency analysis

## Tips for Better AI Discovery

1. **Update sitemap.xml regularly**
   - Helps bots discover new content
   
2. **Use descriptive page titles and meta descriptions**
   - AI models use this data to understand content
   
3. **Interlink related content**
   - Helps bots discover more pages on your site
   
4. **Keep content fresh**
   - Regularly updated pages get more frequent crawls

5. **Optimize page load speed**
   - Faster pages are crawled more efficiently

## Best Practices

### For Content Sites
- Focus on "Never Seen" pages - they need better discoverability
- Update "Stale" content to re-engage AI crawlers
- Monitor "Cooling" pages for optimization opportunities
- Track which content gets the most AI referrals

### For SaaS/Product Sites
- Track documentation pages - critical for AI knowledge bases
- Monitor blog content discovery
- Watch for competitor bot activity
- Optimize landing pages for AI search

### For E-commerce
- Track product page crawl frequency
- Monitor which products get AI referrals
- Optimize category pages for AI discovery
- Use structured data for better AI understanding

## Glossary

| Term | Definition |
|------|------------|
| AI Crawler | Bots that index content for AI training |
| AI Referral | Visitors clicking links from AI chat interfaces |
| AI User Fetch | User-initiated content requests from AI tools |
| Hot Page | Frequently crawled (last 7 days) |
| Stale Page | Not crawled in 30+ days |
| Indexed | Marked as processed by AI systems |

---

Next: [API Reference](./api-reference.md)