# RAG Platform Operator Guide

## Overview

This guide explains how to operate the hardened RAG SaaS platform after implementing the 7 critical fixes.

## For Tenants

### 1. Creating a Tenant Account

```bash
curl -X POST http://localhost:7860/v1/auth/tenants \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Acme Corporation",
    "admin_email": "admin@acme.com",
    "admin_password": "secure_password"
  }'
```

**Response:**
```json
{
  "tenant_id": "abc123",
  "tenant_name": "Acme Corporation",
  "message": "Tenant created successfully. Use tenant_id 'abc123' for all future logins."
}
```

**IMPORTANT:** Save your `tenant_id` - you'll need it for all future logins.

---

### 2. Logging In

```bash
curl -X POST http://localhost:7860/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "tenant_id": "abc123",
    "username": "admin@acme.com",
    "password": "secure_password"
  }'
```

**Response:**
```json
{
  "access_token": "abc123:admin@acme.com:token...",
  "token_type": "bearer",
  "tenant_id": "abc123",
  "tenant_name": "Acme Corporation"
}
```

---

### 3. Adding Sources

#### Document Upload
```bash
curl -X POST http://localhost:7860/v1/sources/ \
  -H "X-Tenant-ID: abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "doc1",
    "tenant_id": "abc123",
    "type": "file",
    "name": "Company Handbook",
    "uri": "/uploads/handbook.pdf",
    "metadata": {}
  }'
```

#### Web URL
```bash
curl -X POST http://localhost:7860/v1/sources/ \
  -H "X-Tenant-ID: abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "url1",
    "tenant_id": "abc123",
    "type": "web",
    "name": "Company Blog",
    "uri": "https://acme.com/blog",
    "metadata": {}
  }'
```

#### YouTube Video (with transcript upload)
```bash
curl -X POST http://localhost:7860/v1/sources/youtube \
  -H "X-Tenant-ID: abc123" \
  -F "youtube_url=https://youtube.com/watch?v=abc123" \
  -F "name=Product Demo" \
  -F "owner_attestation=true" \
  -F "transcript_file=@transcript.vtt"
```

**Note:** If you don't have a transcript file, the system will attempt to use YouTube captions.

---

### 4. Managing Sources

#### List All Sources
```bash
curl -X GET http://localhost:7860/v1/sources/ \
  -H "X-Tenant-ID: abc123"
```

#### List Active Sources Only
```bash
curl -X GET http://localhost:7860/v1/sources/active \
  -H "X-Tenant-ID: abc123"
```

#### Enable/Disable a Source
```bash
# Disable a source
curl -X PUT "http://localhost:7860/v1/sources/doc1/toggle?enabled=false" \
  -H "X-Tenant-ID: abc123"

# Re-enable a source
curl -X PUT "http://localhost:7860/v1/sources/doc1/toggle?enabled=true" \
  -H "X-Tenant-ID: abc123"
```

**Important:** Toggling sources is **additive**, not exclusive. Disabling one source does NOT affect others.

---

### 5. Asking Questions

```bash
curl -X POST http://localhost:7860/v1/ask/ \
  -H "X-Tenant-ID: abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "What is our return policy?",
    "top_k": 10
  }'
```

**Response:**
```json
{
  "answer": "According to our company handbook...",
  "citations": [
    {
      "source_type": "file",
      "document_name": "Company Handbook",
      "page": 15
    },
    {
      "source_type": "web",
      "url": "https://acme.com/blog/returns"
    }
  ],
  "confidence": 0.92,
  "sources_used": ["file", "web"],
  "retrieval_time_ms": 45.2,
  "generation_time_ms": 320.5
}
```

**Note:** The system automatically retrieves from ALL active sources (documents + URLs + YouTube).

---

### 6. Viewing Your Metrics

```bash
curl -X GET "http://localhost:7860/v1/tenants/abc123/metrics?hours=24" \
  -H "X-Tenant-ID: abc123"
```

**Response:**
```json
{
  "tenant_id": "abc123",
  "counters": {
    "retrieval_total": 150,
    "generation_total": 150,
    "ingestion_total": 25,
    "sources_created": 10
  },
  "time_series": {
    "retrieval_latency_ms": [
      {"timestamp": "2026-02-10T10:00:00", "value": 45.2, "labels": {}},
      ...
    ]
  }
}
```

**Security:** You can only view your own tenant's metrics.

---

## For Administrators

### 1. Viewing Global Metrics

```bash
curl -X GET http://localhost:7860/v1/admin/metrics
```

**Response:**
```json
{
  "global": {
    "counters": {
      "retrieval_total": 5000,
      "generation_total": 5000,
      "ingestion_total": 500
    },
    "total_tenants": 25
  },
  "by_tenant": [
    {
      "tenant_id": "abc123",
      "metrics": {
        "retrieval_total": 150,
        "generation_total": 150
      }
    },
    ...
  ]
}
```

---

### 2. Viewing Model Cache Statistics

```bash
curl -X GET http://localhost:7860/v1/metrics/cache-stats
```

**Response:**
```json
{
  "cache_size": 450,
  "cache_max_size": 1000,
  "cache_hits": 2500,
  "cache_misses": 800,
  "hit_rate": 0.757,
  "model_load_time_seconds": 12.5
}
```

---

## System Architecture

### Multi-Source Retrieval Flow

```
User Query
    ↓
Query Rewrite (optional)
    ↓
Parallel Retrieval:
    ├─ Documents Search
    ├─ URLs Search
    └─ YouTube Search
    ↓
Score Normalization
    ↓
RRF Fusion (Reciprocal Rank Fusion)
    ↓
Diversity Filter
    ↓
Top K Results
    ↓
Answer Generation with Citations
```

### Model Lifecycle

- **Startup:** Models load once during server startup
- **Runtime:** All requests reuse the same model instances
- **Caching:** Embeddings are cached with LRU eviction
- **Performance:** Typical latency reduction: 70-90%

---

## Troubleshooting

### "Tenant not found" error
- Verify you're using the correct `tenant_id` from account creation
- Check that you're including the `X-Tenant-ID` header

### No results from YouTube sources
- Verify the video has captions available, or upload a transcript file
- Check that the source is enabled: `GET /v1/sources/active`

### Slow first request
- This is normal - models load on first use
- Subsequent requests will be much faster due to caching

### Missing source types in results
- Verify all sources are enabled: `GET /v1/sources/active`
- Check that content has been ingested for each source

---

## Best Practices

1. **Source Management:**
   - Keep sources enabled unless you have a specific reason to disable them
   - Use descriptive names for sources to aid in citation tracking

2. **Query Optimization:**
   - Use `top_k=10-20` for best balance of quality and speed
   - More sources = slightly higher latency but better coverage

3. **Metrics Monitoring:**
   - Check metrics regularly to understand usage patterns
   - Monitor `retrieval_latency_ms` for performance issues

4. **YouTube Ingestion:**
   - Always provide transcript files when available for best quality
   - Set `owner_attestation=true` to confirm you have rights to the content

---

## API Endpoint Reference

| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/v1/auth/tenants` | Create tenant account |
| POST | `/v1/auth/login` | Login with tenant_id |
| POST | `/v1/sources/` | Create source |
| GET | `/v1/sources/` | List all sources |
| GET | `/v1/sources/active` | List active sources |
| PUT | `/v1/sources/{id}/toggle` | Enable/disable source |
| POST | `/v1/sources/youtube` | Create YouTube source |
| POST | `/v1/ask/` | Ask question |
| GET | `/v1/tenants/{id}/metrics` | Get tenant metrics |
| GET | `/v1/admin/metrics` | Get global metrics (admin) |
| GET | `/v1/metrics/cache-stats` | Get cache statistics |

---

## Support

For issues or questions, contact your system administrator.