OPERATOR_GUIDE.md

RAG Platform Operator Guide

Overview

This guide explains how to operate the hardened RAG SaaS platform after implementing the 7 critical fixes.

For Tenants

1. Creating a Tenant Account

curl -X POST http://localhost:7860/v1/auth/tenants \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Acme Corporation",
    "admin_email": "admin@acme.com",
    "admin_password": "secure_password"
  }'

Response:

{
  "tenant_id": "abc123",
  "tenant_name": "Acme Corporation",
  "message": "Tenant created successfully. Use tenant_id 'abc123' for all future logins."
}

IMPORTANT: Save your tenant_id - you'll need it for all future logins.

---

2. Logging In

curl -X POST http://localhost:7860/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "tenant_id": "abc123",
    "username": "admin@acme.com",
    "password": "secure_password"
  }'

Response:

{
  "access_token": "abc123:admin@acme.com:token...",
  "token_type": "bearer",
  "tenant_id": "abc123",
  "tenant_name": "Acme Corporation"
}

---

3. Adding Sources

Document Upload

curl -X POST http://localhost:7860/v1/sources/ \
  -H "X-Tenant-ID: abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "doc1",
    "tenant_id": "abc123",
    "type": "file",
    "name": "Company Handbook",
    "uri": "/uploads/handbook.pdf",
    "metadata": {}
  }'

Web URL

curl -X POST http://localhost:7860/v1/sources/ \
  -H "X-Tenant-ID: abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "url1",
    "tenant_id": "abc123",
    "type": "web",
    "name": "Company Blog",
    "uri": "https://acme.com/blog",
    "metadata": {}
  }'

YouTube Video (with transcript upload)

curl -X POST http://localhost:7860/v1/sources/youtube \
  -H "X-Tenant-ID: abc123" \
  -F "youtube_url=https://youtube.com/watch?v=abc123" \
  -F "name=Product Demo" \
  -F "owner_attestation=true" \
  -F "transcript_file=@transcript.vtt"

Note: If you don't have a transcript file, the system will attempt to use YouTube captions.

---

4. Managing Sources

List All Sources

curl -X GET http://localhost:7860/v1/sources/ \
  -H "X-Tenant-ID: abc123"

List Active Sources Only

curl -X GET http://localhost:7860/v1/sources/active \
  -H "X-Tenant-ID: abc123"

Enable/Disable a Source

# Disable a source
curl -X PUT "http://localhost:7860/v1/sources/doc1/toggle?enabled=false" \
  -H "X-Tenant-ID: abc123"

# Re-enable a source
curl -X PUT "http://localhost:7860/v1/sources/doc1/toggle?enabled=true" \
  -H "X-Tenant-ID: abc123"

Important: Toggling sources is additive, not exclusive. Disabling one source does NOT affect others.

---

5. Asking Questions

curl -X POST http://localhost:7860/v1/ask/ \
  -H "X-Tenant-ID: abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "What is our return policy?",
    "top_k": 10
  }'

Response:

{
  "answer": "According to our company handbook...",
  "citations": [
    {
      "source_type": "file",
      "document_name": "Company Handbook",
      "page": 15
    },
    {
      "source_type": "web",
      "url": "https://acme.com/blog/returns"
    }
  ],
  "confidence": 0.92,
  "sources_used": ["file", "web"],
  "retrieval_time_ms": 45.2,
  "generation_time_ms": 320.5
}

Note: The system automatically retrieves from ALL active sources (documents + URLs + YouTube).

---

6. Viewing Your Metrics

curl -X GET "http://localhost:7860/v1/tenants/abc123/metrics?hours=24" \
  -H "X-Tenant-ID: abc123"

Response:

{
  "tenant_id": "abc123",
  "counters": {
    "retrieval_total": 150,
    "generation_total": 150,
    "ingestion_total": 25,
    "sources_created": 10
  },
  "time_series": {
    "retrieval_latency_ms": [
      {"timestamp": "2026-02-10T10:00:00", "value": 45.2, "labels": {}},
      ...
    ]
  }
}

Security: You can only view your own tenant's metrics.

---

For Administrators

1. Viewing Global Metrics

curl -X GET http://localhost:7860/v1/admin/metrics

Response:

{
  "global": {
    "counters": {
      "retrieval_total": 5000,
      "generation_total": 5000,
      "ingestion_total": 500
    },
    "total_tenants": 25
  },
  "by_tenant": [
    {
      "tenant_id": "abc123",
      "metrics": {
        "retrieval_total": 150,
        "generation_total": 150
      }
    },
    ...
  ]
}

---

2. Viewing Model Cache Statistics

curl -X GET http://localhost:7860/v1/metrics/cache-stats

Response:

{
  "cache_size": 450,
  "cache_max_size": 1000,
  "cache_hits": 2500,
  "cache_misses": 800,
  "hit_rate": 0.757,
  "model_load_time_seconds": 12.5
}

---

System Architecture

Multi-Source Retrieval Flow

User Query
    ↓
Query Rewrite (optional)
    ↓
Parallel Retrieval:
    ├─ Documents Search
    ├─ URLs Search
    └─ YouTube Search
    ↓
Score Normalization
    ↓
RRF Fusion (Reciprocal Rank Fusion)
    ↓
Diversity Filter
    ↓
Top K Results
    ↓
Answer Generation with Citations

Model Lifecycle

• Startup: Models load once during server startup
• Runtime: All requests reuse the same model instances
• Caching: Embeddings are cached with LRU eviction
• Performance: Typical latency reduction: 70-90%

---

Troubleshooting

"Tenant not found" error

• Verify you're using the correct tenant_id from account creation
• Check that you're including the X-Tenant-ID header

No results from YouTube sources

• Verify the video has captions available, or upload a transcript file
• Check that the source is enabled: GET /v1/sources/active

Slow first request

• This is normal - models load on first use
• Subsequent requests will be much faster due to caching

Missing source types in results

• Verify all sources are enabled: GET /v1/sources/active
• Check that content has been ingested for each source

---

Best Practices

1. Source Management:

   • Keep sources enabled unless you have a specific reason to disable them
   • Use descriptive names for sources to aid in citation tracking

2. Query Optimization:

   • Use top_k=10-20 for best balance of quality and speed
   • More sources = slightly higher latency but better coverage

3. Metrics Monitoring:

   • Check metrics regularly to understand usage patterns
   • Monitor retrieval_latency_ms for performance issues

4. YouTube Ingestion:

   • Always provide transcript files when available for best quality
   • Set owner_attestation=true to confirm you have rights to the content

---

API Endpoint Reference

Method	Endpoint	Description
POST	/v1/auth/tenants	Create tenant account
POST	/v1/auth/login	Login with tenant_id
POST	/v1/sources/	Create source
GET	/v1/sources/	List all sources
GET	/v1/sources/active	List active sources
PUT	/v1/sources/{id}/toggle	Enable/disable source
POST	/v1/sources/youtube	Create YouTube source
POST	/v1/ask/	Ask question
GET	/v1/tenants/{id}/metrics	Get tenant metrics
GET	/v1/admin/metrics	Get global metrics (admin)
GET	/v1/metrics/cache-stats	Get cache statistics

---

Support

For issues or questions, contact your system administrator.