HANDOVER.md

GeminiRAG — Handover Document

Project: GeminiRAG — Multimodal RAG Pipeline
Delivered by: Dhrumil Parikh
Delivery date: 3 June 2026
Client: MasterCRM Internal Engineering

---

How to Run the System

Prerequisites

Requirement	Notes
Python 3.11+	3.13 disables cross-encoder reranker (safe, falls back gracefully)
Node.js 18+	Frontend only
PostgreSQL 16	Native install or Docker
Redis 7+	Docker recommended
ChromaDB 0.5+	Must run as HTTP server on port 8001
Groq API key	Free tier is sufficient for development
Gemini API key	Optional — only needed for /v1/query/stream SSE endpoint

First-time setup

# 1. Copy and fill environment file
cp .env.example .env
# Required keys: GROQ_API_KEY, SECRET_KEY, DATABASE_URL, REDIS_URL
# See .env.example for all options

# 2. Start Redis and ChromaDB
docker compose up -d redis chromadb

# 3. Create the PostgreSQL database (skip if using Docker Compose postgres service)
createdb geminirag && createuser geminirag --password geminirag

# 4. Install Python dependencies
pip install -e .

# 5. Run migrations
alembic upgrade head

# 6. Seed admin user
py scripts/seed_admin.py --email admin@mastercrm.com --password YourSecurePass!

# 7. API server (terminal 1)
py -m uvicorn app.main:app --reload --port 8000

# 8. Celery worker (terminal 2)
py -m celery -A app.workers.celery_app worker --loglevel=info --pool=solo

# 9. Frontend (terminal 3)
cd frontend && npm install && npm run dev
# → http://localhost:5173

Production (Docker)

# Set ALLOWED_ORIGINS=https://your-domain.com in .env first
docker compose -f docker-compose.prod.yml up --build

---

Admin Credentials

# Create or update an admin user
py scripts/seed_admin.py --email demo@mastercrm.com --password Demo2026!

# To reset: delete the row in PostgreSQL and re-run
DELETE FROM users WHERE email = 'demo@mastercrm.com';

---

LLM Configuration

Role	Model	Env var
RAG answer generation	llama-3.3-70b-versatile	GROQ_MODEL
File extraction / summaries / RAGAS	llama-3.1-8b-instant	GROQ_PROCESSING_MODEL
Image OCR / video frames	meta-llama/llama-4-scout-17b-16e-instruct	GROQ_VISION_MODEL
Speech-to-text	whisper-large-v3	WHISPER_MODEL
Streaming query (optional)	gemini-2.0-flash	GEMINI_MODEL
Agent synthesis	llama-3.1-8b-instant	hardcoded in agent/agent.py
Embeddings	BAAI/bge-small-en-v1.5 (local)	EMBEDDING_MODEL

All Groq calls are logged to usage_logs with prompt tokens, completion tokens, and latency.

---

RAGAS Baseline Evaluation

# 1. Create a test set
cat > /tmp/ragas_test_set.json << 'EOF'
[
  {
    "question": "What is the main topic of the document?",
    "ground_truth": "The document covers...",
    "job_id": "<UUID of a completed job>"
  }
]
EOF

# 2. Run baseline
py scripts/ragas_baseline.py --test-set /tmp/ragas_test_set.json
# → /tmp/ragas_baseline.json

Delivery targets: Faithfulness ≥ 0.80 · Answer Relevancy ≥ 0.75 · Context Precision ≥ 0.70

---

Key Source Files

File	Purpose
app/main.py	FastAPI factory, middleware, model warmup
app/config.py	All env vars with startup validation
app/models/db.py	ORM tables (User, Job, UsageLog, QueryHistory)
app/api/files.py	Upload endpoint, file type dispatch
app/api/query.py	RAG query (JSON + SSE streaming)
app/api/admin.py	Usage stats, RAGAS trends, user management
app/processors/base.py	Abstract processor, Groq LLM helpers
app/processors/audio_utils.py	Whisper transcription + SpeechBrain diarization
app/rag/engine.py	Hybrid search, confidence gate, Groq answer
app/rag/chunker.py	Hierarchical (parent/child) chunking
app/rag/vectorstore.py	ChromaDB helpers + RRF merge
app/rag/bm25_index.py	BM25 index (Redis-cached)
app/rag/reranker.py	Cross-encoder reranker
app/agent/agent.py	Intent classification + Groq synthesis
app/agent/tools.py	ingest, status, query, list, summarize tools
app/workers/tasks.py	process_file, compute_ragas, cleanup_old_uploads
app/evaluation/ragas_eval.py	RAGAS metric computation
scripts/seed_admin.py	Create initial admin user
scripts/seed_ragas_scores.py	Seed day-by-day RAGAS demo data
scripts/ragas_baseline.py	Offline RAGAS baseline evaluation

---

Adding a New File Type

1. Create app/processors/newtype.py extending BaseProcessor — implement extract() and summarise().
2. Add the extension(s) to EXTENSION_MAP in app/api/files.py.
3. Add the dispatch case to process_file() in app/workers/tasks.py.
4. Add the extension to the accepted types list in frontend/src/pages/UploadPage.tsx.
5. Add tests in tests/test_processors.py.

---

Job Processing Pipeline

Upload → Job(PENDING) → Celery enqueue
  → PROCESSING / extracting  — processor.extract()
  → PROCESSING / summarising — processor.summarise() + Groq LLM
  → PROCESSING / chunking    — chunk_markdown_hierarchical()
  → PROCESSING / embedding   — embed_chunks() via fastembed (local)
  → PROCESSING / indexing    — ChromaDB upsert + BM25 invalidate
  → COMPLETED (chunk_count set)

On retryable error (rate limit, unknown):
  → FAILED → re-enqueue (60 × 2ⁿ s) → repeat up to 3×
  → FAILED_PERMANENT + Redis dead-letter queue

Speaker embeddings (audio/video only):
  SpeechBrain ECAPA mean embedding per speaker attached as
  speaker_embedding_json metadata on each ChromaDB chunk.

---

Known Limitations

1. Speaker diarization accuracy depends on audio quality. Mono recordings with minimal background noise and clearly distinct voices produce the best results. Overlapping speech is not supported.

2. Large video files (> 500 MB) are rejected at upload. Near-duplicate frame skipping (> 98 % histogram similarity) reduces the number of frames processed.

3. RAGAS token cost — every RAG query triggers a background RAGAS evaluation that calls the Groq LLM again. At high query volumes this can be significant. Disable by removing compute_ragas.delay(str(qh.id)) in app/rag/engine.py.

4. ChromaDB persistence — embeddings live in a Docker named volume. Deleting the volume loses all vectors; documents must be re-uploaded and re-processed. Back up the chromadata Docker volume before infrastructure changes.

5. Agent LLM window — the last 10 conversation turns are sent to Groq; full history is stored in Redis for 7 days but not included in the LLM context after 10 turns.

6. Reranker on Python 3.13+ — disabled by default due to native tokenizer crash. Set GEMINIRAG_RERANKER=1 to force-enable (Python 3.11 / Docker only).

7. Streaming query requires Gemini — POST /v1/query/stream uses the Gemini SDK for SSE streaming. Set GEMINI_API_KEY to use it. The standard POST /v1/query always uses Groq and does not require a Gemini key.