Spaces:
Sleeping
GeminiRAG — Project Context
For use by the next chat session to resume work without losing any context.
What This Project Is
GeminiRAG is a production-ready Multimodal RAG (Retrieval-Augmented Generation) pipeline built for MasterCRM Internal Engineering. It allows users to upload any document (PDF, Word, Excel, CSV, image, audio, video), have it processed by Google Gemini, chunked and embedded into a ChromaDB vector store, and then queried in natural language. The system returns answers with citations and RAGAS quality scores.
Delivered by: Dhrumil Parikh
Delivery date: 3 June 2026
GitHub repo: https://github.com/Dhrumilparikh2806/yaya (branch: master)
Tech Stack
| Layer | Technology |
|---|---|
| LLM / Embeddings | Google Gemini (gemini-2.5-flash, models/gemini-embedding-001) via google-genai SDK |
| Agent | Google ADK (google-adk) with 5 MCP tools |
| API | FastAPI 0.111 + uvicorn, Python 3.11 |
| Task Queue | Celery 5.6 + Redis broker, PostgreSQL result backend |
| Vector Store | ChromaDB (HTTP client, cosine similarity) |
| Database | PostgreSQL 18 (native, NOT Docker) — SQLModel + Alembic |
| RAG Evaluation | RAGAS (faithfulness, answer_relevancy, context_precision, context_recall, answer_correctness) |
| Frontend | React 18 + TypeScript + Vite + TailwindCSS + Recharts |
| Observability | structlog (JSON), OpenTelemetry (stdout exporter), UsageLog DB table |
| Auth | JWT (HS256, python-jose) + bcrypt passwords |
| Rate Limiting | slowapi (10/min on /auth/login) |
Environment (Local, No Docker)
Important: Docker is NOT installed on this machine. All services run natively.
| Service | Port | Status | Notes |
|---|---|---|---|
| PostgreSQL 18 | 5432 | Running natively | DB: geminirag, user: geminirag, pass: geminirag |
| Redis | 6379 | Running natively | |
| ChromaDB | 8001 | Running natively | |
| FastAPI (uvicorn) | 8000 | Must be started manually | See "How to Start" below |
| Celery worker | — | Must be started manually | Uses --pool=solo on Windows |
| React (Vite) | 5173 | Must be started manually |
How to Start Everything
Open 3 separate cmd windows (or use PowerShell Start-Process):
Window 1 — API Server:
cd "c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag"
py -m uvicorn app.main:app --reload --port 8000
Window 2 — Celery Worker:
cd "c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag"
py -m celery -A app.workers.celery_app worker --loglevel=info --concurrency=2 --pool=solo
Window 3 — Frontend:
cd "c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag\frontend"
npm run dev
Or via PowerShell (each in its own visible cmd window that stays alive):
Start-Process cmd.exe -ArgumentList "/k", "cd /d `"...\geminirag`" && py -m uvicorn app.main:app --reload --port 8000"
Start-Process cmd.exe -ArgumentList "/k", "cd /d `"...\geminirag\frontend`" && npm run dev"
Verification
# Check health (should return {"status":"ok","database":"ok","chromadb":"ok"})
Invoke-WebRequest http://localhost:8000/health -UseBasicParsing | Select -Expand Content
.env File Location and Contents
Path: c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag\.env
GEMINI_API_KEY=AIzaSyD-0xYBLCksuwdk0oo1SO3S_gdFXW3DFNs
DATABASE_URL=postgresql://geminirag:geminirag@localhost:5432/geminirag
REDIS_URL=redis://localhost:6379/0
SECRET_KEY=geminirag_secret_key_minimum_32_chars_long_secure
UPLOAD_DIR=C:/tmp/geminirag_uploads
GEMINI_MODEL=gemini-2.5-flash
GEMINI_EMBEDDING_MODEL=models/gemini-embedding-001
.env is gitignored — never commit it.
Database
- Engine: PostgreSQL 18, running on localhost:5432
- Database:
geminirag - Tables:
users,jobs,usage_logs,query_history - Migrations: Alembic (
alembic upgrade head) - Connection pooling: pool_size=10, max_overflow=20, pool_pre_ping=True (in
app/models/db.py)
To view in PgAdmin: connect to localhost:5432, user=geminirag, password=geminirag, DB=geminirag.
Admin Credentials
Email: admin@test.com
Password: Admin1234!
Role: admin
To recreate: py scripts/seed_admin.py --email admin@test.com --password Admin1234!
API Overview
Base URL: http://localhost:8000
Docs: http://localhost:8000/docs
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | /auth/register | No | Register user |
| POST | /auth/login | No | Login → JWT token |
| POST | /v1/files/upload | JWT | Upload file → returns job_id (async) |
| GET | /v1/jobs/{id} | JWT | Get job status |
| GET | /v1/jobs | JWT | List all user's jobs (admin sees all) |
| POST | /v1/jobs/{id}/reprocess | JWT | Re-queue failed job |
| GET | /v1/documents | JWT | List completed documents |
| GET | /v1/documents/{id}/summary | JWT | Get document AI summary |
| POST | /v1/query | JWT | RAG query → answer + citations |
| POST | /v1/query/stream | JWT | Streaming RAG via SSE |
| POST | /v1/agent/chat | JWT | ADK agent chat |
| GET | /v1/admin/usage | Admin | Usage stats |
| GET | /v1/admin/ragas | Admin | RAGAS metric averages |
| GET | /v1/admin/users | Admin | User list with stats |
| PATCH | /v1/admin/users/{id} | Admin | Toggle user is_active |
| GET | /health | No | DB + ChromaDB health check |
Login format (JSON body):
{"email": "admin@test.com", "password": "Admin1234!"}
File Processing Pipeline
User uploads file → POST /v1/files/upload
→ creates Job (PENDING) in PostgreSQL
→ saves file to C:/tmp/geminirag_uploads/{job_id}/
→ enqueues process_file.delay(job_id) in Celery via Redis
Celery worker picks up task → process_file(job_id):
1. EXTRACTING → dispatch to processor by file_type
(PDFProcessor / DOCXProcessor / XLSXProcessor /
ImageProcessor / VideoAudioProcessor)
→ processor.extract() → raw text
→ processor.summarise() → Gemini JSON summary
→ stores summary in job.result
2. CHUNKING → chunk_text() or chunk_video_segments()
800 words/chunk, 100-word overlap, min 50 words
3. EMBEDDING → embed_chunks() → Gemini embedding API (768-dim)
batched 100 at a time, retry on 429
4. INDEXING → add_chunks() → ChromaDB upsert with 3x retry
5. COMPLETED → job.status = COMPLETED, job.chunk_count = N
Supported file types: PDF, DOCX, XLSX, CSV, PNG, JPG, JPEG, WEBP, MP4, MOV, MP3, WAV, M4A
Max file size: 500 MB
RAG Query Flow
POST /v1/query {question, job_ids?}
1. Embed question → 768-dim vector (Gemini)
2. Search ChromaDB → top_k=5 chunks (cosine similarity)
3. Confidence gate: avg_score >= 0.65
→ If fails: return "I don't have enough information..."
4. Format context from chunks
5. Call Gemini (gemini-2.5-flash) with RAG system prompt
→ Answer must only use provided context, must cite [1][2]...
6. Log to UsageLog + QueryHistory
7. Enqueue compute_ragas.delay() async (adds ~15-60s, runs in background)
8. Return: {answer, citations, confidence_gate_passed, avg_similarity_score}
RAGAS scores appear later in QueryHistory (populated by background Celery task)
Streaming variant: POST /v1/query/stream uses StreamingResponse with text/event-stream.
Frontend uses Fetch API (not EventSource) because SSE doesn't support POST/auth headers.
Celery Tasks
| Task | Trigger | Purpose |
|---|---|---|
process_file |
File upload | Full extraction → chunk → embed → index pipeline |
compute_ragas |
After each query | Async RAGAS score computation |
cleanup_old_uploads |
Daily (beat schedule) | Delete upload files for 7-day-old completed jobs |
Retry strategy: max_retries=3, exponential backoff (CELERY_RETRY_BACKOFF * 2^retry).
Dead letter queue: FAILED_PERMANENT jobs pushed to Redis list geminirag:dead_letter.
Windows note: Must use --pool=solo flag on Windows.
ADK Agent
The agent has 5 tools and can hold multi-turn conversations:
| Tool | What it does |
|---|---|
ingest_file |
Upload a file by path → creates job, queues processing |
get_job_status |
Check processing status of a job |
query_rag |
Ask a question against uploaded documents |
list_documents |
List all completed documents |
summarize_document |
Get AI summary of a specific document |
Session service: InMemorySessionService — conversation history resets on server restart.
Production note: Should be replaced with Redis/PostgreSQL-backed session service.
RAGAS Evaluation
Target scores (from spec):
- Faithfulness ≥ 0.80
- Context Precision ≥ 0.60
Running baseline offline:
- Ensure documents are uploaded and COMPLETED
- Create test set JSON:
C:/tmp/ragas_test_set.json[{"question": "...", "ground_truth": "...", "job_id": "uuid-here"}] - Run:
py scripts/ragas_baseline.py --test-set C:/tmp/ragas_test_set.json - Results saved to:
C:/tmp/ragas_baseline.json
Pre-downloaded sample datasets (50 Q&A pairs each, no job_id yet):
Data set/ragas_eval/ms_marco_samples.json— MS MARCO v1.1 validationData set/ragas_eval/natural_questions_samples.json— Natural Questions dev
To use these, upload relevant documents first, then add the returned job_id to the JSON entries.
Frontend Pages
| URL | Page | What it does |
|---|---|---|
/login |
LoginPage | Email/password login, stores JWT |
/register |
RegisterPage | New user registration |
/upload |
UploadPage | Drag-drop upload, job polling, summary drawer |
/query |
QueryPage | Select docs, ask question, streaming mode, citation links |
/agent |
AgentPage | Chat with ADK agent, tool call log sidebar |
/jobs |
JobsPage | Full jobs table, re-process button |
/admin |
AdminPage | Usage/RAGAS/user management tabs (admin only) |
All pages lazy-loaded (React.lazy + Suspense) — main bundle ~211KB after code splitting.
Key Design Decisions (for context)
google-genainotgoogle-generativeai— The old SDK (google-generativeai) is deprecated. Always usegoogle-genai>=1.0.0withfrom google import genai.SSE streaming uses Fetch not EventSource — EventSource API doesn't support POST requests or custom headers. The frontend uses
fetch()withReadableStreamreader to stream answers with auth.ChromaDB 3x retry —
add_chunks()retries 3 times with 5s backoff because ChromaDB can have transient write failures.RAGAS is async — Computing RAGAS after every query adds 15-60s latency. It runs in a background Celery task (
compute_ragas). The query response returns immediately; RAGAS scores appear in QueryHistory later.ALLOWED_ORIGINS is env-driven — Set
ALLOWED_ORIGINS=https://your-domain.comin .env for production. In dev it defaults tohttp://localhost:5173.No Docker — All infrastructure runs natively on this machine. PostgreSQL (v18), Redis, ChromaDB are already running as system services.
Windows Celery — Must use
--pool=soloflag. Celery's default prefork pool doesn't work on Windows.
Known Limitations
- Speaker diarization accuracy — depends on audio quality. Mono recordings work best.
- Large video files — >500 MB rejected. Close-to-limit files may hit Gemini context window.
- RAGAS cost — ~15-60s and token cost per query. Disable by removing
compute_ragas.delay()fromapp/rag/engine.py. - In-memory agent sessions — ADK InMemorySessionService resets on server restart.
- ChromaDB not backed up — Lives in Docker named volume. If deleted, re-upload all documents.
- No email notifications — No notification when long jobs complete. Planned future feature.
Adding a New File Type
- Create
app/processors/newtype.pyextendingBaseProcessor— implementextract()andsummarise() - Add extension to
EXTENSION_MAPinapp/api/files.py - Add extension to
EXT_TO_TYPEinfrontend/src/pages/UploadPage.tsx - Add dispatch case in
process_file()inapp/workers/tasks.py - Write a test in
tests/test_processors.py
Git History (recent)
5965edb feat: day 8-10 + buffer — frontend, security hardening, streaming, delivery docs
636fd39 fix: phase-1 checklist gaps — otel spans, from_status logging, user_id in http middleware
48f3cfa day-7: ragas evaluation, observability audit, baseline
39c873f day-6: rag engine, citations, confidence gate, admin api
a8fbf32 day-4: image and video/audio processors, diarization, 14 tests pass
3f24a13 day-3: migrate to google.genai SDK, verify processors end-to-end
Project Directory
playbook_final/
└── geminirag/ ← main project root
├── app/ ← FastAPI backend
│ ├── main.py
│ ├── config.py
│ ├── deps.py
│ ├── security.py
│ ├── limiter.py
│ ├── api/ ← route handlers
│ ├── models/ ← SQLModel ORM
│ ├── processors/ ← file type processors
│ ├── rag/ ← chunker, embedder, vectorstore, engine
│ ├── workers/ ← Celery tasks
│ ├── agent/ ← ADK agent + tools
│ ├── evaluation/ ← RAGAS eval
│ └── observability/ ← logging + tracing
├── frontend/ ← React + TypeScript
│ └── src/
│ ├── pages/ ← 7 pages
│ ├── context/ ← Auth + Toast contexts
│ ├── components/ ← NavBar, PrivateRoute
│ ├── hooks/ ← useToast
│ └── api/ ← Axios client
├── scripts/ ← seed_admin, ragas_baseline, download_datasets
├── tests/ ← pytest test suite
├── migrations/ ← Alembic migration files
├── Data set/ ← test datasets
│ └── ragas_eval/ ← ms_marco_samples.json, natural_questions_samples.json
├── .env ← secrets (gitignored)
├── .env.example ← template
├── pyproject.toml ← Python deps
├── docker-compose.yml ← dev (not used locally)
├── docker-compose.prod.yml
├── Dockerfile
├── alembic.ini
├── README.md
├── HANDOVER.md
└── DEMO_SCRIPT.md