Spaces:
Sleeping
Sleeping
| # GeminiRAG — Project Context | |
| **For use by the next chat session to resume work without losing any context.** | |
| --- | |
| ## What This Project Is | |
| GeminiRAG is a **production-ready Multimodal RAG (Retrieval-Augmented Generation) pipeline** built for MasterCRM Internal Engineering. It allows users to upload any document (PDF, Word, Excel, CSV, image, audio, video), have it processed by Google Gemini, chunked and embedded into a ChromaDB vector store, and then queried in natural language. The system returns answers with citations and RAGAS quality scores. | |
| **Delivered by:** Dhrumil Parikh | |
| **Delivery date:** 3 June 2026 | |
| **GitHub repo:** https://github.com/Dhrumilparikh2806/yaya (branch: master) | |
| --- | |
| ## Tech Stack | |
| | Layer | Technology | | |
| |---|---| | |
| | LLM / Embeddings | Google Gemini (`gemini-2.5-flash`, `models/gemini-embedding-001`) via `google-genai` SDK | | |
| | Agent | Google ADK (`google-adk`) with 5 MCP tools | | |
| | API | FastAPI 0.111 + uvicorn, Python 3.11 | | |
| | Task Queue | Celery 5.6 + Redis broker, PostgreSQL result backend | | |
| | Vector Store | ChromaDB (HTTP client, cosine similarity) | | |
| | Database | PostgreSQL 18 (native, NOT Docker) — SQLModel + Alembic | | |
| | RAG Evaluation | RAGAS (faithfulness, answer_relevancy, context_precision, context_recall, answer_correctness) | | |
| | Frontend | React 18 + TypeScript + Vite + TailwindCSS + Recharts | | |
| | Observability | structlog (JSON), OpenTelemetry (stdout exporter), UsageLog DB table | | |
| | Auth | JWT (HS256, python-jose) + bcrypt passwords | | |
| | Rate Limiting | slowapi (10/min on /auth/login) | | |
| --- | |
| ## Environment (Local, No Docker) | |
| **Important:** Docker is NOT installed on this machine. All services run natively. | |
| | Service | Port | Status | Notes | | |
| |---|---|---|---| | |
| | PostgreSQL 18 | 5432 | Running natively | DB: `geminirag`, user: `geminirag`, pass: `geminirag` | | |
| | Redis | 6379 | Running natively | | | |
| | ChromaDB | 8001 | Running natively | | | |
| | FastAPI (uvicorn) | 8000 | Must be started manually | See "How to Start" below | | |
| | Celery worker | — | Must be started manually | Uses `--pool=solo` on Windows | | |
| | React (Vite) | 5173 | Must be started manually | | | |
| ### How to Start Everything | |
| Open 3 separate cmd windows (or use PowerShell `Start-Process`): | |
| **Window 1 — API Server:** | |
| ``` | |
| cd "c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag" | |
| py -m uvicorn app.main:app --reload --port 8000 | |
| ``` | |
| **Window 2 — Celery Worker:** | |
| ``` | |
| cd "c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag" | |
| py -m celery -A app.workers.celery_app worker --loglevel=info --concurrency=2 --pool=solo | |
| ``` | |
| **Window 3 — Frontend:** | |
| ``` | |
| cd "c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag\frontend" | |
| npm run dev | |
| ``` | |
| Or via PowerShell (each in its own visible cmd window that stays alive): | |
| ```powershell | |
| Start-Process cmd.exe -ArgumentList "/k", "cd /d `"...\geminirag`" && py -m uvicorn app.main:app --reload --port 8000" | |
| Start-Process cmd.exe -ArgumentList "/k", "cd /d `"...\geminirag\frontend`" && npm run dev" | |
| ``` | |
| ### Verification | |
| ```powershell | |
| # Check health (should return {"status":"ok","database":"ok","chromadb":"ok"}) | |
| Invoke-WebRequest http://localhost:8000/health -UseBasicParsing | Select -Expand Content | |
| ``` | |
| --- | |
| ## .env File Location and Contents | |
| Path: `c:\Users\Dhrumil.parikh\OneDrive - Taazaa Tech Pvt Ltd\Desktop\playbook_final\geminirag\.env` | |
| ``` | |
| GEMINI_API_KEY=AIzaSyD-0xYBLCksuwdk0oo1SO3S_gdFXW3DFNs | |
| DATABASE_URL=postgresql://geminirag:geminirag@localhost:5432/geminirag | |
| REDIS_URL=redis://localhost:6379/0 | |
| SECRET_KEY=geminirag_secret_key_minimum_32_chars_long_secure | |
| UPLOAD_DIR=C:/tmp/geminirag_uploads | |
| GEMINI_MODEL=gemini-2.5-flash | |
| GEMINI_EMBEDDING_MODEL=models/gemini-embedding-001 | |
| ``` | |
| `.env` is gitignored — never commit it. | |
| --- | |
| ## Database | |
| - **Engine:** PostgreSQL 18, running on localhost:5432 | |
| - **Database:** `geminirag` | |
| - **Tables:** `users`, `jobs`, `usage_logs`, `query_history` | |
| - **Migrations:** Alembic (`alembic upgrade head`) | |
| - **Connection pooling:** pool_size=10, max_overflow=20, pool_pre_ping=True (in `app/models/db.py`) | |
| To view in PgAdmin: connect to localhost:5432, user=geminirag, password=geminirag, DB=geminirag. | |
| --- | |
| ## Admin Credentials | |
| ``` | |
| Email: admin@test.com | |
| Password: Admin1234! | |
| Role: admin | |
| ``` | |
| To recreate: `py scripts/seed_admin.py --email admin@test.com --password Admin1234!` | |
| --- | |
| ## API Overview | |
| Base URL: `http://localhost:8000` | |
| Docs: `http://localhost:8000/docs` | |
| | Method | Path | Auth | Description | | |
| |---|---|---|---| | |
| | POST | /auth/register | No | Register user | | |
| | POST | /auth/login | No | Login → JWT token | | |
| | POST | /v1/files/upload | JWT | Upload file → returns job_id (async) | | |
| | GET | /v1/jobs/{id} | JWT | Get job status | | |
| | GET | /v1/jobs | JWT | List all user's jobs (admin sees all) | | |
| | POST | /v1/jobs/{id}/reprocess | JWT | Re-queue failed job | | |
| | GET | /v1/documents | JWT | List completed documents | | |
| | GET | /v1/documents/{id}/summary | JWT | Get document AI summary | | |
| | POST | /v1/query | JWT | RAG query → answer + citations | | |
| | POST | /v1/query/stream | JWT | Streaming RAG via SSE | | |
| | POST | /v1/agent/chat | JWT | ADK agent chat | | |
| | GET | /v1/admin/usage | Admin | Usage stats | | |
| | GET | /v1/admin/ragas | Admin | RAGAS metric averages | | |
| | GET | /v1/admin/users | Admin | User list with stats | | |
| | PATCH | /v1/admin/users/{id} | Admin | Toggle user is_active | | |
| | GET | /health | No | DB + ChromaDB health check | | |
| **Login format** (JSON body): | |
| ```json | |
| {"email": "admin@test.com", "password": "Admin1234!"} | |
| ``` | |
| --- | |
| ## File Processing Pipeline | |
| ``` | |
| User uploads file → POST /v1/files/upload | |
| → creates Job (PENDING) in PostgreSQL | |
| → saves file to C:/tmp/geminirag_uploads/{job_id}/ | |
| → enqueues process_file.delay(job_id) in Celery via Redis | |
| Celery worker picks up task → process_file(job_id): | |
| 1. EXTRACTING → dispatch to processor by file_type | |
| (PDFProcessor / DOCXProcessor / XLSXProcessor / | |
| ImageProcessor / VideoAudioProcessor) | |
| → processor.extract() → raw text | |
| → processor.summarise() → Gemini JSON summary | |
| → stores summary in job.result | |
| 2. CHUNKING → chunk_text() or chunk_video_segments() | |
| 800 words/chunk, 100-word overlap, min 50 words | |
| 3. EMBEDDING → embed_chunks() → Gemini embedding API (768-dim) | |
| batched 100 at a time, retry on 429 | |
| 4. INDEXING → add_chunks() → ChromaDB upsert with 3x retry | |
| 5. COMPLETED → job.status = COMPLETED, job.chunk_count = N | |
| ``` | |
| **Supported file types:** PDF, DOCX, XLSX, CSV, PNG, JPG, JPEG, WEBP, MP4, MOV, MP3, WAV, M4A | |
| **Max file size:** 500 MB | |
| --- | |
| ## RAG Query Flow | |
| ``` | |
| POST /v1/query {question, job_ids?} | |
| 1. Embed question → 768-dim vector (Gemini) | |
| 2. Search ChromaDB → top_k=5 chunks (cosine similarity) | |
| 3. Confidence gate: avg_score >= 0.65 | |
| → If fails: return "I don't have enough information..." | |
| 4. Format context from chunks | |
| 5. Call Gemini (gemini-2.5-flash) with RAG system prompt | |
| → Answer must only use provided context, must cite [1][2]... | |
| 6. Log to UsageLog + QueryHistory | |
| 7. Enqueue compute_ragas.delay() async (adds ~15-60s, runs in background) | |
| 8. Return: {answer, citations, confidence_gate_passed, avg_similarity_score} | |
| RAGAS scores appear later in QueryHistory (populated by background Celery task) | |
| ``` | |
| Streaming variant: `POST /v1/query/stream` uses `StreamingResponse` with `text/event-stream`. | |
| Frontend uses Fetch API (not EventSource) because SSE doesn't support POST/auth headers. | |
| --- | |
| ## Celery Tasks | |
| | Task | Trigger | Purpose | | |
| |---|---|---| | |
| | `process_file` | File upload | Full extraction → chunk → embed → index pipeline | | |
| | `compute_ragas` | After each query | Async RAGAS score computation | | |
| | `cleanup_old_uploads` | Daily (beat schedule) | Delete upload files for 7-day-old completed jobs | | |
| **Retry strategy:** max_retries=3, exponential backoff (CELERY_RETRY_BACKOFF * 2^retry). | |
| **Dead letter queue:** FAILED_PERMANENT jobs pushed to Redis list `geminirag:dead_letter`. | |
| **Windows note:** Must use `--pool=solo` flag on Windows. | |
| --- | |
| ## ADK Agent | |
| The agent has 5 tools and can hold multi-turn conversations: | |
| | Tool | What it does | | |
| |---|---| | |
| | `ingest_file` | Upload a file by path → creates job, queues processing | | |
| | `get_job_status` | Check processing status of a job | | |
| | `query_rag` | Ask a question against uploaded documents | | |
| | `list_documents` | List all completed documents | | |
| | `summarize_document` | Get AI summary of a specific document | | |
| **Session service:** InMemorySessionService — conversation history resets on server restart. | |
| **Production note:** Should be replaced with Redis/PostgreSQL-backed session service. | |
| --- | |
| ## RAGAS Evaluation | |
| **Target scores (from spec):** | |
| - Faithfulness ≥ 0.80 | |
| - Context Precision ≥ 0.60 | |
| **Running baseline offline:** | |
| 1. Ensure documents are uploaded and COMPLETED | |
| 2. Create test set JSON: `C:/tmp/ragas_test_set.json` | |
| ```json | |
| [{"question": "...", "ground_truth": "...", "job_id": "uuid-here"}] | |
| ``` | |
| 3. Run: `py scripts/ragas_baseline.py --test-set C:/tmp/ragas_test_set.json` | |
| 4. Results saved to: `C:/tmp/ragas_baseline.json` | |
| **Pre-downloaded sample datasets** (50 Q&A pairs each, no job_id yet): | |
| - `Data set/ragas_eval/ms_marco_samples.json` — MS MARCO v1.1 validation | |
| - `Data set/ragas_eval/natural_questions_samples.json` — Natural Questions dev | |
| To use these, upload relevant documents first, then add the returned `job_id` to the JSON entries. | |
| --- | |
| ## Frontend Pages | |
| | URL | Page | What it does | | |
| |---|---|---| | |
| | `/login` | LoginPage | Email/password login, stores JWT | | |
| | `/register` | RegisterPage | New user registration | | |
| | `/upload` | UploadPage | Drag-drop upload, job polling, summary drawer | | |
| | `/query` | QueryPage | Select docs, ask question, streaming mode, citation links | | |
| | `/agent` | AgentPage | Chat with ADK agent, tool call log sidebar | | |
| | `/jobs` | JobsPage | Full jobs table, re-process button | | |
| | `/admin` | AdminPage | Usage/RAGAS/user management tabs (admin only) | | |
| **All pages lazy-loaded** (React.lazy + Suspense) — main bundle ~211KB after code splitting. | |
| --- | |
| ## Key Design Decisions (for context) | |
| 1. **`google-genai` not `google-generativeai`** — The old SDK (`google-generativeai`) is deprecated. Always use `google-genai>=1.0.0` with `from google import genai`. | |
| 2. **SSE streaming uses Fetch not EventSource** — EventSource API doesn't support POST requests or custom headers. The frontend uses `fetch()` with `ReadableStream` reader to stream answers with auth. | |
| 3. **ChromaDB 3x retry** — `add_chunks()` retries 3 times with 5s backoff because ChromaDB can have transient write failures. | |
| 4. **RAGAS is async** — Computing RAGAS after every query adds 15-60s latency. It runs in a background Celery task (`compute_ragas`). The query response returns immediately; RAGAS scores appear in QueryHistory later. | |
| 5. **ALLOWED_ORIGINS is env-driven** — Set `ALLOWED_ORIGINS=https://your-domain.com` in .env for production. In dev it defaults to `http://localhost:5173`. | |
| 6. **No Docker** — All infrastructure runs natively on this machine. PostgreSQL (v18), Redis, ChromaDB are already running as system services. | |
| 7. **Windows Celery** — Must use `--pool=solo` flag. Celery's default prefork pool doesn't work on Windows. | |
| --- | |
| ## Known Limitations | |
| 1. **Speaker diarization accuracy** — depends on audio quality. Mono recordings work best. | |
| 2. **Large video files** — >500 MB rejected. Close-to-limit files may hit Gemini context window. | |
| 3. **RAGAS cost** — ~15-60s and token cost per query. Disable by removing `compute_ragas.delay()` from `app/rag/engine.py`. | |
| 4. **In-memory agent sessions** — ADK InMemorySessionService resets on server restart. | |
| 5. **ChromaDB not backed up** — Lives in Docker named volume. If deleted, re-upload all documents. | |
| 6. **No email notifications** — No notification when long jobs complete. Planned future feature. | |
| --- | |
| ## Adding a New File Type | |
| 1. Create `app/processors/newtype.py` extending `BaseProcessor` — implement `extract()` and `summarise()` | |
| 2. Add extension to `EXTENSION_MAP` in `app/api/files.py` | |
| 3. Add extension to `EXT_TO_TYPE` in `frontend/src/pages/UploadPage.tsx` | |
| 4. Add dispatch case in `process_file()` in `app/workers/tasks.py` | |
| 5. Write a test in `tests/test_processors.py` | |
| --- | |
| ## Git History (recent) | |
| ``` | |
| 5965edb feat: day 8-10 + buffer — frontend, security hardening, streaming, delivery docs | |
| 636fd39 fix: phase-1 checklist gaps — otel spans, from_status logging, user_id in http middleware | |
| 48f3cfa day-7: ragas evaluation, observability audit, baseline | |
| 39c873f day-6: rag engine, citations, confidence gate, admin api | |
| a8fbf32 day-4: image and video/audio processors, diarization, 14 tests pass | |
| 3f24a13 day-3: migrate to google.genai SDK, verify processors end-to-end | |
| ``` | |
| --- | |
| ## Project Directory | |
| ``` | |
| playbook_final/ | |
| └── geminirag/ ← main project root | |
| ├── app/ ← FastAPI backend | |
| │ ├── main.py | |
| │ ├── config.py | |
| │ ├── deps.py | |
| │ ├── security.py | |
| │ ├── limiter.py | |
| │ ├── api/ ← route handlers | |
| │ ├── models/ ← SQLModel ORM | |
| │ ├── processors/ ← file type processors | |
| │ ├── rag/ ← chunker, embedder, vectorstore, engine | |
| │ ├── workers/ ← Celery tasks | |
| │ ├── agent/ ← ADK agent + tools | |
| │ ├── evaluation/ ← RAGAS eval | |
| │ └── observability/ ← logging + tracing | |
| ├── frontend/ ← React + TypeScript | |
| │ └── src/ | |
| │ ├── pages/ ← 7 pages | |
| │ ├── context/ ← Auth + Toast contexts | |
| │ ├── components/ ← NavBar, PrivateRoute | |
| │ ├── hooks/ ← useToast | |
| │ └── api/ ← Axios client | |
| ├── scripts/ ← seed_admin, ragas_baseline, download_datasets | |
| ├── tests/ ← pytest test suite | |
| ├── migrations/ ← Alembic migration files | |
| ├── Data set/ ← test datasets | |
| │ └── ragas_eval/ ← ms_marco_samples.json, natural_questions_samples.json | |
| ├── .env ← secrets (gitignored) | |
| ├── .env.example ← template | |
| ├── pyproject.toml ← Python deps | |
| ├── docker-compose.yml ← dev (not used locally) | |
| ├── docker-compose.prod.yml | |
| ├── Dockerfile | |
| ├── alembic.ini | |
| ├── README.md | |
| ├── HANDOVER.md | |
| └── DEMO_SCRIPT.md | |
| ``` | |