Spaces:
Running
Running
| # Product Requirements Document — FastAPI Multi-Agent + RAG Service | |
| | | | | |
| |---|---| | |
| | **Product** | FastAPI Multi-Agent + RAG backend ("Compass" API) | | |
| | **Status** | Proof of concept, deployed on Hugging Face Spaces (Docker) | | |
| | **Version** | 1.0 | | |
| | **Last updated** | 2026-07-02 | | |
| | **Owner** | Vince | | |
| --- | |
| ## 1. Overview | |
| A single FastAPI backend that serves as the AI/ML engine for a web frontend | |
| (Next.js/Angular apps hosted on Vercel). It does three things: | |
| 1. **Multi-agent chat** — routes a user's natural-language question to the best | |
| specialist agent: web search, RAG over the user's uploaded documents, | |
| SQL over a business database, or image generation. | |
| 2. **Document memory (RAG)** — lets users upload PDFs or URLs, chunks and | |
| embeds them, and answers questions grounded in those documents with | |
| per-user isolation. | |
| 3. **ML/DL model catalog** — ~28 classic machine-learning and deep-learning | |
| endpoints (regression, classification, clustering, association rules, | |
| recommenders, CNN/RNN, sentiment) trained from bundled notebooks and CSVs. | |
| Access is gated: every endpoint requires a valid Supabase JWT **and** an | |
| admin-approved account. | |
| ### Problem statement | |
| Users need one place to ask questions that may require very different | |
| capabilities — "what's in my contract PDF?", "what were sales last month?", | |
| "what's trending today?", "draw me a logo". Wiring each capability into a | |
| frontend separately is slow and duplicates auth, caching, memory, and logging. | |
| This service centralizes routing, guardrails, and observability behind one | |
| chat endpoint, plus a library of ready-made ML demos. | |
| --- | |
| ## 2. Goals and non-goals | |
| ### Goals | |
| - **G1** — Answer questions from the right specialist agent with a single | |
| `POST /multi_agent_chat` call; no client-side routing logic. | |
| - **G2** — Grounded, per-user document Q&A: answers cite retrieved chunks and | |
| are scored for groundedness; User A can never see User B's documents. | |
| - **G3** — Keep median chat latency low via Redis response caching, embedding | |
| caching, and bounded retrieval (`TOP_K`/`FETCH_K`). | |
| - **G4** — Gate the service behind admin approval so a public deployment | |
| (HF Spaces) can't be used by unapproved signups. | |
| - **G5** — Full observability: chat history, token usage, feedback, and login | |
| events persisted for later evaluation. | |
| - **G6** — Showcase a broad catalog of classic ML/DL techniques as callable | |
| REST endpoints for demo/learning purposes. | |
| ### Non-goals | |
| - No frontend/UI (owned by the separate Vercel apps). | |
| - No user self-service signup approval — approval is manual by an admin. | |
| - No fine-tuning or model training at request time beyond the bundled demo | |
| models. | |
| - No multi-tenant SQL agent — it targets one configured business database. | |
| - No horizontal-scale/HA guarantees; this is a proof of concept on a single | |
| container. | |
| --- | |
| ## 3. Users and personas | |
| | Persona | Description | Needs | | |
| |---------|-------------|-------| | |
| | **End user (approved)** | Uses the chat UI on the Vercel frontend | Ask anything; upload PDFs/URLs; get grounded answers with sources; generate images; fill insurance forms | | |
| | **Admin** | Manages access via `/admin` endpoints | List users by status; approve or reject pending signups; cannot demote themselves | | |
| | **Developer / learner** | Explores the ML catalog | Call `/models/*` endpoints with sample payloads; reproduce results from the bundled notebooks | | |
| | **Business analyst** | Asks data questions in plain English | SQL agent translates questions about sales, orders, inventory, overdues into queries against the star-schema tables | | |
| --- | |
| ## 4. Functional requirements | |
| ### 4.1 Authentication and admin approval | |
| - **FR-1** Every router (except `/` health and `/admin/*`) requires a valid | |
| Supabase JWT resolved to a user (`verify_approved_user`). | |
| - **FR-2** New signups have `profiles.status = 'pending'`; only | |
| `'approved'` users may call the API. Status checks are cached and the cache | |
| is invalidated on status change. | |
| - **FR-3** Admin endpoints (`GET /admin/users`, `POST /admin/users/{id}/status`) | |
| require `profiles.role = 'admin'`; allowed statuses are `pending`, | |
| `approved`, `rejected`; an admin cannot change their own status. | |
| - **FR-4** CORS restricted to the known frontend origins plus localhost dev | |
| ports; rate limiting via SlowAPI on all routes. | |
| ### 4.2 Multi-agent chat (`POST /multi_agent_chat`) | |
| - **FR-5** Keyword-based routing selects one agent per request: | |
| | Signal in query | Agent | | |
| |---|---| | |
| | draw, image, picture, logo… | Image generation | | |
| | memory, document, pdf, recall… | RAG over user documents | | |
| | sales, orders, inventory, stock… | SQL over business DB | | |
| | latest, news, today, trending… / no match | Tavily web search (default) | | |
| - **FR-6** Fallbacks: Tavily → RAG when `TAVILY_API_KEY` is unset; SQL → RAG | |
| on error. | |
| - **FR-7** Response caching in Redis keyed on the normalized query; cache is | |
| bypassed when a `session_id` is present so answers stay personal. | |
| - **FR-8** Conversation memory: with a `session_id`, the last 10 turns are | |
| loaded from Redis and passed to the agent. | |
| - **FR-9** Answers are scored (groundedness for RAG) before returning; chat | |
| history and logs are written in the background so they don't add latency. | |
| - **FR-10** `POST /feedback` records thumbs-up/down per answer; | |
| `POST /log_login` records login events. | |
| - **FR-11** RAG and SQL agents run in a threadpool so blocking work doesn't | |
| stall the async event loop. | |
| ### 4.3 Document ingestion and RAG memory | |
| - **FR-12** `POST /chunk_pdf` accepts a PDF (validated by MIME, size, and | |
| `%PDF` magic bytes); `POST /chunk_url` accepts a URL guarded against SSRF | |
| (private/loopback IPs blocked). Both stream NDJSON progress events. | |
| - **FR-13** Parsing: `pdfplumber` for text, `camelot` for tables | |
| (lattice → stream), `PyMuPDFLoader` fallback; `BeautifulSoup` for URLs. | |
| - **FR-14** Chunking: semantic chunking for prose; 16-row windows with 3-row | |
| overlap for tables; recursive fallback splits chunks > 1,200 chars into | |
| ~400-char pieces. Every chunk is prefixed with a | |
| `Document: <name> | Page: N` header. | |
| - **FR-15** Embedding: `BAAI/bge-base-en-v1.5`, L2-normalized, batched (32), | |
| with an embedding cache to cut repeat-query latency. Stored in Supabase | |
| pgvector (`rag_user_documents`), keyed per user. | |
| - **FR-16** Retrieval: history-aware query rewriting (follow-ups condensed to | |
| standalone queries, with a lexical guard to skip the LLM rewrite when | |
| unnecessary); hybrid vector + full-text search fused with Reciprocal Rank | |
| Fusion (RPC `match_rag_documents`); metadata filters (`source_type`, `url`, | |
| `created_after`); Chroma similarity/MMR fallback if the RPC errors. | |
| - **FR-17** Reranking: cross-encoder `ms-marco-MiniLM-L-6-v2` (toggle via | |
| `RERANK_ENABLED`), with a token-overlap fallback; near-duplicate chunks | |
| deduped before the final top-k. | |
| - **FR-18** Generation: DeepSeek answers strictly from numbered context plus | |
| the last 3 turns; a groundedness post-hook floors ungrounded answers. | |
| - **FR-19** Document CRUD: list, get, update, delete one/all, and fetch a | |
| public URL (`/documents*` endpoints), always scoped to the calling user. | |
| ### 4.4 SQL agent | |
| - **FR-20** Translates natural-language business questions into SQL against | |
| the configured Postgres database (star schema: `fact_sales`, `fact_orders`, | |
| `fact_inventory`, `fact_overdues` + master data), returning results in | |
| natural language. Falls back to RAG on failure. | |
| ### 4.5 Form-filling agent (`POST /fill_pdf_form`) | |
| - **FR-21** Extracts structured fields (policy number, plan type, life | |
| assured, etc.) from a prior AI response via DeepSeek JSON extraction and | |
| renders a pre-fill summary PDF (ReportLab) for the Great Eastern Life PSF02 | |
| insurance form, returned as a streaming download. | |
| ### 4.6 ML/DL model catalog (`/models/*`, `/predict/*`) | |
| - **FR-22** Expose the following as authenticated REST endpoints, each backed | |
| by a training notebook in `ml/` and demo datasets in `public/data/`: | |
| - **Regression:** linear, logistic, ridge, lasso, polynomial, decision-tree | |
| regressor, random forest, gradient boosting (churn, house/car prices, | |
| sales, bike rentals, taxi fares). | |
| - **Classification:** decision tree (iris), random forest (credit approval), | |
| KNN, naive Bayes, SVM (wine quality). | |
| - **Clustering / decomposition:** K-means, DBSCAN, mean-shift, PCA, ICA. | |
| - **Association rules:** Apriori, FP-Growth, ECLAT (grocery recommender). | |
| - **Deep learning:** CNN digit recognition, RNN/LSTM stock prediction. | |
| - **NLP / recommenders:** sentiment analysis, collaborative filtering | |
| (books, SVD), content filtering (movies). | |
| --- | |
| ## 5. Non-functional requirements | |
| | Category | Requirement | | |
| |----------|-------------| | |
| | **Security** | JWT auth on every route; admin-approval gate; per-user data isolation in RAG storage; SSRF guard on URL ingestion; PDF magic-byte/MIME/size validation; service-role Supabase key server-side only | | |
| | **Performance** | Redis response cache for repeat queries; embedding cache; `FETCH_K=20` candidates reranked to `TOP_K=7`; background writes for logs/history; threadpool for blocking agents | | |
| | **Rate limiting** | SlowAPI middleware with per-route limits | | |
| | **Availability** | Single container (HF Spaces, port 7860); graceful degradation via agent fallback chain (Tavily→RAG, SQL→RAG, RPC→Chroma, cross-encoder→lexical) | | |
| | **Observability** | Chat history, token usage, feedback, and login logs persisted to Supabase (`user_logs` tables) | | |
| | **Portability** | Docker image (Python 3.11.9, non-root user), `.env`-driven config, runs locally with uvicorn | | |
| --- | |
| ## 6. Architecture and tech stack | |
| ``` | |
| Vercel frontends ──JWT──▶ FastAPI (HF Spaces, Docker :7860) | |
| │ | |
| ┌─────────┬───────────┼────────────┬─────────────┐ | |
| ▼ ▼ ▼ ▼ ▼ | |
| Tavily API RAG agent SQL agent Image agent ML/DL routes | |
| │ │ | |
| ▼ ▼ | |
| Supabase pgvector Postgres (star schema) | |
| + Chroma fallback | |
| │ | |
| DeepSeek LLM Redis (cache + session memory) | |
| ``` | |
| | Layer | Choice | | |
| |-------|--------| | |
| | API | FastAPI + uvicorn, SlowAPI rate limiting | | |
| | LLM | DeepSeek (`deepseek-chat`) via OpenAI-compatible client | | |
| | Embeddings | HuggingFace `BAAI/bge-base-en-v1.5` | | |
| | Reranker | `cross-encoder/ms-marco-MiniLM-L-6-v2` | | |
| | Vector store | Supabase pgvector (primary), Chroma (fallback) | | |
| | Auth / DB / storage | Supabase (JWT, `profiles`, `rag_user_documents`, file storage) | | |
| | Cache / memory | Redis | | |
| | Web search | Tavily | | |
| | PDF | pdfplumber, camelot, PyMuPDF (parse); ReportLab (generate) | | |
| | ML | scikit-learn, TensorFlow/Keras, mlxtend, NLTK | | |
| --- | |
| ## 7. Success metrics | |
| | Metric | Target (PoC) | | |
| |--------|--------------| | |
| | Routing accuracy (correct agent chosen) | ≥ 90% on a labeled test set of queries | | |
| | Groundedness score on RAG answers | ≥ 0.7 average; ungrounded answers suppressed | | |
| | Cache hit rate on repeat queries | ≥ 30% of non-session chat traffic | | |
| | P50 chat latency (cache miss, RAG route) | ≤ 8 s end-to-end | | |
| | Ingestion success rate (valid PDFs/URLs) | ≥ 95% complete to 100% progress | | |
| | Unauthorized access | 0 approved-only endpoints reachable without approval | | |
| --- | |
| ## 8. Risks and mitigations | |
| | Risk | Mitigation | | |
| |------|------------| | |
| | Keyword routing misroutes ambiguous queries | Tavily default fallback; future: LLM-based router | | |
| | LLM hallucination in RAG answers | Groundedness scoring floors/suppresses ungrounded output | | |
| | External dependency outage (Tavily, DeepSeek, Redis) | Agent fallback chain; cache/memory degrade gracefully | | |
| | SSRF / malicious uploads | IP-range guard, MIME + magic-byte + size validation | | |
| | Free-tier cold starts on HF Spaces | Acceptable for PoC; document expected warm-up | | |
| | Single admin lockout | Admins cannot reject/demote themselves | | |
| --- | |
| ## 9. Open questions / future work | |
| - Replace keyword routing with LLM- or classifier-based intent routing. | |
| - Streaming (SSE) responses for chat, not just ingestion progress. | |
| - Automated RAG evaluation harness (retrieval hit-rate, answer quality) over | |
| the logged history and feedback data. | |
| - Self-service or email-notified approval flow instead of manual admin polling. | |
| - Generalize the form-filling agent beyond the single PSF02 form (template-driven). | |
| - Multi-database support for the SQL agent. | |