# Product Requirements Document — FastAPI Multi-Agent + RAG Service

| | |
|---|---|
| **Product** | FastAPI Multi-Agent + RAG backend ("Compass" API) |
| **Status** | Proof of concept, deployed on Hugging Face Spaces (Docker) |
| **Version** | 1.0 |
| **Last updated** | 2026-07-02 |
| **Owner** | Vince |

---

## 1. Overview

A single FastAPI backend that serves as the AI/ML engine for a web frontend
(Next.js/Angular apps hosted on Vercel). It does three things:

1. **Multi-agent chat** — routes a user's natural-language question to the best
   specialist agent: web search, RAG over the user's uploaded documents,
   SQL over a business database, or image generation.
2. **Document memory (RAG)** — lets users upload PDFs or URLs, chunks and
   embeds them, and answers questions grounded in those documents with
   per-user isolation.
3. **ML/DL model catalog** — ~28 classic machine-learning and deep-learning
   endpoints (regression, classification, clustering, association rules,
   recommenders, CNN/RNN, sentiment) trained from bundled notebooks and CSVs.

Access is gated: every endpoint requires a valid Supabase JWT **and** an
admin-approved account.

### Problem statement

Users need one place to ask questions that may require very different
capabilities — "what's in my contract PDF?", "what were sales last month?",
"what's trending today?", "draw me a logo". Wiring each capability into a
frontend separately is slow and duplicates auth, caching, memory, and logging.
This service centralizes routing, guardrails, and observability behind one
chat endpoint, plus a library of ready-made ML demos.

---

## 2. Goals and non-goals

### Goals

- **G1** — Answer questions from the right specialist agent with a single
  `POST /multi_agent_chat` call; no client-side routing logic.
- **G2** — Grounded, per-user document Q&A: answers cite retrieved chunks and
  are scored for groundedness; User A can never see User B's documents.
- **G3** — Keep median chat latency low via Redis response caching, embedding
  caching, and bounded retrieval (`TOP_K`/`FETCH_K`).
- **G4** — Gate the service behind admin approval so a public deployment
  (HF Spaces) can't be used by unapproved signups.
- **G5** — Full observability: chat history, token usage, feedback, and login
  events persisted for later evaluation.
- **G6** — Showcase a broad catalog of classic ML/DL techniques as callable
  REST endpoints for demo/learning purposes.

### Non-goals

- No frontend/UI (owned by the separate Vercel apps).
- No user self-service signup approval — approval is manual by an admin.
- No fine-tuning or model training at request time beyond the bundled demo
  models.
- No multi-tenant SQL agent — it targets one configured business database.
- No horizontal-scale/HA guarantees; this is a proof of concept on a single
  container.

---

## 3. Users and personas

| Persona | Description | Needs |
|---------|-------------|-------|
| **End user (approved)** | Uses the chat UI on the Vercel frontend | Ask anything; upload PDFs/URLs; get grounded answers with sources; generate images; fill insurance forms |
| **Admin** | Manages access via `/admin` endpoints | List users by status; approve or reject pending signups; cannot demote themselves |
| **Developer / learner** | Explores the ML catalog | Call `/models/*` endpoints with sample payloads; reproduce results from the bundled notebooks |
| **Business analyst** | Asks data questions in plain English | SQL agent translates questions about sales, orders, inventory, overdues into queries against the star-schema tables |

---

## 4. Functional requirements

### 4.1 Authentication and admin approval

- **FR-1** Every router (except `/` health and `/admin/*`) requires a valid
  Supabase JWT resolved to a user (`verify_approved_user`).
- **FR-2** New signups have `profiles.status = 'pending'`; only
  `'approved'` users may call the API. Status checks are cached and the cache
  is invalidated on status change.
- **FR-3** Admin endpoints (`GET /admin/users`, `POST /admin/users/{id}/status`)
  require `profiles.role = 'admin'`; allowed statuses are `pending`,
  `approved`, `rejected`; an admin cannot change their own status.
- **FR-4** CORS restricted to the known frontend origins plus localhost dev
  ports; rate limiting via SlowAPI on all routes.

### 4.2 Multi-agent chat (`POST /multi_agent_chat`)

- **FR-5** Keyword-based routing selects one agent per request:

  | Signal in query | Agent |
  |---|---|
  | draw, image, picture, logo… | Image generation |
  | memory, document, pdf, recall… | RAG over user documents |
  | sales, orders, inventory, stock… | SQL over business DB |
  | latest, news, today, trending… / no match | Tavily web search (default) |

- **FR-6** Fallbacks: Tavily → RAG when `TAVILY_API_KEY` is unset; SQL → RAG
  on error.
- **FR-7** Response caching in Redis keyed on the normalized query; cache is
  bypassed when a `session_id` is present so answers stay personal.
- **FR-8** Conversation memory: with a `session_id`, the last 10 turns are
  loaded from Redis and passed to the agent.
- **FR-9** Answers are scored (groundedness for RAG) before returning; chat
  history and logs are written in the background so they don't add latency.
- **FR-10** `POST /feedback` records thumbs-up/down per answer;
  `POST /log_login` records login events.
- **FR-11** RAG and SQL agents run in a threadpool so blocking work doesn't
  stall the async event loop.

### 4.3 Document ingestion and RAG memory

- **FR-12** `POST /chunk_pdf` accepts a PDF (validated by MIME, size, and
  `%PDF` magic bytes); `POST /chunk_url` accepts a URL guarded against SSRF
  (private/loopback IPs blocked). Both stream NDJSON progress events.
- **FR-13** Parsing: `pdfplumber` for text, `camelot` for tables
  (lattice → stream), `PyMuPDFLoader` fallback; `BeautifulSoup` for URLs.
- **FR-14** Chunking: semantic chunking for prose; 16-row windows with 3-row
  overlap for tables; recursive fallback splits chunks > 1,200 chars into
  ~400-char pieces. Every chunk is prefixed with a
  `Document: <name> | Page: N` header.
- **FR-15** Embedding: `BAAI/bge-base-en-v1.5`, L2-normalized, batched (32),
  with an embedding cache to cut repeat-query latency. Stored in Supabase
  pgvector (`rag_user_documents`), keyed per user.
- **FR-16** Retrieval: history-aware query rewriting (follow-ups condensed to
  standalone queries, with a lexical guard to skip the LLM rewrite when
  unnecessary); hybrid vector + full-text search fused with Reciprocal Rank
  Fusion (RPC `match_rag_documents`); metadata filters (`source_type`, `url`,
  `created_after`); Chroma similarity/MMR fallback if the RPC errors.
- **FR-17** Reranking: cross-encoder `ms-marco-MiniLM-L-6-v2` (toggle via
  `RERANK_ENABLED`), with a token-overlap fallback; near-duplicate chunks
  deduped before the final top-k.
- **FR-18** Generation: DeepSeek answers strictly from numbered context plus
  the last 3 turns; a groundedness post-hook floors ungrounded answers.
- **FR-19** Document CRUD: list, get, update, delete one/all, and fetch a
  public URL (`/documents*` endpoints), always scoped to the calling user.

### 4.4 SQL agent

- **FR-20** Translates natural-language business questions into SQL against
  the configured Postgres database (star schema: `fact_sales`, `fact_orders`,
  `fact_inventory`, `fact_overdues` + master data), returning results in
  natural language. Falls back to RAG on failure.

### 4.5 Form-filling agent (`POST /fill_pdf_form`)

- **FR-21** Extracts structured fields (policy number, plan type, life
  assured, etc.) from a prior AI response via DeepSeek JSON extraction and
  renders a pre-fill summary PDF (ReportLab) for the Great Eastern Life PSF02
  insurance form, returned as a streaming download.

### 4.6 ML/DL model catalog (`/models/*`, `/predict/*`)

- **FR-22** Expose the following as authenticated REST endpoints, each backed
  by a training notebook in `ml/` and demo datasets in `public/data/`:
  - **Regression:** linear, logistic, ridge, lasso, polynomial, decision-tree
    regressor, random forest, gradient boosting (churn, house/car prices,
    sales, bike rentals, taxi fares).
  - **Classification:** decision tree (iris), random forest (credit approval),
    KNN, naive Bayes, SVM (wine quality).
  - **Clustering / decomposition:** K-means, DBSCAN, mean-shift, PCA, ICA.
  - **Association rules:** Apriori, FP-Growth, ECLAT (grocery recommender).
  - **Deep learning:** CNN digit recognition, RNN/LSTM stock prediction.
  - **NLP / recommenders:** sentiment analysis, collaborative filtering
    (books, SVD), content filtering (movies).

---

## 5. Non-functional requirements

| Category | Requirement |
|----------|-------------|
| **Security** | JWT auth on every route; admin-approval gate; per-user data isolation in RAG storage; SSRF guard on URL ingestion; PDF magic-byte/MIME/size validation; service-role Supabase key server-side only |
| **Performance** | Redis response cache for repeat queries; embedding cache; `FETCH_K=20` candidates reranked to `TOP_K=7`; background writes for logs/history; threadpool for blocking agents |
| **Rate limiting** | SlowAPI middleware with per-route limits |
| **Availability** | Single container (HF Spaces, port 7860); graceful degradation via agent fallback chain (Tavily→RAG, SQL→RAG, RPC→Chroma, cross-encoder→lexical) |
| **Observability** | Chat history, token usage, feedback, and login logs persisted to Supabase (`user_logs` tables) |
| **Portability** | Docker image (Python 3.11.9, non-root user), `.env`-driven config, runs locally with uvicorn |

---

## 6. Architecture and tech stack

```
Vercel frontends ──JWT──▶ FastAPI (HF Spaces, Docker :7860)
                              │
        ┌─────────┬───────────┼────────────┬─────────────┐
        ▼         ▼           ▼            ▼             ▼
   Tavily API  RAG agent  SQL agent   Image agent   ML/DL routes
                  │           │
                  ▼           ▼
        Supabase pgvector  Postgres (star schema)
        + Chroma fallback
                  │
              DeepSeek LLM        Redis (cache + session memory)
```

| Layer | Choice |
|-------|--------|
| API | FastAPI + uvicorn, SlowAPI rate limiting |
| LLM | DeepSeek (`deepseek-chat`) via OpenAI-compatible client |
| Embeddings | HuggingFace `BAAI/bge-base-en-v1.5` |
| Reranker | `cross-encoder/ms-marco-MiniLM-L-6-v2` |
| Vector store | Supabase pgvector (primary), Chroma (fallback) |
| Auth / DB / storage | Supabase (JWT, `profiles`, `rag_user_documents`, file storage) |
| Cache / memory | Redis |
| Web search | Tavily |
| PDF | pdfplumber, camelot, PyMuPDF (parse); ReportLab (generate) |
| ML | scikit-learn, TensorFlow/Keras, mlxtend, NLTK |

---

## 7. Success metrics

| Metric | Target (PoC) |
|--------|--------------|
| Routing accuracy (correct agent chosen) | ≥ 90% on a labeled test set of queries |
| Groundedness score on RAG answers | ≥ 0.7 average; ungrounded answers suppressed |
| Cache hit rate on repeat queries | ≥ 30% of non-session chat traffic |
| P50 chat latency (cache miss, RAG route) | ≤ 8 s end-to-end |
| Ingestion success rate (valid PDFs/URLs) | ≥ 95% complete to 100% progress |
| Unauthorized access | 0 approved-only endpoints reachable without approval |

---

## 8. Risks and mitigations

| Risk | Mitigation |
|------|------------|
| Keyword routing misroutes ambiguous queries | Tavily default fallback; future: LLM-based router |
| LLM hallucination in RAG answers | Groundedness scoring floors/suppresses ungrounded output |
| External dependency outage (Tavily, DeepSeek, Redis) | Agent fallback chain; cache/memory degrade gracefully |
| SSRF / malicious uploads | IP-range guard, MIME + magic-byte + size validation |
| Free-tier cold starts on HF Spaces | Acceptable for PoC; document expected warm-up |
| Single admin lockout | Admins cannot reject/demote themselves |

---

## 9. Open questions / future work

- Replace keyword routing with LLM- or classifier-based intent routing.
- Streaming (SSE) responses for chat, not just ingestion progress.
- Automated RAG evaluation harness (retrieval hit-rate, answer quality) over
  the logged history and feedback data.
- Self-service or email-notified approval flow instead of manual admin polling.
- Generalize the form-filling agent beyond the single PSF02 form (template-driven).
- Multi-database support for the SQL agent.