Hugging Face's logo

Spaces:
looh2
/
fastapi_hf
Running

App Files Community
fastapi_hf / PRD.md
looh2's picture
looh2
Add detailed Product Requirements Document for FastAPI Multi-Agent + RAG Service
0fb7b50
preview code
|
Raw
History Blame Contribute Delete
12.6 kB

Product Requirements Document — FastAPI Multi-Agent + RAG Service

Product FastAPI Multi-Agent + RAG backend ("Compass" API)
Status Proof of concept, deployed on Hugging Face Spaces (Docker)
Version 1.0
Last updated 2026-07-02
Owner Vince

1. Overview

A single FastAPI backend that serves as the AI/ML engine for a web frontend (Next.js/Angular apps hosted on Vercel). It does three things:

  1. Multi-agent chat — routes a user's natural-language question to the best specialist agent: web search, RAG over the user's uploaded documents, SQL over a business database, or image generation.
  2. Document memory (RAG) — lets users upload PDFs or URLs, chunks and embeds them, and answers questions grounded in those documents with per-user isolation.
  3. ML/DL model catalog — ~28 classic machine-learning and deep-learning endpoints (regression, classification, clustering, association rules, recommenders, CNN/RNN, sentiment) trained from bundled notebooks and CSVs.

Access is gated: every endpoint requires a valid Supabase JWT and an admin-approved account.

Problem statement

Users need one place to ask questions that may require very different capabilities — "what's in my contract PDF?", "what were sales last month?", "what's trending today?", "draw me a logo". Wiring each capability into a frontend separately is slow and duplicates auth, caching, memory, and logging. This service centralizes routing, guardrails, and observability behind one chat endpoint, plus a library of ready-made ML demos.

2. Goals and non-goals

Goals

  • G1 — Answer questions from the right specialist agent with a single POST /multi_agent_chat call; no client-side routing logic.
  • G2 — Grounded, per-user document Q&A: answers cite retrieved chunks and are scored for groundedness; User A can never see User B's documents.
  • G3 — Keep median chat latency low via Redis response caching, embedding caching, and bounded retrieval (TOP_K/FETCH_K).
  • G4 — Gate the service behind admin approval so a public deployment (HF Spaces) can't be used by unapproved signups.
  • G5 — Full observability: chat history, token usage, feedback, and login events persisted for later evaluation.
  • G6 — Showcase a broad catalog of classic ML/DL techniques as callable REST endpoints for demo/learning purposes.

Non-goals

  • No frontend/UI (owned by the separate Vercel apps).
  • No user self-service signup approval — approval is manual by an admin.
  • No fine-tuning or model training at request time beyond the bundled demo models.
  • No multi-tenant SQL agent — it targets one configured business database.
  • No horizontal-scale/HA guarantees; this is a proof of concept on a single container.

3. Users and personas

Persona Description Needs
End user (approved) Uses the chat UI on the Vercel frontend Ask anything; upload PDFs/URLs; get grounded answers with sources; generate images; fill insurance forms
Admin Manages access via /admin endpoints List users by status; approve or reject pending signups; cannot demote themselves
Developer / learner Explores the ML catalog Call /models/* endpoints with sample payloads; reproduce results from the bundled notebooks
Business analyst Asks data questions in plain English SQL agent translates questions about sales, orders, inventory, overdues into queries against the star-schema tables

4. Functional requirements

4.1 Authentication and admin approval

  • FR-1 Every router (except / health and /admin/*) requires a valid Supabase JWT resolved to a user (verify_approved_user).
  • FR-2 New signups have profiles.status = 'pending'; only 'approved' users may call the API. Status checks are cached and the cache is invalidated on status change.
  • FR-3 Admin endpoints (GET /admin/users, POST /admin/users/{id}/status) require profiles.role = 'admin'; allowed statuses are pending, approved, rejected; an admin cannot change their own status.
  • FR-4 CORS restricted to the known frontend origins plus localhost dev ports; rate limiting via SlowAPI on all routes.

4.2 Multi-agent chat (POST /multi_agent_chat)

  • FR-5 Keyword-based routing selects one agent per request:

    Signal in query Agent
    draw, image, picture, logo… Image generation
    memory, document, pdf, recall… RAG over user documents
    sales, orders, inventory, stock… SQL over business DB
    latest, news, today, trending… / no match Tavily web search (default)

  • FR-6 Fallbacks: Tavily → RAG when TAVILY_API_KEY is unset; SQL → RAG on error.

  • FR-7 Response caching in Redis keyed on the normalized query; cache is bypassed when a session_id is present so answers stay personal.

  • FR-8 Conversation memory: with a session_id, the last 10 turns are loaded from Redis and passed to the agent.

  • FR-9 Answers are scored (groundedness for RAG) before returning; chat history and logs are written in the background so they don't add latency.

  • FR-10 POST /feedback records thumbs-up/down per answer; POST /log_login records login events.

  • FR-11 RAG and SQL agents run in a threadpool so blocking work doesn't stall the async event loop.

4.3 Document ingestion and RAG memory

  • FR-12 POST /chunk_pdf accepts a PDF (validated by MIME, size, and %PDF magic bytes); POST /chunk_url accepts a URL guarded against SSRF (private/loopback IPs blocked). Both stream NDJSON progress events.
  • FR-13 Parsing: pdfplumber for text, camelot for tables (lattice → stream), PyMuPDFLoader fallback; BeautifulSoup for URLs.
  • FR-14 Chunking: semantic chunking for prose; 16-row windows with 3-row overlap for tables; recursive fallback splits chunks > 1,200 chars into ~400-char pieces. Every chunk is prefixed with a Document: <name> | Page: N header.
  • FR-15 Embedding: BAAI/bge-base-en-v1.5, L2-normalized, batched (32), with an embedding cache to cut repeat-query latency. Stored in Supabase pgvector (rag_user_documents), keyed per user.
  • FR-16 Retrieval: history-aware query rewriting (follow-ups condensed to standalone queries, with a lexical guard to skip the LLM rewrite when unnecessary); hybrid vector + full-text search fused with Reciprocal Rank Fusion (RPC match_rag_documents); metadata filters (source_type, url, created_after); Chroma similarity/MMR fallback if the RPC errors.
  • FR-17 Reranking: cross-encoder ms-marco-MiniLM-L-6-v2 (toggle via RERANK_ENABLED), with a token-overlap fallback; near-duplicate chunks deduped before the final top-k.
  • FR-18 Generation: DeepSeek answers strictly from numbered context plus the last 3 turns; a groundedness post-hook floors ungrounded answers.
  • FR-19 Document CRUD: list, get, update, delete one/all, and fetch a public URL (/documents* endpoints), always scoped to the calling user.

4.4 SQL agent

  • FR-20 Translates natural-language business questions into SQL against the configured Postgres database (star schema: fact_sales, fact_orders, fact_inventory, fact_overdues + master data), returning results in natural language. Falls back to RAG on failure.

4.5 Form-filling agent (POST /fill_pdf_form)

  • FR-21 Extracts structured fields (policy number, plan type, life assured, etc.) from a prior AI response via DeepSeek JSON extraction and renders a pre-fill summary PDF (ReportLab) for the Great Eastern Life PSF02 insurance form, returned as a streaming download.

4.6 ML/DL model catalog (/models/*, /predict/*)

  • FR-22 Expose the following as authenticated REST endpoints, each backed by a training notebook in ml/ and demo datasets in public/data/:
    • Regression: linear, logistic, ridge, lasso, polynomial, decision-tree regressor, random forest, gradient boosting (churn, house/car prices, sales, bike rentals, taxi fares).
    • Classification: decision tree (iris), random forest (credit approval), KNN, naive Bayes, SVM (wine quality).
    • Clustering / decomposition: K-means, DBSCAN, mean-shift, PCA, ICA.
    • Association rules: Apriori, FP-Growth, ECLAT (grocery recommender).
    • Deep learning: CNN digit recognition, RNN/LSTM stock prediction.
    • NLP / recommenders: sentiment analysis, collaborative filtering (books, SVD), content filtering (movies).

5. Non-functional requirements

Category Requirement
Security JWT auth on every route; admin-approval gate; per-user data isolation in RAG storage; SSRF guard on URL ingestion; PDF magic-byte/MIME/size validation; service-role Supabase key server-side only
Performance Redis response cache for repeat queries; embedding cache; FETCH_K=20 candidates reranked to TOP_K=7; background writes for logs/history; threadpool for blocking agents
Rate limiting SlowAPI middleware with per-route limits
Availability Single container (HF Spaces, port 7860); graceful degradation via agent fallback chain (Tavily→RAG, SQL→RAG, RPC→Chroma, cross-encoder→lexical)
Observability Chat history, token usage, feedback, and login logs persisted to Supabase (user_logs tables)
Portability Docker image (Python 3.11.9, non-root user), .env-driven config, runs locally with uvicorn

6. Architecture and tech stack 

Vercel frontends ──JWT──▶ FastAPI (HF Spaces, Docker :7860)
                              │
        ┌─────────┬───────────┼────────────┬─────────────┐
        ▼         ▼           ▼            ▼             ▼
   Tavily API  RAG agent  SQL agent   Image agent   ML/DL routes
                  │           │
                  ▼           ▼
        Supabase pgvector  Postgres (star schema)
        + Chroma fallback
                  │
              DeepSeek LLM        Redis (cache + session memory)
Layer Choice
API FastAPI + uvicorn, SlowAPI rate limiting
LLM DeepSeek (deepseek-chat) via OpenAI-compatible client
Embeddings HuggingFace BAAI/bge-base-en-v1.5
Reranker cross-encoder/ms-marco-MiniLM-L-6-v2
Vector store Supabase pgvector (primary), Chroma (fallback)
Auth / DB / storage Supabase (JWT, profiles, rag_user_documents, file storage)
Cache / memory Redis
Web search Tavily
PDF pdfplumber, camelot, PyMuPDF (parse); ReportLab (generate)
ML scikit-learn, TensorFlow/Keras, mlxtend, NLTK

7. Success metrics

Metric Target (PoC)
Routing accuracy (correct agent chosen) ≥ 90% on a labeled test set of queries
Groundedness score on RAG answers ≥ 0.7 average; ungrounded answers suppressed
Cache hit rate on repeat queries ≥ 30% of non-session chat traffic
P50 chat latency (cache miss, RAG route) ≤ 8 s end-to-end
Ingestion success rate (valid PDFs/URLs) ≥ 95% complete to 100% progress
Unauthorized access 0 approved-only endpoints reachable without approval

8. Risks and mitigations

Risk Mitigation
Keyword routing misroutes ambiguous queries Tavily default fallback; future: LLM-based router
LLM hallucination in RAG answers Groundedness scoring floors/suppresses ungrounded output
External dependency outage (Tavily, DeepSeek, Redis) Agent fallback chain; cache/memory degrade gracefully
SSRF / malicious uploads IP-range guard, MIME + magic-byte + size validation
Free-tier cold starts on HF Spaces Acceptable for PoC; document expected warm-up
Single admin lockout Admins cannot reject/demote themselves

9. Open questions / future work

  • Replace keyword routing with LLM- or classifier-based intent routing.
  • Streaming (SSE) responses for chat, not just ingestion progress.
  • Automated RAG evaluation harness (retrieval hit-rate, answer quality) over the logged history and feedback data.
  • Self-service or email-notified approval flow instead of manual admin polling.
  • Generalize the form-filling agent beyond the single PSF02 form (template-driven).
  • Multi-database support for the SQL agent.