backend/README.md

RAG Agent Workbench – Backend

Lightweight FastAPI backend for ingesting documents into Pinecone (with integrated embeddings), searching over them, and serving a production-style RAG chat endpoint.

Prerequisites

• Python 3.11+
• A Pinecone account and an index configured with integrated embeddings
• A Groq account and API key for chat
• (Optional) Tavily API key for web search fallback
• (Optional) LangSmith account + API key for tracing
• Environment variables set (see .env.example)

Setup

cd backend
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env  # then edit with your Pinecone, Groq, and optional Tavily/LangSmith credentials

Required .env values:

• PINECONE_API_KEY – your Pinecone API key
• PINECONE_INDEX_NAME – the index name (used for configuration checks)
• PINECONE_HOST – the index host URL (use host targeting for production)
• PINECONE_NAMESPACE – default namespace (e.g. dev)
• PINECONE_TEXT_FIELD – text field name used by the integrated embedding index (e.g. chunk_text or content)
• LOG_LEVEL – e.g. INFO, DEBUG

Required for /chat:

• GROQ_API_KEY – your Groq API key
• GROQ_BASE_URL – Groq OpenAI-compatible endpoint (default https://api.groq.com/openai/v1)
• GROQ_MODEL – Groq chat model name (default llama-3.1-8b-instant)

Optional for web search fallback:

• TAVILY_API_KEY – Tavily API key (enables web search in /chat when retrieval is weak)

Optional for LangSmith tracing:

• LANGCHAIN_TRACING_V2 – set to true to enable tracing
• LANGCHAIN_API_KEY – your LangSmith API key
• LANGCHAIN_PROJECT – project name for traces (e.g. rag-agent-workbench)

Optional for basic API protection:

• API_KEY – when set, all routers except /health are protected by X-API-Key (including /chat, /search, /documents/*, /ingest/*, /metrics, and the OpenAPI/Swagger docs).
  • In production-like environments (ENV=production or on Hugging Face Spaces), API_KEY must be set or the backend will fail to start.
  • In local development (no Spaces and ENV not set to production), API_KEY is optional; when omitted, the API (including docs) is open.

Optional for CORS:

• ALLOWED_ORIGINS – comma-separated list of allowed origins.
  • If unset, defaults to "*" (useful for local dev and quick demos).

Optional for rate limiting and caching:

• RATE_LIMIT_ENABLED – defaults to true. Set to false to disable SlowAPI limits.
• CACHE_ENABLED – defaults to true. Set to false to disable in-memory TTL caches.

Your Pinecone index must be configured for integrated embeddings (e.g. via create_index_for_model or configure_index(embed=...)), with a field mapping that includes the configured PINECONE_TEXT_FIELD.

Run locally

cd backend
uvicorn app.main:app --reload --port 8000

The API will be available at http://localhost:8000.

Sample endpoints

Health

curl http://localhost:8000/health

Ingest from arXiv

curl -X POST "http://localhost:8000/ingest/arxiv" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "retrieval augmented generation",
    "max_docs": 5,
    "namespace": "dev",
    "category": "papers"
  }'

Ingest from OpenAlex

curl -X POST "http://localhost:8000/ingest/openalex" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "retrieval augmented generation",
    "max_docs": 5,
    "namespace": "dev",
    "mailto": "you@example.com"
  }'

Ingest from Wikipedia

curl -X POST "http://localhost:8000/ingest/wiki" \
  -H "Content-Type: application/json" \
  -d '{
    "titles": ["Retrieval-augmented generation", "Vector database"],
    "namespace": "dev"
  }'

Manual text upload

curl -X POST "http://localhost:8000/documents/upload-text" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "My manual note",
    "source": "manual",
    "text": "This is some example text describing RAG pipelines...",
    "namespace": "dev",
    "metadata": {
      "url": "https://example.com/my-note"
    }
  }'

Search

curl -X POST "http://localhost:8000/search" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $API_KEY" \  # only if API_KEY is enabled
  -d '{
    "query": "what is RAG",
    "top_k": 5,
    "namespace": "dev",
    "filters": {"source": "arxiv"}
  }'

Document stats

curl "http://localhost:8000/documents/stats?namespace=dev"

Chat (non-streaming)

curl -X POST "http://localhost:8000/chat" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $API_KEY" \  # only if API_KEY is enabled
  -d '{
    "query": "What is retrieval-augmented generation?",
    "namespace": "dev",
    "top_k": 5,
    "use_web_fallback": true,
    "min_score": 0.25,
    "max_web_results": 5,
    "chat_history": [
      {"role": "user", "content": "You are helping me understand RAG."}
    ]
  }'

Example JSON response:

{
  "answer": "...",
  "sources": [
    {
      "source": "wiki",
      "title": "Retrieval-augmented generation",
      "url": "https://en.wikipedia.org/wiki/...",
      "score": 0.91,
      "chunk_text": "..."
    }
  ],
  "timings": {
    "retrieve_ms": 35.2,
    "web_ms": 0.0,
    "generate_ms": 420.7,
    "total_ms": 470.1
  },
  "trace": {
    "langsmith_project": "rag-agent-workbench",
    "trace_enabled": true
  }
}

Chat (SSE streaming)

curl -N -X POST "http://localhost:8000/chat/stream" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $API_KEY" \  # only if API_KEY is enabled
  -d '{
    "query": "Summarise retrieval-augmented generation.",
    "namespace": "dev",
    "top_k": 5,
    "use_web_fallback": true
  }'

• The response will be text/event-stream.
• Individual SSE events stream tokens (space-delimited).
• The final event (event: end) includes the full JSON payload as in /chat.

Metrics

curl "http://localhost:8000/metrics"

Returns JSON with:

• requests_by_path and errors_by_path
• timings (average and p50/p95 for retrieve_ms, web_ms, generate_ms, total_ms)
• cache stats
• Last 20 timing samples for chat.

Seeding data

A helper script is provided to seed the index with multiple arXiv and OpenAlex queries:

python ../scripts/seed_ingest.py --base-url http://localhost:8000 --namespace dev --mailto you@example.com

Docling integration (external scripts)

Docling is used via separate scripts so the backend container stays small and does not depend on Docling. To convert local documents and upload them as text:

Single file

cd scripts
pip install docling  # optional but recommended for rich formats
python docling_convert_and_upload.py \
  --file /path/to/file.pdf \
  --backend-url http://localhost:8000 \
  --namespace dev \
  --title "My local document" \
  --source local-file \
  --api-key "$API_KEY"

• Supported formats when Docling is installed include: PDF, DOCX, PPT/PPTX, XLS/XLSX, HTML/HTM, MD, AsciiDoc, and TXT.
• If Docling is not installed:
  • .txt and .md files are ingested as raw text.
  • Other formats will fail with a clear message instructing you to install Docling.

Batch ingest a folder

cd scripts
pip install docling  # optional but recommended
python batch_ingest_local_folder.py \
  --folder /path/to/folder \
  --backend-url http://localhost:8000 \
  --namespace dev \
  --source local-folder \
  --max-files 200 \
  --api-key "$API_KEY"

• Recursively scans the folder for supported extensions and ingests up to max-files documents.
• Each file is converted via docling_convert_and_upload.py logic and uploaded to /documents/upload-text.

Upload documents via UI (Streamlit dialog)

The Streamlit chat frontend also supports uploading documents directly from the browser:

• Click the “📄 Upload Document” button at the top of the chat page.
• A modal dialog opens with:
  • File chooser (.pdf, .md, .txt, .docx, .pptx, .xlsx, .html, .htm).
  • Title (defaults to filename without extension).
  • Namespace (defaults to the backend namespace, e.g. dev).
  • Source label (defaults to ui-upload).
  • Optional metadata: tags (comma-separated) and free-form notes.
• On upload:
  • The frontend converts the file to markdown/text and calls POST /documents/upload-text with:
    • title, source, text, namespace, and a metadata dictionary containing conversion and UI metadata.
  • On success, the upload is recorded in a “Recent uploads” section in the sidebar and can be quickly queried via “Search this document”.

Notes:

• Conversion happens entirely in the frontend:
  • .txt and .md files are read as raw text.
  • For richer formats (PDF/Office/HTML), the frontend attempts to use Docling if installed.
  • If Docling is not available, an informative error is shown and the user is asked to upload .md/.txt instead.
• On Streamlit Cloud, Docling must be added to the app’s Python environment (e.g. requirements.txt) for PDF/Office uploads to work.
• Streamlit’s file uploader has a default maximum size (typically 200 MB); check Streamlit documentation if you need to increase or restrict this limit.

Deploy Backend on Hugging Face Spaces (Docker)

1. Create a new Space

   • Go to Hugging Face → New Space.
   • Choose:
     • SDK: Docker
     • Space name: e.g. your-name/rag-agent-workbench-backend.
   • Point the Space to this repository and configure it to use the backend/ subdirectory (or copy backend/Dockerfile to the root if you prefer).

2. Environment variables / secrets

   In the Space settings, configure the following (as “Secrets” where appropriate):

   Required:

   • PINECONE_API_KEY
   • PINECONE_HOST
   • PINECONE_INDEX_NAME
   • PINECONE_NAMESPACE
   • PINECONE_TEXT_FIELD=content (or your actual text field)
   • GROQ_API_KEY
   • GROQ_BASE_URL (optional, defaults to https://api.groq.com/openai/v1)
   • GROQ_MODEL (optional, defaults to llama-3.1-8b-instant)

   Optional:

   • TAVILY_API_KEY (web search fallback for /chat)
   • LANGCHAIN_TRACING_V2
   • LANGCHAIN_API_KEY
   • LANGCHAIN_PROJECT
   • API_KEY (to protect /ingest/*, /documents/*, /search, /chat*)
   • ALLOWED_ORIGINS (e.g. your Streamlit frontend origin)
   • RATE_LIMIT_ENABLED and CACHE_ENABLED (rarely need to change from defaults)

3. Ports and startup

   • The Docker image exposes port 7860 by default.
   • Hugging Face Spaces sets the PORT environment variable; the CMD honours it:
     • uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-7860}
   • On successful startup, logs include:
     • Starting on port=<port> hf_spaces_mode=<bool>

4. Verify

   • Open your Space URL:
     • https://<your-space>.hf.space/docs – interactive API docs.
     • https://<your-space>.hf.space/health – health check.
   • If API_KEY is set, test protected endpoints using X-API-Key.

Deploy Frontend on Streamlit Community Cloud

1. Prepare the repo

   • The minimal Streamlit frontend lives under frontend/app.py.
   • Root requirements.txt includes:
     • streamlit
     • httpx

2. Create Streamlit app

   • Go to Streamlit Community Cloud and create a new app.
   • Point it at this repository.
   • Set the main file to frontend/app.py.

3. Configure Streamlit secrets

   • In the Streamlit app settings, configure Secrets (YAML):

     BACKEND_BASE_URL: "https://<your-backend-space>.hf.space"
     API_KEY: "your-backend-api-key"  # only if backend API_KEY is set
     

   • Do not commit secrets into the repo.

4. Verify connectivity

   • Open the Streamlit app.
   • In the sidebar “Connectivity” panel:
     • Confirm the backend URL is correct.
     • Click “Ping /health” to verify backend connectivity.
   • Use the chat panel to send a question:
     • The app will call /chat on the backend and display answer, timings, and sources.

Local Test Checklist – Work Package C

1. Configure environment

   • Set PINECONE_* variables for an integrated embeddings index.
   • Set GROQ_API_KEY (and optionally override GROQ_BASE_URL, GROQ_MODEL).
   • Optionally set TAVILY_API_KEY for web fallback.
   • Optionally enable LangSmith:
     • LANGCHAIN_TRACING_V2=true
     • LANGCHAIN_API_KEY=...
     • LANGCHAIN_PROJECT=rag-agent-workbench
   • Optionally set:
     • API_KEY for basic protection.
     • ALLOWED_ORIGINS if you are calling from a browser origin.
     • RATE_LIMIT_ENABLED / CACHE_ENABLED for tuning.

2. Start the backend

   cd backend
   uvicorn app.main:app --reload --port 8000
   

3. Ingest data

   • Quick Wikipedia smoke test (also see scripts/smoke_chat.py):

     python ../scripts/smoke_chat.py --backend-url http://localhost:8000 --namespace dev
     

4. Test /search

   curl -X POST "http://localhost:8000/search" \
     -H "Content-Type: application/json" \
     -H "X-API-Key: $API_KEY" \  # only if API_KEY is enabled
     -d '{"query": "what is RAG", "namespace": "dev", "top_k": 5}'
   

5. Test /chat

   • Use the curl example above or run:

     curl -X POST "http://localhost:8000/chat" \
       -H "Content-Type: application/json" \
       -H "X-API-Key: $API_KEY" \  # only if API_KEY is enabled
       -d '{"query": "What is retrieval-augmented generation?", "namespace": "dev"}'
     

6. Test /chat with web fallback

   • Requires TAVILY_API_KEY:

     python ../scripts/smoke_chat_web.py --backend-url http://localhost:8000 --namespace dev
     

7. Inspect /metrics

   curl "http://localhost:8000/metrics"
   

   • Confirm:
     • Request counts are increasing.
     • Timing stats (average_ms, p50_ms, p95_ms) are populated after several /chat calls.
     • Cache hit/miss counters change when repeating identical /search or /chat requests.

8. Run the benchmark script

   • From the repo root:

     python scripts/bench_local.py \
       --backend-url http://localhost:8000 \
       --namespace dev \
       --concurrency 10 \
       --requests 50 \
       --api-key "$API_KEY"
     

   • Review reported:

     • Average latency.
     • p50 / p95 latency.
     • Error rate.

9. Optional: Test Streamlit frontend locally

   • Install root requirements:

     pip install -r requirements.txt
     

   • Run:

     streamlit run frontend/app.py
     

   • Configure BACKEND_BASE_URL and API_KEY via environment or .streamlit/secrets.toml, and verify chat works end-to-end.