backend/README.md

Backend Documentation

This folder contains the production-ready FastAPI stack plus the companion MCP servers that power IntegraChat.

Directory Overview

• api/ – FastAPI application (routes, services, storage helpers, MCP clients)
• mcp_server/ – Unified MCP server exposing rag/web/admin tools via namespaces
• workers/ – Celery workers and schedulers for async ingestion + analytics maintenance

Prerequisites

• Python 3.10+
• PostgreSQL (with the vector extension) for RAG data, or Supabase with pgvector enabled
• SQLite (auto-created in data/) for analytics and admin rules
• Optional: Ollama running locally (default) or Groq API credentials for remote LLMs

Create a virtual environment at the repo root, then:

pip install -r requirements.txt
cp env.example .env   # update MCP URLs + LLM settings

Running the Services Locally

1. FastAPI core

   uvicorn backend.api.main:app --port 8000 --reload
   

2. Unified MCP server (rag/web/admin)

   python backend/mcp_server/server.py
   

   This single endpoint exposes the following namespaced tools:

   • rag.search, rag.ingest, rag.delete
   • web.search
   • admin.getRules, admin.addRule, admin.deleteRule, admin.logViolation

3. Optional workers (if running Celery-based ingestion/analytics jobs):

   celery -A backend.workers.ingestion_worker worker --loglevel=info
   celery -A backend.workers.analytics_worker worker --loglevel=info
   

The Gradio UI (python app.py) and the Next.js operator console (see frontend/README.md) both talk to the FastAPI layer at http://localhost:8000.

Key Endpoints

All endpoints require the x-tenant-id header unless otherwise noted.

Service	Path	Notes
Agent	POST /agent/message	Autonomous orchestration (RAG/Web/Admin/LLM)
Agent Debug	POST /agent/debug	Full reasoning trace + tool plan
Agent Plan	POST /agent/plan	Dry-run planning without executing tools
RAG	POST /rag/ingest-document	Rich ingestion (text, URL, metadata)
RAG	GET /rag/list	Paginated document listing per tenant
Admin	POST /admin/rules	Regex + severity rule ingestion
Analytics	GET /analytics/overview	Summary metrics (queries, tokens, red flags)

Refer to the root README.md for the complete endpoint tables.

Diagnostics & Tenant Isolation

Use the helper scripts in the repo root when validating backend changes:

• python verify_tenant_isolation.py – Exercises analytics logging, admin rule CRUD, API reachability, and proves RAG tenant isolation by ingesting + querying as multiple tenants.
• python check_rag_database.py – Talks directly to the pgvector database to list tenant IDs, preview stored chunks, and run safeguarded searches via search_vectors(). Helpful when troubleshooting suspected cross-tenant leakage.
• python test_manual.py – Legacy manual smoke test harness (analytics store, admin rules, API surface).

Troubleshooting tip: If the isolation script reports a failure, first run check_rag_database.py to confirm documents are tagged with the correct tenant_id, then restart the unified MCP server so it reloads the updated SQL filtering logic.

Environment Variables (excerpt)

Defined in env.example:

• RAG_MCP_URL, WEB_MCP_URL, ADMIN_MCP_URL
• OLLAMA_URL, OLLAMA_MODEL (or GROQ_API_KEY + LLM_BACKEND=groq)
• SUPABASE_URL, SUPABASE_SERVICE_KEY (optional admin integrations)
• APP_ENV, LOG_LEVEL, API_PORT

Update these before starting the servers to ensure the agent can reach every MCP endpoint and LLM runtime.

Unified MCP tool instructions

Agents that speak the Model Context Protocol should connect to the integrachat server id defined in backend/mcp_server/server.py and call the namespaced tools directly:

Namespace	Tool	Purpose
rag	search	Retrieve tenant-scoped document chunks
rag	ingest	Chunk + store new knowledge
rag	delete	Remove one/all stored documents
web	search	DuckDuckGo English-biased search
admin	getRules	Fetch tenant governance rules (list or detailed)
admin	addRule	Insert or update a rule
admin	deleteRule	Remove a rule by text
admin	logViolation	Persist a red-flag event into analytics

Always send tenant_id, and optionally user_id, in the payload so the shared middleware can enforce isolation and log analytics.