update the readme files

README.md

@@ -6,6 +6,23 @@
 
 ---
 
+## 📋 Table of Contents
+
+- [Overview](#overview)
+- [Quick Start](#quick-start)
+- [Features](#features)
+- [Installation & Setup](#installation--setup)
+- [Usage](#usage)
+- [API Endpoints](#api-endpoints)
+- [Architecture](#architecture)
+- [Supabase Setup & Migration](#supabase-setup--migration)
+- [Troubleshooting](#troubleshooting)
+- [Testing & Diagnostics](#testing--diagnostics)
+- [Technical Stack](#technical-stack)
+- [License](#license)
+
+---
+
 ## Overview
 
 **IntegraChat** is an enterprise-grade, multi-tenant AI platform that demonstrates the full capabilities of the **Model Context Protocol (MCP)** in a production-style environment. Built with enterprise governance and observability in mind, IntegraChat combines autonomous tool-using agents, RAG retrieval, live web search, and admin compliance under strict tenant isolation.
@@ -14,6 +31,47 @@ This platform showcases how MCP can power intelligent, governed, multi-tenant AI
 
 ---
 
+## 🚀 Quick Start
+
+### Windows Users
+```bash
+# 1. Install dependencies
+pip install -r requirements.txt
+
+# 2. Configure environment (copy and edit .env)
+cp env.example .env
+# Edit .env with your credentials (Supabase, LLM, etc.)
+
+# 3. Start all services
+start.bat
+```
+
+### Manual Setup
+```bash
+# 1. Install dependencies
+pip install -r requirements.txt
+
+# 2. Configure environment
+cp env.example .env
+# Edit .env with your credentials
+
+# 3. Start FastAPI backend (Terminal 1)
+uvicorn backend.api.main:app --port 8000 --reload
+
+# 4. Start unified MCP server (Terminal 2)
+python backend/mcp_server/server.py
+
+# 5. Start Gradio UI (Terminal 3)
+python app.py
+```
+
+Then access:
+- **Gradio UI**: `http://localhost:7860`
+- **FastAPI Docs**: `http://localhost:8000/docs`
+- **Next.js Frontend** (optional): `cd frontend && npm install && npm run dev` → `http://localhost:3000`
+
+---
+
 ## Features
 
 ### Core Capabilities
@@ -58,184 +116,320 @@ This platform showcases how MCP can power intelligent, governed, multi-tenant AI
 
 ---
 
-## How to Run the Space
+## Installation & Setup
+
 ### Prerequisites
 
-1. **Backend services running**:
-   - FastAPI API (`uvicorn backend.api.main:app --port 8000`)
-   - Unified MCP server (port 8900) as described in `backend/README.md`
-   - Optional: Ollama / Groq credentials for the LLM client
-   
-   **Quick Start**: Run `start.bat` (Windows) to launch all services automatically.
-2. **Python 3.10+** with the dependencies in `requirements.txt`
-3. **Google Custom Search credentials** for live web search:
-   - Enable the *Custom Search API* in the [Google Cloud Console](https://console.cloud.google.com/) and create an API key → set it as `GOOGLE_SEARCH_API_KEY` in `.env`
-   - Create a Programmable Search Engine that searches the entire web and copy its *Search engine ID* → set it as `GOOGLE_SEARCH_CX_ID` in `.env`
-   - Restart the backend after updating `.env` so the new variables are picked up
+- **Python 3.10+** with pip
+- **PostgreSQL** (with pgvector extension) or **Supabase** for RAG storage
+- **Supabase** (recommended) or SQLite for admin rules and analytics
+- **Ollama** (local) or **Groq API** credentials for LLM
+- **Google Custom Search API** (optional, for web search):
+  - Enable Custom Search API in [Google Cloud Console](https://console.cloud.google.com/)
+  - Create API key → set as `GOOGLE_SEARCH_API_KEY` in `.env`
+  - Create Programmable Search Engine → set ID as `GOOGLE_SEARCH_CX_ID` in `.env`
 
-### Installation
+### Step-by-Step Installation
 
-1. **Install dependencies**:
+1. **Clone and navigate to the project**:
    ```bash
-   pip install -r requirements.txt
+   cd IntegraChat
    ```
 
-2. **Start the Gradio app**:
+2. **Create and activate virtual environment** (recommended):
    ```bash
-   python app.py
+   # Windows
+   python -m venv venv
+   venv\Scripts\activate
+   
+   # Linux/Mac
+   python3 -m venv venv
+   source venv/bin/activate
    ```
 
-3. **Access the interface**:
-   - Local: `http://localhost:7860`
-   - The app will automatically connect to the backend at `http://localhost:8000`
-
-### Usage
-
-The Gradio UI exposes four tabs once you launch `app.py`:
-
-1. **Chat** – enter your Tenant ID, ask questions, and see multi-tool MCP responses with autonomous tool orchestration.
+3. **Install Python dependencies**:
+   ```bash
+   pip install -r requirements.txt
+   ```
 
-2. **Document Ingestion** – toggle between Raw Text, URL, or File Upload to populate the tenant RAG index. View and manage your ingested documents with delete functionality.
+4. **Configure environment variables**:
+   ```bash
+   cp env.example .env
+   # Edit .env with your credentials:
+   # - SUPABASE_URL and SUPABASE_SERVICE_KEY (for production storage)
+   # - POSTGRESQL_URL (for RAG vector database)
+   # - OLLAMA_URL/OLLAMA_MODEL or GROQ_API_KEY (for LLM)
+   # - GOOGLE_SEARCH_API_KEY and GOOGLE_SEARCH_CX_ID (optional, for web search)
+   ```
 
-3. **Knowledge Base Library** – comprehensive document management interface with:
-   - **Statistics Dashboard**: Visual cards showing total documents, document types (Text, PDF, FAQ, Link), and average length
-   - **Interactive Charts**: Plotly pie chart displaying document type distribution
-   - **Semantic Search**: Search your knowledge base with relevance scoring
-   - **Type Filtering**: Filter documents by type (all, text, pdf, faq, link)
-   - **Document Management**: View all documents in a table with preview, delete individual documents, or delete all at once
-   - **Auto-refresh**: Document lists automatically update after ingestion or deletion
+5. **Set up Supabase** (recommended for production):
+   - Create a Supabase project at [supabase.com](https://supabase.com)
+   - Run `supabase_admin_rules_table.sql` in Supabase SQL Editor
+   - Run `supabase_analytics_tables.sql` in Supabase SQL Editor
+   - Copy your project URL and service role key to `.env`
+   - Verify setup: `python verify_supabase_setup.py`
 
-4. **Admin Analytics** – comprehensive analytics dashboard with visualizations:
-   - **Statistics Cards**: Total queries, active users, red flags, and RAG searches
-   - **Interactive Bar Charts**: 
-     - Tool Usage Count (RAG, Web, Admin tools)
-     - Average Tool Latency (performance metrics)
-     - RAG Quality Metrics (hits, scores, recall indicators)
-   - **Tool Usage Table**: Detailed breakdown of tool performance with counts, latency, success/error rates, and token usage
-   - **Formatted Summary**: Key metrics displayed in an easy-to-read format
-   - Click "🔄 Fetch Analytics Snapshot" to load the latest data
+6. **Start the services**:
 
-5. **Admin Rules & Compliance** – comprehensive rule management with:
-   - **Text Input**: Paste rules one per line (comment lines starting with # are automatically ignored)
-   - **File Upload**: Upload rules from TXT, PDF, DOC, or DOCX files with drag-and-drop support
-   - **LLM Enhancement**: Rules are automatically enhanced by LLM to identify edge cases, improve patterns, and suggest severity levels
-   - **Chunk Processing**: Large rule sets processed in chunks (5 rules at a time) to avoid timeouts
-   - **Rule-Based Behavior**: Rules are checked FIRST before normal processing - brief response rules (low severity) return quick responses, blocking rules (high severity) block requests
-   - **Streaming Responses**: Chat responses stream word-by-word for better user experience
-   - **Refresh Button**: Refresh rules directly from the Rule Set table
+   **Option A: Windows Quick Start** (recommended for Windows):
+   ```bash
+   start.bat
+   ```
+   This automatically starts:
+   - FastAPI backend on port 8000
+   - Unified MCP server on port 8900
 
-**Tip:** Every action requires a tenant ID. The tenant ID is now managed centrally and persists across page refreshes. The Knowledge Base Library and Admin Analytics tabs feature beautiful, modern UI with dark theme styling and interactive Plotly visualizations.
+   **Option B: Manual Start**:
+   ```bash
+   # Terminal 1: FastAPI backend
+   uvicorn backend.api.main:app --port 8000 --reload
+   
+   # Terminal 2: Unified MCP server
+   python backend/mcp_server/server.py
+   ```
 
-### Frontend (Next.js) Operator Console
+7. **Launch the UI**:
 
-The companion Next.js frontend (`frontend/`) now exposes dedicated pages for each workflow:
+   **Gradio Interface** (full-featured):
+   ```bash
+   python app.py
+   ```
+   Access at `http://localhost:7860`
 
-| URL | Description |
+   **Next.js Frontend** (optional, modern UI):
+   ```bash
+   cd frontend
+   npm install
+   npm run dev
+   ```
+   Access at `http://localhost:3000`
+
+## Usage
+
+### Gradio Interface (`app.py`)
+
+The Gradio UI provides a comprehensive interface with five main tabs:
+
+#### 1. **Chat** 💬
+- Enter your Tenant ID and start chatting with the MCP-powered agent
+- Real-time streaming responses (word-by-word using SSE)
+- Autonomous tool orchestration (RAG, Web, Admin, LLM)
+- Multi-step planning with memory of previous tool outputs
+
+#### 2. **Document Ingestion** 📚
+- **Raw Text**: Paste text directly
+- **URL**: Ingest content from web URLs
+- **File Upload**: Upload PDF, DOCX, TXT, or Markdown files
+- Rich metadata support (filename, URL, document ID, custom JSON)
+- View and manage ingested documents
+
+#### 3. **Knowledge Base Library** 📖
+- **Statistics Dashboard**: Visual cards showing document counts by type
+- **Interactive Charts**: Plotly pie chart for document type distribution
+- **Semantic Search**: Search knowledge base with relevance scoring
+- **Type Filtering**: Filter by document type (text, PDF, FAQ, link)
+- **Document Management**: View, preview, and delete documents
+- **Auto-refresh**: Lists update automatically after operations
+
+#### 4. **Admin Analytics** 📊
+- **Statistics Cards**: Total queries, active users, red flags, RAG searches
+- **Interactive Bar Charts**: 
+  - Tool Usage Count (RAG, Web, Admin, LLM)
+  - Average Tool Latency (performance metrics)
+  - RAG Quality Metrics (hits, scores, recall indicators)
+- **Tool Usage Table**: Detailed performance breakdown
+- **Formatted Summary**: Key metrics in easy-to-read format
+- Click "🔄 Fetch Analytics Snapshot" to load latest data
+
+#### 5. **Admin Rules & Compliance** 🛡️
+- **Text Input**: Paste rules one per line (comments starting with # are ignored)
+- **File Upload**: Upload rules from TXT, PDF, DOC, or DOCX files
+- **LLM Enhancement**: Automatic rule enhancement (edge cases, pattern improvements, severity suggestions)
+- **Chunk Processing**: Large rule sets processed in chunks (5 at a time)
+- **Rule-Based Behavior**: Rules checked FIRST - brief responses or blocking based on severity
+- **Streaming Responses**: Real-time word-by-word streaming
+- **Refresh Button**: Update rules table directly
+
+> **💡 Tip:** Every action requires a Tenant ID. The Tenant ID persists across page refreshes and is managed centrally.
+
+### Next.js Frontend (`frontend/`)
+
+The modern Next.js operator console provides dedicated pages for each workflow:
+
+| Route | Description |
 | --- | --- |
-| `/` | Landing page with hero + quick access panels |
+| `/` | Landing page with hero section and quick access panels |
 | `/ingestion` | Data ingestion walkthrough (text/URL/files) with document management |
-| `/chat` | Chat console wrapper around the MCP agent |
-| `/analytics` | Analytics overview and explainer |
-| `/admin-rules` | Admin rule ingestion explainer |
-| `/knowledge-base` | View all ingested documents with search, filter, and delete functionality |
+| `/chat` | Chat console with MCP agent integration |
+| `/analytics` | Analytics overview with visualizations |
+| `/admin-rules` | Admin rule management interface |
+| `/knowledge-base` | Document library with search, filter, and delete functionality |
 
 **Key Features:**
-- **Centralized Tenant ID Management** – Tenant ID is managed globally via React Context and persists in localStorage
-- **Document Management** – View, search, filter, and delete documents from the knowledge base
-- **Improved Error Handling** – Clear error messages with retry options for failed operations
-- **Real-time Updates** – Document lists automatically refresh after ingestion or deletion
+- **Centralized Tenant ID Management** – Global React Context with localStorage persistence
+- **Document Management** – Full CRUD operations for knowledge base
+- **Improved Error Handling** – Clear error messages with retry options
+- **Real-time Updates** – Automatic refresh after operations
+- **Modern UI** – Tailwind CSS with responsive design
 
-Run the console locally with:
+**To run:**
 ```bash
 cd frontend
 npm install
 npm run dev
 ```
-Then open `http://localhost:3000`. The navbar links on the landing page route to each section, and you can link directly to those URLs for demo purposes. The tenant ID selector is available in the navbar on all pages.
+Then open `http://localhost:3000`. The tenant ID selector is available in the navbar on all pages.
 
 ---
 
 ## API Endpoints
 
+All endpoints are served by the FastAPI backend at `http://localhost:8000`. Most endpoints require the `x-tenant-id` header for tenant isolation.
+
+> **📖 API Documentation**: Interactive Swagger docs available at `http://localhost:8000/docs` when the backend is running.
+
 ### Agent Endpoints
 
-| Purpose | Method & Path | Description |
+| Method | Endpoint | Description |
 | --- | --- | --- |
-| Chat with agent | `POST /agent/message` | Main chat endpoint with `tenant_id`, `message`, optional history |
-| Chat with agent (streaming) | `POST /agent/message/stream` | Streams response word-by-word using Server-Sent Events (SSE). Returns status messages and tokens as they're generated |
-| Agent debug | `POST /agent/debug` | Returns detailed debugging info: reasoning trace, tool selection, intent classification |
-| Agent plan | `POST /agent/plan` | Returns tool selection plan without execution (intent, tool scores, planned steps) |
+| `POST` | `/agent/message` | Main chat endpoint with `tenant_id`, `message`, optional history |
+| `POST` | `/agent/message/stream` | Streaming chat endpoint using Server-Sent Events (SSE). Returns tokens word-by-word |
+| `POST` | `/agent/debug` | Detailed debugging info: reasoning trace, tool selection, intent classification |
+| `POST` | `/agent/plan` | Tool selection plan without execution (intent, tool scores, planned steps) |
 
 ### RAG Endpoints
 
-| Purpose | Method & Path | Description |
+| Method | Endpoint | Description |
 | --- | --- | --- |
-| Ingest document | `POST /rag/ingest-document` | Accepts `source_type`, `content`, metadata (filename, URL, doc_id). Supports raw text, URLs, PDFs, DOCX, TXT, and Markdown files |
-| Ingest file | `POST /rag/ingest-file` | Multipart upload with `x-tenant-id` header (PDF/DOCX/TXT/MD) |
-| List documents | `GET /rag/list?tenant_id={id}&limit={n}&offset={n}` | Returns all documents for a tenant with pagination. Requires `x-tenant-id` header or `tenant_id` query parameter |
-| Delete document | `DELETE /rag/delete/{document_id}?tenant_id={id}` | Deletes a specific document by ID. Requires `x-tenant-id` header or `tenant_id` query parameter |
-| Delete all documents | `DELETE /rag/delete-all?tenant_id={id}` | Deletes all documents for a tenant. Requires `x-tenant-id` header or `tenant_id` query parameter |
+| `POST` | `/rag/ingest-document` | Ingest document with `source_type`, `content`, metadata. Supports raw text, URLs, PDFs, DOCX, TXT, Markdown |
+| `POST` | `/rag/ingest-file` | Multipart file upload (PDF/DOCX/TXT/MD) with `x-tenant-id` header |
+| `GET` | `/rag/list?tenant_id={id}&limit={n}&offset={n}` | List all documents for a tenant with pagination |
+| `DELETE` | `/rag/delete/{document_id}?tenant_id={id}` | Delete a specific document by ID |
+| `DELETE` | `/rag/delete-all?tenant_id={id}` | Delete all documents for a tenant |
+
+**Note:** RAG endpoints support both `x-tenant-id` header and `tenant_id` query parameter.
 
 ### Admin & Governance Endpoints
 
-| Purpose | Method & Path | Description |
+| Method | Endpoint | Description |
 | --- | --- | --- |
-| List rules | `GET /admin/rules?detailed=true` | Get all rules (use `detailed=true` for regex/severity metadata) |
-| Add rule | `POST /admin/rules?enhance=true` | Add single rule with optional `pattern` (regex), `severity` (low/medium/high/critical), `description`. Set `enhance=true` for LLM enhancement |
-| Add rules bulk | `POST /admin/rules/bulk?enhance=true` | Add multiple rules at once. Processed in chunks of 5 to avoid timeouts. LLM enhancement applied automatically |
-| Upload rules file | `POST /admin/rules/upload-file?enhance=true` | Upload rules from file (TXT, PDF, DOC, DOCX). Text extracted server-side, rules processed with LLM enhancement |
-| Delete rule | `DELETE /admin/rules/{rule}` | Delete a specific rule |
-| List violations | `GET /admin/violations?days=30&limit=50` | Get red-flag violations with timestamps and confidence scores |
-| Tool logs | `GET /admin/tools/logs?tool_name=rag&days=7` | Get detailed tool usage logs with latency and token counts |
-| Manage tenants | `GET/POST/DELETE /admin/tenants` | Tenant management endpoints (placeholder implementation) |
-| Setup Supabase table | `POST /admin/setup/table` | Create admin_rules table in Supabase if it doesn't exist |
+| `GET` | `/admin/rules?detailed=true` | Get all rules (use `detailed=true` for regex/severity metadata) |
+| `POST` | `/admin/rules?enhance=true` | Add single rule with optional `pattern` (regex), `severity`, `description`. Set `enhance=true` for LLM enhancement |
+| `POST` | `/admin/rules/bulk?enhance=true` | Add multiple rules at once (processed in chunks of 5). LLM enhancement applied automatically |
+| `POST` | `/admin/rules/upload-file?enhance=true` | Upload rules from file (TXT, PDF, DOC, DOCX). Text extracted server-side |
+| `DELETE` | `/admin/rules/{rule}` | Delete a specific rule |
+| `GET` | `/admin/violations?days=30&limit=50` | Get red-flag violations with timestamps and confidence scores |
+| `GET` | `/admin/tools/logs?tool_name=rag&days=7` | Get detailed tool usage logs with latency and token counts |
+| `GET/POST/DELETE` | `/admin/tenants` | Tenant management endpoints |
+| `POST` | `/admin/setup/table` | Create admin_rules table in Supabase if it doesn't exist |
 
 ### Analytics Endpoints
 
-| Purpose | Method & Path | Description |
+| Method | Endpoint | Description |
 | --- | --- | --- |
-| Overview | `GET /analytics/overview?days=30` | Comprehensive analytics: total queries, tool usage, red-flag count, RAG quality |
-| Tool usage | `GET /analytics/tool-usage?days=30` | Detailed tool usage stats: counts, latency, tokens, success/error rates |
-| Red flags | `GET /analytics/redflags?limit=50&days=30` | Recent red-flag violations for tenant |
-| Activity | `GET /analytics/activity?days=30` | Tenant activity summary: queries, active users, last query timestamp |
-| RAG quality | `GET /analytics/rag-quality?days=30` | RAG quality metrics: avg hits, scores, latency (recall/precision indicators) |
+| `GET` | `/analytics/overview?days=30` | Comprehensive analytics: total queries, tool usage, red-flag count, RAG quality |
+| `GET` | `/analytics/tool-usage?days=30` | Detailed tool usage stats: counts, latency, tokens, success/error rates |
+| `GET` | `/analytics/redflags?limit=50&days=30` | Recent red-flag violations for tenant |
+| `GET` | `/analytics/activity?days=30` | Tenant activity summary: queries, active users, last query timestamp |
+| `GET` | `/analytics/rag-quality?days=30` | RAG quality metrics: avg hits, scores, latency (recall/precision indicators) |
+
+### Request Headers
 
-All calls are proxied through the FastAPI backend running at `http://localhost:8000`. Ensure those services are online before launching the Space.
+Most endpoints require:
+- `x-tenant-id`: Tenant identifier for multi-tenant isolation
+- `Content-Type: application/json`: For POST requests with JSON payloads
+
+### Example Request
+
+```bash
+curl -X POST http://localhost:8000/agent/message \
+  -H "Content-Type: application/json" \
+  -H "x-tenant-id: tenant123" \
+  -d '{
+    "message": "What is our refund policy?",
+    "tenant_id": "tenant123"
+  }'
+```
 
 ---
 
-## Architecture Highlights
+## Architecture
+
+### System Overview
+
+IntegraChat follows a modular architecture with clear separation of concerns:
+
+```
+┌─────────────────┐
+│   Frontend UI   │  (Gradio + Next.js)
+│  Port 7860/3000 │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│  FastAPI Backend│  (API Gateway)
+│    Port 8000    │
+└────────┬────────┘
+         │
+         ├──► Unified MCP Server (Port 8900)
+         │    ├── RAG Tools (search, ingest, list, delete)
+         │    ├── Web Tools (search)
+         │    └── Admin Tools (rules, violations)
+         │
+         ├──► PostgreSQL/Supabase (RAG Vector Store)
+         ├──► Supabase/SQLite (Rules & Analytics)
+         └──► LLM Backend (Ollama/Groq)
+```
 
 ### Enterprise-Grade Features
 
-1. **Autonomous Multi-Step Planning**: The agent uses LLM-powered planning to determine optimal tool sequences, with memory of previous tool outputs in multi-step workflows.
+1. **Autonomous Multi-Step Planning**: LLM-powered planning determines optimal tool sequences with memory of previous tool outputs in multi-step workflows.
 
 2. **Regex-Based Governance**: Admin rules support regex patterns with fallback to keyword matching and semantic similarity scoring for flexible policy enforcement.
 
-3. **Comprehensive Analytics**: All tool usage, RAG searches, LLM calls, and red-flag violations are logged to SQLite with indexed queries for fast analytics retrieval.
+3. **Comprehensive Analytics**: All tool usage, RAG searches, LLM calls, and red-flag violations are logged with indexed queries for fast analytics retrieval.
 
-4. **Enhanced RAG Pipeline**: Documents are chunked with optimal size (400-600 tokens) and enriched with metadata (source URL, timestamp, document type) for better retrieval.
+4. **Enhanced RAG Pipeline**: Documents chunked optimally (400-600 tokens) and enriched with metadata (source URL, timestamp, document type) for better retrieval.
 
-5. **Structured Error Handling**: All errors are logged with context, and the system gracefully falls back (e.g., if RAG fails → use LLM-only, if web fails → skip web).
+5. **Structured Error Handling**: All errors logged with context, with graceful fallbacks (e.g., RAG fails → LLM-only, web fails → skip web).
 
-### Data Storage
+### Data Storage Architecture
+
+IntegraChat uses **dual-backend storage** with automatic fallback for production flexibility:
+
+#### Supabase (Production/Preferred)
+
+**When to use:** Production deployments, multi-user environments, scalable applications
+
+**Storage:**
+- `admin_rules` - Admin rules with regex patterns and severity levels
+- `tool_usage_events` - Tool invocation logs with latency and token tracking
+- `redflag_violations` - Red-flag violation events with timestamps
+- `rag_search_events` - RAG search metrics and quality indicators
+- `agent_query_events` - Agent query logs and analytics
+
+**Features:**
+- Row Level Security (RLS) for multi-tenant isolation
+- Automatic backups and scaling
+- Real-time capabilities
+- Production-ready infrastructure
 
-IntegraChat supports **dual-backend storage** with automatic fallback:
+**Setup:** Configure `SUPABASE_URL` and `SUPABASE_SERVICE_KEY` in `.env`
 
-- **Supabase (Production/Preferred)**:
-  - `admin_rules` table - Admin rules with regex patterns and severity
-  - `tool_usage_events`, `redflag_violations`, `rag_search_events`, `agent_query_events` - Analytics tables
-  - Automatically used when `SUPABASE_URL` and `SUPABASE_SERVICE_KEY` are configured
-  - Supports Row Level Security (RLS) for multi-tenant isolation
-  - Scalable, production-ready with automatic backups
+#### SQLite (Development Fallback)
 
-- **SQLite (Development Fallback)**:
-  - `data/admin_rules.db` - Admin rules (local)
-  - `data/analytics.db` - Analytics events (local)
-  - Used automatically when Supabase credentials are not available
-  - Perfect for local development and testing
+**When to use:** Local development, testing, single-user scenarios
 
-**Migration**: Use `python migrate_sqlite_to_supabase.py` to copy existing SQLite data to Supabase. See `SUPABASE_SETUP.md` for detailed setup instructions.
+**Storage:**
+- `data/admin_rules.db` - Admin rules (local file)
+- `data/analytics.db` - Analytics events (local file)
+
+**Features:**
+- Zero configuration required
+- Perfect for local development
+- Automatic fallback when Supabase not configured
+
+**Migration:** Use `python migrate_sqlite_to_supabase.py` to migrate existing SQLite data to Supabase. See `SUPABASE_SETUP.md` for detailed instructions.
 
 ---
 
@@ -269,6 +463,79 @@ See `SUPABASE_SETUP.md` and `SUPABASE_MIGRATION_COMPLETE.md` for detailed instru
 
 ---
 
+## Troubleshooting
+
+### Common Issues
+
+#### Backend Not Starting
+- **Issue**: FastAPI backend fails to start
+- **Solution**: 
+  - Check if port 8000 is already in use: `netstat -ano | findstr :8000` (Windows) or `lsof -i :8000` (Linux/Mac)
+  - Verify Python virtual environment is activated
+  - Check `.env` file exists and has required variables
+  - Review error logs for missing dependencies
+
+#### MCP Server Connection Errors
+- **Issue**: "Could not connect to MCP server" errors
+- **Solution**:
+  - Ensure unified MCP server is running: `python backend/mcp_server/server.py`
+  - Check MCP server is on port 8900 (default)
+  - Verify `MCP_SERVER_ID` in `.env` matches server configuration
+  - Check firewall settings if running on different machines
+
+#### RAG Search Not Returning Results
+- **Issue**: RAG searches return no results despite ingested documents
+- **Solution**:
+  - Check similarity threshold (default 0.3) - try lowering to 0.2 or 0.1
+  - Verify documents exist: `GET /rag/list?tenant_id={id}`
+  - Ensure tenant_id matches between ingestion and search
+  - Check PostgreSQL/pgvector connection and vector extension
+  - Review MCP server logs for search metrics
+
+#### Supabase Configuration Issues
+- **Issue**: Data still going to SQLite instead of Supabase
+- **Solution**:
+  - Verify `SUPABASE_URL` and `SUPABASE_SERVICE_KEY` in `.env` (no quotes, no spaces)
+  - Use **service_role** key (not anon key) from Supabase Dashboard
+  - Run `python verify_supabase_setup.py` to check configuration
+  - Ensure tables exist: run SQL scripts in Supabase SQL Editor
+  - Check FastAPI startup logs for backend detection messages
+
+#### LLM Connection Errors
+- **Issue**: Agent responses fail with LLM errors
+- **Solution**:
+  - For Ollama: Ensure Ollama is running (`ollama serve`)
+  - Check `OLLAMA_URL` and `OLLAMA_MODEL` in `.env`
+  - For Groq: Verify `GROQ_API_KEY` is set correctly
+  - Check `LLM_BACKEND` setting (ollama or groq)
+  - Test LLM connection: `curl http://localhost:11434/api/tags` (Ollama)
+
+#### Document Ingestion Failures
+- **Issue**: File uploads or document ingestion fails
+- **Solution**:
+  - Check file size limits (default may be 10MB)
+  - Verify file format is supported (PDF, DOCX, TXT, MD)
+  - Ensure tenant_id is provided in request
+  - Check backend logs for specific error messages
+  - Verify PostgreSQL connection for RAG storage
+
+#### Tenant Isolation Issues
+- **Issue**: Documents or data leaking between tenants
+- **Solution**:
+  - Run `python verify_tenant_isolation.py` to test isolation
+  - Check database queries include `WHERE tenant_id = ...` filters
+  - Verify tenant ID normalization is working correctly
+  - Review `python check_rag_database.py` output for tenant IDs
+
+### Getting Help
+
+1. **Check Logs**: Review FastAPI and MCP server logs for detailed error messages
+2. **Run Diagnostics**: Use helper scripts in the Testing & Diagnostics section
+3. **Verify Configuration**: Run `python verify_supabase_setup.py` and check `.env` file
+4. **Review Documentation**: See `backend/README.md` for backend-specific issues
+
+---
+
 ## Testing & Diagnostics
 
 IntegraChat ships with several helper scripts to validate the full stack end-to-end:
@@ -331,17 +598,38 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 
 ## Technical Stack
 
-- **Backend**: FastAPI with async/await for high-performance MCP orchestration
-- **Frontend**: Gradio interface with Plotly visualizations + Next.js operator console
-- **UI Libraries**: Plotly for interactive charts, Gradio for web interface
-- **LLM Integration**: Ollama (local) or Groq (cloud) via configurable backend with streaming support
-- **Vector Store**: pgvector (via Supabase) or SQLite embeddings
-- **Analytics**: Supabase (production) or SQLite (development) with indexed queries for fast analytics
-- **Rules Storage**: Supabase (production) or SQLite (development) with automatic detection and fallback
+### Backend
+- **Framework**: FastAPI with async/await for high-performance MCP orchestration
 - **MCP Server**: Unified MCP server (port 8900) exposing all tools via namespaces
-- **Database**: PostgreSQL with pgvector extension for RAG embeddings, SQLite for analytics
-- **File Processing**: Support for TXT, PDF, DOC, DOCX with server-side text extraction (PyPDF2, python-docx)
+- **API**: RESTful API with Server-Sent Events (SSE) for streaming responses
+- **LLM Integration**: 
+  - Ollama (local, default) - `http://localhost:11434`
+  - Groq (cloud) - via API key
+  - Configurable backend with streaming support
+
+### Frontend
+- **Gradio UI**: Full-featured interface with Plotly visualizations (`app.py`)
+- **Next.js Console**: Modern React-based operator console (`frontend/`)
+- **UI Libraries**: 
+  - Plotly for interactive charts and visualizations
+  - Tailwind CSS for modern styling (Next.js)
+  - React 19 with TypeScript
+
+### Data Storage
+- **RAG Vector Store**: PostgreSQL with pgvector extension (via Supabase or direct connection)
+- **Analytics**: Supabase (production) or SQLite (development) with indexed queries
+- **Rules Storage**: Supabase (production) or SQLite (development) with automatic fallback
+- **Database**: PostgreSQL for RAG embeddings, Supabase/SQLite for analytics and rules
+
+### File Processing
+- **Supported Formats**: TXT, PDF, DOC, DOCX, Markdown
+- **Libraries**: PyPDF2, python-docx for server-side text extraction
+- **Metadata**: Rich metadata support (source URL, timestamp, document type)
+
+### Communication
 - **Streaming**: Server-Sent Events (SSE) for real-time word-by-word response streaming
+- **Protocol**: Model Context Protocol (MCP) for tool communication
+- **HTTP**: RESTful endpoints with JSON payloads
 
 ## Recent Enhancements