README.md

# IntegraChat — Enterprise MCP Autonomous Agent Platform

**Track:** MCP in Action  
**Category:** Enterprise  
**Tag:** `mcp-in-action-track-enterprise`

---

## Overview

**IntegraChat** is an enterprise-grade, multi-tenant AI platform that demonstrates the full capabilities of the **Model Context Protocol (MCP)** in a production-style environment. Built with enterprise governance and observability in mind, IntegraChat combines autonomous tool-using agents, RAG retrieval, live web search, and admin compliance under strict tenant isolation.

This platform showcases how MCP can power intelligent, governed, multi-tenant AI systems with real-time analytics, regex-based red-flag detection, and comprehensive tool orchestration.

---

## Features

### Core Capabilities

- 🤖 **Autonomous Multi-Step MCP Agents** – Intelligent tool-aware agent that plans and executes multi-step workflows across RAG, Web, Admin, and LLM tools with memory of previous tool outputs
- 📚 **Enhanced Knowledge Base Management** – Upload raw text, URLs, or documents (PDF/DOCX/TXT/MD) with rich metadata (source URL, timestamp, document type) and optimized chunking (400-600 tokens)
- 🔍 **Optimized RAG Search** – Semantic search with configurable similarity threshold (default 0.3) for better recall, with fallback to return top results even if below threshold
- 🗑️ **Document Management** – Delete individual documents or bulk delete all documents for a tenant with confirmation dialogs
- 🛡️ **Enterprise Admin Governance** – Regex-based red-flag pattern matching with severity levels (low/medium/high/critical) and automatic admin alerts
- 📊 **Comprehensive Analytics & Observability** – Full tenant-level analytics logging with SQLite backend:
  - Tool usage breakdown (RAG, Web, Admin, LLM) with latency and token tracking
  - RAG recall/precision indicators (average hits, scores, top scores)
  - Per-tenant query volume and active users
  - Red-flag violations with timestamps and confidence scores
  - LLM token logs and latency metrics
- 🌐 **Live Web Search** – DuckDuckGo-based MCP server with English-biased results
- 🏢 **Multi-Tenant Isolation** – Complete tenant isolation with centralized tenant ID management; backend enforces strict isolation for chat, ingestion, and admin ops
- 🔄 **Intelligent Multi-Tool Orchestration** – MCP agent orchestrator autonomously selects optimal tool chains (RAG + Web + LLM, etc.) based on query intent and context
- ⚡ **Robust Error Handling** – Structured error responses, retry mechanisms, and graceful fallbacks (e.g., if RAG fails → fallback to LLM-only)

### Enterprise Features

- 🔍 **Regex-Based Red-Flag Detection** – Support for complex regex patterns with keyword fallback and semantic scoring
- 📈 **Real-Time Analytics Dashboard** – Per-tenant analytics with configurable time windows (7, 30, 90 days)
- 🛠️ **Admin API Endpoints** – `/admin/violations`, `/admin/tools/logs`, `/admin/tenants` for comprehensive governance
- 🧠 **Agent Debug & Planning** – `/agent/debug` and `/agent/plan` endpoints for observability and tool selection inspection
- 💾 **Persistent Analytics Storage** – SQLite-based analytics store with indexes for fast queries

---

## How to Run the Space
### Prerequisites

1. **Backend services running**:
   - FastAPI API (`uvicorn backend.api.main:app --port 8000`)
   - Unified MCP server (port 8900) as described in `backend/README.md`
   - Optional: Ollama / Groq credentials for the LLM client
   
   **Quick Start**: Run `start.bat` (Windows) to launch all services automatically.
2. **Python 3.10+** with the dependencies in `requirements.txt`

### Installation

1. **Install dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

2. **Start the Gradio app**:
   ```bash
   python app.py
   ```

3. **Access the interface**:
   - Local: `http://localhost:7860`
   - The app will automatically connect to the backend at `http://localhost:8000`

### Usage

The Gradio UI exposes four tabs once you launch `app.py`:

1. **Chat** – enter your Tenant ID, ask questions, and see multi-tool MCP responses with autonomous tool orchestration.

2. **Document Ingestion** – toggle between Raw Text, URL, or File Upload to populate the tenant RAG index. View and manage your ingested documents with delete functionality.

3. **Knowledge Base Library** – comprehensive document management interface with:
   - **Statistics Dashboard**: Visual cards showing total documents, document types (Text, PDF, FAQ, Link), and average length
   - **Interactive Charts**: Plotly pie chart displaying document type distribution
   - **Semantic Search**: Search your knowledge base with relevance scoring
   - **Type Filtering**: Filter documents by type (all, text, pdf, faq, link)
   - **Document Management**: View all documents in a table with preview, delete individual documents, or delete all at once
   - **Auto-refresh**: Document lists automatically update after ingestion or deletion

4. **Admin Analytics** – comprehensive analytics dashboard with visualizations:
   - **Statistics Cards**: Total queries, active users, red flags, and RAG searches
   - **Interactive Bar Charts**: 
     - Tool Usage Count (RAG, Web, Admin tools)
     - Average Tool Latency (performance metrics)
     - RAG Quality Metrics (hits, scores, recall indicators)
   - **Tool Usage Table**: Detailed breakdown of tool performance with counts, latency, success/error rates, and token usage
   - **Formatted Summary**: Key metrics displayed in an easy-to-read format
   - Click "🔄 Fetch Analytics Snapshot" to load the latest data

5. **Admin Rules & Compliance** – upload/delete governance rules that are stored via the backend `/admin/rules` API.

**Tip:** Every action requires a tenant ID. The tenant ID is now managed centrally and persists across page refreshes. The Knowledge Base Library and Admin Analytics tabs feature beautiful, modern UI with dark theme styling and interactive Plotly visualizations.

### Frontend (Next.js) Operator Console

The companion Next.js frontend (`frontend/`) now exposes dedicated pages for each workflow:

| URL | Description |
| --- | --- |
| `/` | Landing page with hero + quick access panels |
| `/ingestion` | Data ingestion walkthrough (text/URL/files) with document management |
| `/chat` | Chat console wrapper around the MCP agent |
| `/analytics` | Analytics overview and explainer |
| `/admin-rules` | Admin rule ingestion explainer |
| `/knowledge-base` | View all ingested documents with search, filter, and delete functionality |

**Key Features:**
- **Centralized Tenant ID Management** – Tenant ID is managed globally via React Context and persists in localStorage
- **Document Management** – View, search, filter, and delete documents from the knowledge base
- **Improved Error Handling** – Clear error messages with retry options for failed operations
- **Real-time Updates** – Document lists automatically refresh after ingestion or deletion

Run the console locally with:
```bash
cd frontend
npm install
npm run dev
```
Then open `http://localhost:3000`. The navbar links on the landing page route to each section, and you can link directly to those URLs for demo purposes. The tenant ID selector is available in the navbar on all pages.

---

## API Endpoints

### Agent Endpoints

| Purpose | Method & Path | Description |
| --- | --- | --- |
| Chat with agent | `POST /agent/message` | Main chat endpoint with `tenant_id`, `message`, optional history |
| Agent debug | `POST /agent/debug` | Returns detailed debugging info: reasoning trace, tool selection, intent classification |
| Agent plan | `POST /agent/plan` | Returns tool selection plan without execution (intent, tool scores, planned steps) |

### RAG Endpoints

| Purpose | Method & Path | Description |
| --- | --- | --- |
| Ingest document | `POST /rag/ingest-document` | Accepts `source_type`, `content`, metadata (filename, URL, doc_id). Supports raw text, URLs, PDFs, DOCX, TXT, and Markdown files |
| Ingest file | `POST /rag/ingest-file` | Multipart upload with `x-tenant-id` header (PDF/DOCX/TXT/MD) |
| List documents | `GET /rag/list?tenant_id={id}&limit={n}&offset={n}` | Returns all documents for a tenant with pagination. Requires `x-tenant-id` header or `tenant_id` query parameter |
| Delete document | `DELETE /rag/delete/{document_id}?tenant_id={id}` | Deletes a specific document by ID. Requires `x-tenant-id` header or `tenant_id` query parameter |
| Delete all documents | `DELETE /rag/delete-all?tenant_id={id}` | Deletes all documents for a tenant. Requires `x-tenant-id` header or `tenant_id` query parameter |

### Admin & Governance Endpoints

| Purpose | Method & Path | Description |
| --- | --- | --- |
| List rules | `GET /admin/rules?detailed=true` | Get all rules (use `detailed=true` for regex/severity metadata) |
| Add rule | `POST /admin/rules` | Add rule with optional `pattern` (regex), `severity` (low/medium/high/critical), `description` |
| Delete rule | `DELETE /admin/rules/{rule}` | Delete a specific rule |
| List violations | `GET /admin/violations?days=30&limit=50` | Get red-flag violations with timestamps and confidence scores |
| Tool logs | `GET /admin/tools/logs?tool_name=rag&days=7` | Get detailed tool usage logs with latency and token counts |
| Manage tenants | `GET/POST/DELETE /admin/tenants` | Tenant management endpoints (placeholder implementation) |

### Analytics Endpoints

| Purpose | Method & Path | Description |
| --- | --- | --- |
| Overview | `GET /analytics/overview?days=30` | Comprehensive analytics: total queries, tool usage, red-flag count, RAG quality |
| Tool usage | `GET /analytics/tool-usage?days=30` | Detailed tool usage stats: counts, latency, tokens, success/error rates |
| Red flags | `GET /analytics/redflags?limit=50&days=30` | Recent red-flag violations for tenant |
| Activity | `GET /analytics/activity?days=30` | Tenant activity summary: queries, active users, last query timestamp |
| RAG quality | `GET /analytics/rag-quality?days=30` | RAG quality metrics: avg hits, scores, latency (recall/precision indicators) |

All calls are proxied through the FastAPI backend running at `http://localhost:8000`. Ensure those services are online before launching the Space.

---

## Architecture Highlights

### Enterprise-Grade Features

1. **Autonomous Multi-Step Planning**: The agent uses LLM-powered planning to determine optimal tool sequences, with memory of previous tool outputs in multi-step workflows.

2. **Regex-Based Governance**: Admin rules support regex patterns with fallback to keyword matching and semantic similarity scoring for flexible policy enforcement.

3. **Comprehensive Analytics**: All tool usage, RAG searches, LLM calls, and red-flag violations are logged to SQLite with indexed queries for fast analytics retrieval.

4. **Enhanced RAG Pipeline**: Documents are chunked with optimal size (400-600 tokens) and enriched with metadata (source URL, timestamp, document type) for better retrieval.

5. **Structured Error Handling**: All errors are logged with context, and the system gracefully falls back (e.g., if RAG fails → use LLM-only, if web fails → skip web).

### Data Storage

- **SQLite Databases** (for demo/development):
  - `data/admin_rules.db` - Admin rules with regex patterns and severity
  - `data/analytics.db` - Analytics events, tool usage, violations, RAG metrics

- **Production Ready**: Can easily swap SQLite for PostgreSQL/Supabase for production deployments.

---

## Testing & Diagnostics

IntegraChat ships with several helper scripts to validate the full stack end-to-end:

- `python verify_tenant_isolation.py`  
  Runs a comprehensive suite that covers analytics logging, admin rule storage, API reachability, and—most importantly—multi-tenant RAG isolation.  
  - ✅ **Prerequisites:** FastAPI backend plus all MCP servers (RAG/Web/Admin) running locally.  
  - ✅ **What it checks:**  
    1. Direct database writes via the analytics and rules stores  
    2. CRUD over the `/admin/*` and `/analytics/*` endpoints  
    3. RAG ingestion and isolation by issuing queries as multiple tenants and ensuring secrets never leak across IDs  
  - ✅ **Pass criteria:** At least 80 % of the sub-tests succeed (the RAG isolation test must pass for overall success).

- `python check_rag_database.py`  
  Provides a low-level inspection of the RAG datastore. It connects straight to the pgvector/Postgres instance, lists all tenant IDs, prints sample chunks, and runs `search_vectors()` directly to ensure the SQL `WHERE tenant_id = …` filter is behaving as expected. Use this script when diagnosing suspected cross-tenant leakage or when seeding demo data.

- `python test_manual.py`  
  The existing manual test runner remains useful for smoke-testing analytics logging, admin rule CRUD, and API response codes. Run it whenever you adjust schemas or update MCP endpoints.

> **Tip:** All scripts assume the Python virtual environment is active (`source venv/bin/activate` or `.\venv\Scripts\activate`) and that `.env` contains the MCP server URLs/LLM settings noted earlier.

---

## Demo Video

🎥 **[Demo Video Placeholder]** - Coming soon!

Watch how IntegraChat uses MCP to power autonomous agents with multi-tool selection, RAG retrieval, and enterprise governance.

---

## Social Media

📱 **[Social Media Post Placeholder]** - Coming soon!

Follow us for updates and demos of IntegraChat in action!

---

## Team Member(s)

- **Your Name Here** - Developer & MCP Enthusiast

---

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

## Technical Stack

- **Backend**: FastAPI with async/await for high-performance MCP orchestration
- **Frontend**: Gradio interface with Plotly visualizations + Next.js operator console
- **UI Libraries**: Plotly for interactive charts, Gradio for web interface
- **LLM Integration**: Ollama (local) or Groq (cloud) via configurable backend
- **Vector Store**: pgvector (via Supabase) or SQLite embeddings
- **Analytics**: SQLite with indexed queries for fast analytics
- **MCP Server**: Unified MCP server (port 8900) exposing all tools via namespaces
- **Database**: PostgreSQL with pgvector extension for RAG embeddings, SQLite for analytics

## Key Technical Features

### Tenant Isolation & Normalization
- **Strict tenant isolation** enforced at database level with `WHERE tenant_id = ...` filters
- **Automatic tenant ID normalization** handles whitespace and formatting differences
- Documents can be listed and deleted consistently across different tenant_id formats
- All operations validate tenant ownership before execution

### RAG Search & Retrieval
- **Optimized similarity threshold** (default 0.3) for better recall of relevant documents
- **Intelligent fallback** returns top result even if below threshold to ensure knowledge base content is accessible
- **Pattern-based tool selection** automatically triggers RAG for admin questions, fact lookups, and internal knowledge queries
- **Response unwrapping** ensures seamless integration between MCP server and orchestrator

### MCP Server Architecture
- **Unified server** running on a single port (default 8900) for all namespaced tools
- **Dual protocol support**: Both MCP protocol (POST with JSON) and RESTful HTTP (GET/DELETE)
- **Response wrapping**: Standardized response format with automatic unwrapping in clients
- **Error handling**: Comprehensive error responses with detailed messages for debugging

## UI Features

### Knowledge Base Library
- **Visual Statistics**: Real-time document counts and type distribution
- **Interactive Charts**: Plotly pie charts for document type visualization
- **Advanced Search**: Semantic search across all ingested documents with relevance scoring
- **Smart Filtering**: Filter by document type (text, PDF, FAQ, link)
- **Bulk Operations**: Delete individual documents or all documents at once
- **Auto-refresh**: Lists automatically update after operations

### Admin Analytics Dashboard
- **Statistics Cards**: Key metrics displayed in visually appealing cards with icons
- **Tool Usage Visualization**: Bar charts showing tool invocation counts and performance
- **Latency Metrics**: Visual representation of tool response times
- **RAG Quality Analysis**: Charts displaying search quality metrics (hits, scores, recall)
- **Detailed Tables**: Comprehensive tool usage breakdown with success/error rates
- **Dark Theme**: Modern UI with dark background and white text for better readability
- **Real-time Updates**: Fetch latest analytics data with a single click

## Acknowledgments

- Built with [Model Context Protocol (MCP)](https://modelcontextprotocol.io/)
- Powered by [Gradio](https://gradio.app/) for the interface
- Visualizations created with [Plotly](https://plotly.com/python/)
- Backend built with [FastAPI](https://fastapi.tiangolo.com/)
- Analytics and governance features inspired by enterprise AI platform requirements

---

<div align="center">

**Made with ❤️ for the MCP Hackathon**

**IntegraChat: Enterprise-Grade MCP Autonomous Agent Platform**

[⬆ Back to Top](#integrachat--enterprise-mcp-autonomous-agent-platform)

</div>