Spaces:

nothingworry
/

IntegraChat

Sleeping

App Files Files Community

IntegraChat / README.md

nothingworry

imporvement in the UI in app.py

b6650bb 21 days ago

preview code

raw

history blame

17.3 kB

IntegraChat — Enterprise MCP Autonomous Agent Platform

Track: MCP in Action
Category: Enterprise
Tag: mcp-in-action-track-enterprise

Overview

IntegraChat is an enterprise-grade, multi-tenant AI platform that demonstrates the full capabilities of the Model Context Protocol (MCP) in a production-style environment. Built with enterprise governance and observability in mind, IntegraChat combines autonomous tool-using agents, RAG retrieval, live web search, and admin compliance under strict tenant isolation.

This platform showcases how MCP can power intelligent, governed, multi-tenant AI systems with real-time analytics, regex-based red-flag detection, and comprehensive tool orchestration.

Features

Core Capabilities

🤖 Autonomous Multi-Step MCP Agents – Intelligent tool-aware agent that plans and executes multi-step workflows across RAG, Web, Admin, and LLM tools with memory of previous tool outputs
📚 Enhanced Knowledge Base Management – Upload raw text, URLs, or documents (PDF/DOCX/TXT/MD) with rich metadata (source URL, timestamp, document type) and optimized chunking (400-600 tokens)
🔍 Optimized RAG Search – Semantic search with configurable similarity threshold (default 0.3) for better recall, with fallback to return top results even if below threshold
🗑️ Document Management – Delete individual documents or bulk delete all documents for a tenant with confirmation dialogs
🛡️ Enterprise Admin Governance – Regex-based red-flag pattern matching with severity levels (low/medium/high/critical) and automatic admin alerts
📊 Comprehensive Analytics & Observability – Full tenant-level analytics logging with SQLite backend:
- Tool usage breakdown (RAG, Web, Admin, LLM) with latency and token tracking
- RAG recall/precision indicators (average hits, scores, top scores)
- Per-tenant query volume and active users
- Red-flag violations with timestamps and confidence scores
- LLM token logs and latency metrics
🌐 Live Web Search – DuckDuckGo-based MCP server with English-biased results
🏢 Multi-Tenant Isolation – Complete tenant isolation with centralized tenant ID management; backend enforces strict isolation for chat, ingestion, and admin ops
🔄 Intelligent Multi-Tool Orchestration – MCP agent orchestrator autonomously selects optimal tool chains (RAG + Web + LLM, etc.) based on query intent and context
⚡ Robust Error Handling – Structured error responses, retry mechanisms, and graceful fallbacks (e.g., if RAG fails → fallback to LLM-only)

Enterprise Features

🔍 Regex-Based Red-Flag Detection – Support for complex regex patterns with keyword fallback and semantic scoring
📈 Real-Time Analytics Dashboard – Per-tenant analytics with configurable time windows (7, 30, 90 days)
🛠️ Admin API Endpoints – /admin/violations, /admin/tools/logs, /admin/tenants for comprehensive governance
🧠 Agent Debug & Planning – /agent/debug and /agent/plan endpoints for observability and tool selection inspection
💾 Persistent Analytics Storage – SQLite-based analytics store with indexes for fast queries

How to Run the Space

Prerequisites

Backend services running:
- FastAPI API (uvicorn backend.api.main:app --port 8000)
- Unified MCP server (port 8900) as described in backend/README.md
- Optional: Ollama / Groq credentials for the LLM client
Quick Start: Run start.bat (Windows) to launch all services automatically.
Python 3.10+ with the dependencies in requirements.txt

Installation

Install dependencies:
```
pip install -r requirements.txt
```
Start the Gradio app:
```
python app.py
```
Access the interface:
- Local: http://localhost:7860
- The app will automatically connect to the backend at http://localhost:8000

Usage

The Gradio UI exposes four tabs once you launch app.py:

Chat – enter your Tenant ID, ask questions, and see multi-tool MCP responses with autonomous tool orchestration.
Document Ingestion – toggle between Raw Text, URL, or File Upload to populate the tenant RAG index. View and manage your ingested documents with delete functionality.
Knowledge Base Library – comprehensive document management interface with:
- Statistics Dashboard: Visual cards showing total documents, document types (Text, PDF, FAQ, Link), and average length
- Interactive Charts: Plotly pie chart displaying document type distribution
- Semantic Search: Search your knowledge base with relevance scoring
- Type Filtering: Filter documents by type (all, text, pdf, faq, link)
- Document Management: View all documents in a table with preview, delete individual documents, or delete all at once
- Auto-refresh: Document lists automatically update after ingestion or deletion
Admin Analytics – comprehensive analytics dashboard with visualizations:
- Statistics Cards: Total queries, active users, red flags, and RAG searches
- Interactive Bar Charts:
  - Tool Usage Count (RAG, Web, Admin tools)
  - Average Tool Latency (performance metrics)
  - RAG Quality Metrics (hits, scores, recall indicators)
- Tool Usage Table: Detailed breakdown of tool performance with counts, latency, success/error rates, and token usage
- Formatted Summary: Key metrics displayed in an easy-to-read format
- Click "🔄 Fetch Analytics Snapshot" to load the latest data
Admin Rules & Compliance – upload/delete governance rules that are stored via the backend /admin/rules API.

Tip: Every action requires a tenant ID. The tenant ID is now managed centrally and persists across page refreshes. The Knowledge Base Library and Admin Analytics tabs feature beautiful, modern UI with dark theme styling and interactive Plotly visualizations.

Frontend (Next.js) Operator Console

The companion Next.js frontend (frontend/) now exposes dedicated pages for each workflow:

URL	Description
`/`	Landing page with hero + quick access panels
`/ingestion`	Data ingestion walkthrough (text/URL/files) with document management
`/chat`	Chat console wrapper around the MCP agent
`/analytics`	Analytics overview and explainer
`/admin-rules`	Admin rule ingestion explainer
`/knowledge-base`	View all ingested documents with search, filter, and delete functionality

Key Features:

Centralized Tenant ID Management – Tenant ID is managed globally via React Context and persists in localStorage
Document Management – View, search, filter, and delete documents from the knowledge base
Improved Error Handling – Clear error messages with retry options for failed operations
Real-time Updates – Document lists automatically refresh after ingestion or deletion

Run the console locally with:

cd frontend
npm install
npm run dev

Then open http://localhost:3000. The navbar links on the landing page route to each section, and you can link directly to those URLs for demo purposes. The tenant ID selector is available in the navbar on all pages.

API Endpoints

Agent Endpoints

Purpose	Method & Path	Description
Chat with agent	`POST /agent/message`	Main chat endpoint with `tenant_id`, `message`, optional history
Agent debug	`POST /agent/debug`	Returns detailed debugging info: reasoning trace, tool selection, intent classification
Agent plan	`POST /agent/plan`	Returns tool selection plan without execution (intent, tool scores, planned steps)

RAG Endpoints

Purpose	Method & Path	Description
Ingest document	`POST /rag/ingest-document`	Accepts `source_type`, `content`, metadata (filename, URL, doc_id). Supports raw text, URLs, PDFs, DOCX, TXT, and Markdown files
Ingest file	`POST /rag/ingest-file`	Multipart upload with `x-tenant-id` header (PDF/DOCX/TXT/MD)
List documents	`GET /rag/list?tenant_id={id}&limit={n}&offset={n}`	Returns all documents for a tenant with pagination. Requires `x-tenant-id` header or `tenant_id` query parameter
Delete document	`DELETE /rag/delete/{document_id}?tenant_id={id}`	Deletes a specific document by ID. Requires `x-tenant-id` header or `tenant_id` query parameter
Delete all documents	`DELETE /rag/delete-all?tenant_id={id}`	Deletes all documents for a tenant. Requires `x-tenant-id` header or `tenant_id` query parameter

Admin & Governance Endpoints

Purpose	Method & Path	Description
List rules	`GET /admin/rules?detailed=true`	Get all rules (use `detailed=true` for regex/severity metadata)
Add rule	`POST /admin/rules`	Add rule with optional `pattern` (regex), `severity` (low/medium/high/critical), `description`
Delete rule	`DELETE /admin/rules/{rule}`	Delete a specific rule
List violations	`GET /admin/violations?days=30&limit=50`	Get red-flag violations with timestamps and confidence scores
Tool logs	`GET /admin/tools/logs?tool_name=rag&days=7`	Get detailed tool usage logs with latency and token counts
Manage tenants	`GET/POST/DELETE /admin/tenants`	Tenant management endpoints (placeholder implementation)

Analytics Endpoints

Purpose	Method & Path	Description
Overview	`GET /analytics/overview?days=30`	Comprehensive analytics: total queries, tool usage, red-flag count, RAG quality
Tool usage	`GET /analytics/tool-usage?days=30`	Detailed tool usage stats: counts, latency, tokens, success/error rates
Red flags	`GET /analytics/redflags?limit=50&days=30`	Recent red-flag violations for tenant
Activity	`GET /analytics/activity?days=30`	Tenant activity summary: queries, active users, last query timestamp
RAG quality	`GET /analytics/rag-quality?days=30`	RAG quality metrics: avg hits, scores, latency (recall/precision indicators)

All calls are proxied through the FastAPI backend running at http://localhost:8000. Ensure those services are online before launching the Space.

Architecture Highlights

Enterprise-Grade Features

Autonomous Multi-Step Planning: The agent uses LLM-powered planning to determine optimal tool sequences, with memory of previous tool outputs in multi-step workflows.
Regex-Based Governance: Admin rules support regex patterns with fallback to keyword matching and semantic similarity scoring for flexible policy enforcement.
Comprehensive Analytics: All tool usage, RAG searches, LLM calls, and red-flag violations are logged to SQLite with indexed queries for fast analytics retrieval.
Enhanced RAG Pipeline: Documents are chunked with optimal size (400-600 tokens) and enriched with metadata (source URL, timestamp, document type) for better retrieval.
Structured Error Handling: All errors are logged with context, and the system gracefully falls back (e.g., if RAG fails → use LLM-only, if web fails → skip web).

Data Storage

SQLite Databases (for demo/development):
- data/admin_rules.db - Admin rules with regex patterns and severity
- data/analytics.db - Analytics events, tool usage, violations, RAG metrics
Production Ready: Can easily swap SQLite for PostgreSQL/Supabase for production deployments.

Testing & Diagnostics

IntegraChat ships with several helper scripts to validate the full stack end-to-end:

python verify_tenant_isolation.py
Runs a comprehensive suite that covers analytics logging, admin rule storage, API reachability, and—most importantly—multi-tenant RAG isolation.
- ✅ Prerequisites: FastAPI backend plus all MCP servers (RAG/Web/Admin) running locally.
- ✅ What it checks:
  1. Direct database writes via the analytics and rules stores
  2. CRUD over the /admin/* and /analytics/* endpoints
  3. RAG ingestion and isolation by issuing queries as multiple tenants and ensuring secrets never leak across IDs
- ✅ Pass criteria: At least 80 % of the sub-tests succeed (the RAG isolation test must pass for overall success).
python check_rag_database.py
Provides a low-level inspection of the RAG datastore. It connects straight to the pgvector/Postgres instance, lists all tenant IDs, prints sample chunks, and runs search_vectors() directly to ensure the SQL WHERE tenant_id = … filter is behaving as expected. Use this script when diagnosing suspected cross-tenant leakage or when seeding demo data.
python test_manual.py
The existing manual test runner remains useful for smoke-testing analytics logging, admin rule CRUD, and API response codes. Run it whenever you adjust schemas or update MCP endpoints.

Tip: All scripts assume the Python virtual environment is active (source venv/bin/activate or .\venv\Scripts\activate) and that .env contains the MCP server URLs/LLM settings noted earlier.

Demo Video

🎥 [Demo Video Placeholder] - Coming soon!

Watch how IntegraChat uses MCP to power autonomous agents with multi-tool selection, RAG retrieval, and enterprise governance.

Social Media

📱 [Social Media Post Placeholder] - Coming soon!

Team Member(s)

Your Name Here - Developer & MCP Enthusiast

License

This project is licensed under the MIT License - see the LICENSE file for details.

Technical Stack

Backend: FastAPI with async/await for high-performance MCP orchestration
Frontend: Gradio interface with Plotly visualizations + Next.js operator console
UI Libraries: Plotly for interactive charts, Gradio for web interface
LLM Integration: Ollama (local) or Groq (cloud) via configurable backend
Vector Store: pgvector (via Supabase) or SQLite embeddings
Analytics: SQLite with indexed queries for fast analytics
MCP Server: Unified MCP server (port 8900) exposing all tools via namespaces
Database: PostgreSQL with pgvector extension for RAG embeddings, SQLite for analytics

Key Technical Features

Tenant Isolation & Normalization

Strict tenant isolation enforced at database level with WHERE tenant_id = ... filters
Automatic tenant ID normalization handles whitespace and formatting differences
Documents can be listed and deleted consistently across different tenant_id formats
All operations validate tenant ownership before execution

RAG Search & Retrieval

Optimized similarity threshold (default 0.3) for better recall of relevant documents
Intelligent fallback returns top result even if below threshold to ensure knowledge base content is accessible
Pattern-based tool selection automatically triggers RAG for admin questions, fact lookups, and internal knowledge queries
Response unwrapping ensures seamless integration between MCP server and orchestrator

MCP Server Architecture

Unified server running on a single port (default 8900) for all namespaced tools
Dual protocol support: Both MCP protocol (POST with JSON) and RESTful HTTP (GET/DELETE)
Response wrapping: Standardized response format with automatic unwrapping in clients
Error handling: Comprehensive error responses with detailed messages for debugging

UI Features

Knowledge Base Library

Visual Statistics: Real-time document counts and type distribution
Interactive Charts: Plotly pie charts for document type visualization
Advanced Search: Semantic search across all ingested documents with relevance scoring
Smart Filtering: Filter by document type (text, PDF, FAQ, link)
Bulk Operations: Delete individual documents or all documents at once
Auto-refresh: Lists automatically update after operations

Admin Analytics Dashboard

Statistics Cards: Key metrics displayed in visually appealing cards with icons
Tool Usage Visualization: Bar charts showing tool invocation counts and performance
Latency Metrics: Visual representation of tool response times
RAG Quality Analysis: Charts displaying search quality metrics (hits, scores, recall)
Detailed Tables: Comprehensive tool usage breakdown with success/error rates
Dark Theme: Modern UI with dark background and white text for better readability
Real-time Updates: Fetch latest analytics data with a single click

Acknowledgments

Built with Model Context Protocol (MCP)
Powered by Gradio for the interface
Visualizations created with Plotly
Backend built with FastAPI
Analytics and governance features inspired by enterprise AI platform requirements

Made with ❤️ for the MCP Hackathon

IntegraChat: Enterprise-Grade MCP Autonomous Agent Platform

⬆ Back to Top