MissionControlMCP / ARCHITECTURE.md
AlBaraa63's picture
Upload 44 files
32dc112 verified

A newer version of the Gradio SDK is available: 6.2.0

Upgrade

๐Ÿ—๏ธ System Architecture

MissionControlMCP system design and architecture documentation.


๐Ÿ“Š High-Level Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                        Client Layer                          โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”‚
โ”‚  โ”‚ Claude       โ”‚  โ”‚  Custom      โ”‚  โ”‚  Other MCP   โ”‚      โ”‚
โ”‚  โ”‚ Desktop      โ”‚  โ”‚  Client      โ”‚  โ”‚  Clients     โ”‚      โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚ MCP Protocol (stdio)
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    MCP Server Layer                          โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
โ”‚  โ”‚              mcp_server.py                             โ”‚ โ”‚
โ”‚  โ”‚  โ€ข Tool Registration                                   โ”‚ โ”‚
โ”‚  โ”‚  โ€ข Request Routing                                     โ”‚ โ”‚
โ”‚  โ”‚  โ€ข Response Formatting                                 โ”‚ โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    Business Logic Layer                      โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”            โ”‚
โ”‚  โ”‚ PDF      โ”‚ Text     โ”‚ Web      โ”‚ RAG      โ”‚            โ”‚
โ”‚  โ”‚ Reader   โ”‚ Extract  โ”‚ Fetcher  โ”‚ Search   โ”‚            โ”‚
โ”‚  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค            โ”‚
โ”‚  โ”‚ Data     โ”‚ File     โ”‚ Email    โ”‚ KPI      โ”‚            โ”‚
โ”‚  โ”‚ Visual   โ”‚ Convert  โ”‚ Classify โ”‚ Generate โ”‚            โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜            โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    Utility Layer                             โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
โ”‚  โ”‚  โ€ข helpers.py      - Text processing utilities         โ”‚ โ”‚
โ”‚  โ”‚  โ€ข rag_utils.py    - Vector search & FAISS             โ”‚ โ”‚
โ”‚  โ”‚  โ€ข schemas.py      - Pydantic models                   โ”‚ โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

๐Ÿงฉ Component Architecture

1. MCP Server (mcp_server.py)

Responsibilities:

  • Register all 8 tools with MCP SDK
  • Handle incoming tool requests
  • Route requests to appropriate tool functions
  • Format and return responses
  • Error handling and logging

Flow:

Client Request โ†’ MCP Protocol โ†’ Server โ†’ Tool โ†’ Response โ†’ Client

Code Structure:

# Tool Registration
server.register_tool(name, description, input_schema)

# Request Handler
async def call_tool(name, arguments):
    if name == "pdf_reader":
        return await pdf_reader.read_pdf(**arguments)
    elif name == "text_extractor":
        return await text_extractor.extract_text(**arguments)
    # ... other tools

# Server Startup
async with stdio_server() as (read_stream, write_stream):
    await server.run(read_stream, write_stream)

2. Tool Layer (tools/)

Each tool is independent and follows this pattern:

Tool Structure:

"""
Tool Name - Description
"""
import logging
from typing import Dict, Any

logger = logging.getLogger(__name__)

def tool_function(param: str) -> Dict[str, Any]:
    """
    Tool description.
    
    Args:
        param: Parameter description
        
    Returns:
        Standardized result dictionary
    """
    try:
        # Validation
        if not param:
            raise ValueError("Invalid input")
        
        # Processing
        result = process_data(param)
        
        # Return standardized format
        return {
            "success": True,
            "data": result,
            "metadata": {}
        }
        
    except Exception as e:
        logger.error(f"Error: {e}")
        raise

Tool Independence:

  • Each tool is self-contained
  • No dependencies between tools
  • Can be tested individually
  • Easy to add/remove tools

3. Utility Layer (utils/)

helpers.py - Text Processing:

โ€ข clean_text() - Remove extra whitespace
โ€ข extract_keywords() - NLP keyword extraction
โ€ข chunk_text() - Text splitting with overlap
โ€ข validate_url() - URL validation

rag_utils.py - Vector Search:

โ€ข SimpleRAGStore - FAISS-based vector database
โ€ข semantic_search() - Sentence transformer embeddings
โ€ข create_rag_store() - Initialize vector store

Models (models/schemas.py):

โ€ข Pydantic models for type validation
โ€ข Input/output schemas
โ€ข Data validation

๐Ÿ”„ Data Flow

Request Flow

1. Client sends MCP request
   โ†“
2. mcp_server.py receives request
   โ†“
3. Server validates input schema
   โ†“
4. Server routes to tool function
   โ†“
5. Tool processes data
   โ†“
6. Tool returns result dict
   โ†“
7. Server formats MCP response
   โ†“
8. Client receives response

Example: PDF Reading Flow

Client: "Read this PDF"
   โ†“
MCP Server: Receives pdf_reader request
   โ†“
pdf_reader.py: read_pdf(file_path)
   โ†“
PyPDF2: Extract text from pages
   โ†“
Return: {text, pages, metadata}
   โ†“
MCP Server: Format response
   โ†“
Client: Receives extracted text

๐Ÿ—‚๏ธ Project Structure

mission_control_mcp/
โ”‚
โ”œโ”€โ”€ mcp_server.py              # MCP server entry point
โ”‚
โ”œโ”€โ”€ tools/                     # 8 independent tools
โ”‚   โ”œโ”€โ”€ pdf_reader.py          # PDF text extraction
โ”‚   โ”œโ”€โ”€ text_extractor.py      # Text processing (4 ops)
โ”‚   โ”œโ”€โ”€ web_fetcher.py         # Web scraping
โ”‚   โ”œโ”€โ”€ rag_search.py          # Semantic search
โ”‚   โ”œโ”€โ”€ data_visualizer.py     # Chart generation
โ”‚   โ”œโ”€โ”€ file_converter.py      # File format conversion
โ”‚   โ”œโ”€โ”€ email_intent_classifier.py  # Email classification
โ”‚   โ””โ”€โ”€ kpi_generator.py       # Business metrics
โ”‚
โ”œโ”€โ”€ utils/                     # Shared utilities
โ”‚   โ”œโ”€โ”€ helpers.py             # Text processing helpers
โ”‚   โ””โ”€โ”€ rag_utils.py           # Vector search utilities
โ”‚
โ”œโ”€โ”€ models/                    # Data models
โ”‚   โ””โ”€โ”€ schemas.py             # Pydantic schemas
โ”‚
โ”œโ”€โ”€ examples/                  # Sample test data
โ”‚   โ”œโ”€โ”€ sample_report.txt      # Business report
โ”‚   โ”œโ”€โ”€ business_data.csv      # Financial data
โ”‚   โ”œโ”€โ”€ sample_email_*.txt     # Email samples
โ”‚   โ””โ”€โ”€ sample_documents.txt   # RAG search docs
โ”‚
โ”œโ”€โ”€ app.py                     # Gradio web interface
โ”œโ”€โ”€ demo.py                    # Demo & test suite
โ”‚
โ”œโ”€โ”€ docs/                      # Documentation
โ”‚   โ”œโ”€โ”€ README.md              # Main documentation
โ”‚   โ”œโ”€โ”€ API.md                 # API reference
โ”‚   โ”œโ”€โ”€ EXAMPLES.md            # Use cases
โ”‚   โ”œโ”€โ”€ TESTING.md             # Testing guide
โ”‚   โ”œโ”€โ”€ ARCHITECTURE.md        # This file
โ”‚   โ””โ”€โ”€ CONTRIBUTING.md        # Contribution guide
โ”‚
โ”œโ”€โ”€ requirements.txt           # Python dependencies
โ”œโ”€โ”€ .gitignore                 # Git ignore rules
โ””โ”€โ”€ LICENSE                    # MIT License

๐Ÿ”Œ Integration Points

MCP Protocol Integration

from mcp.server import Server
from mcp.types import Tool, TextContent

# Create server
server = Server("mission-control")

# Register tool
@server.tool()
async def pdf_reader(file_path: str) -> str:
    result = read_pdf(file_path)
    return json.dumps(result)

# Run server
await server.run(stdin, stdout)

Claude Desktop Integration

Configuration:

{
  "mcpServers": {
    "mission-control": {
      "command": "python",
      "args": ["path/to/mcp_server.py"]
    }
  }
}

Communication:

Claude Desktop โ†โ†’ MCP Protocol โ†โ†’ mcp_server.py โ†โ†’ Tools

๐Ÿš€ Scalability Design

Horizontal Scaling

Current: Single-process server Future: Multi-process with load balancing

             Load Balancer
                   โ”‚
        โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
        โ”‚          โ”‚          โ”‚
   Server 1    Server 2    Server 3
        โ”‚          โ”‚          โ”‚
        โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                Tools

Caching Strategy

Implemented:

  • RAG model caching (sentence transformers)
  • NLTK data caching

Future Improvements:

  • Redis for result caching
  • Database for document storage
  • CDN for static assets

๐Ÿ”’ Security Architecture

Input Validation

# Pydantic schemas
from pydantic import BaseModel, validator

class PDFReaderInput(BaseModel):
    file_path: str
    
    @validator('file_path')
    def validate_path(cls, v):
        if not Path(v).exists():
            raise ValueError("File not found")
        return v

Error Handling

try:
    result = tool_function(input)
except FileNotFoundError:
    return {"error": "File not found", "code": 404}
except ValueError:
    return {"error": "Invalid input", "code": 400}
except Exception:
    return {"error": "Internal error", "code": 500}

Authentication

Current: None (local tool execution) Production Considerations:

  • API key authentication
  • Rate limiting
  • Request logging
  • User permissions

๐Ÿ“Š Performance Characteristics

Tool Performance

Tool Avg Time Memory Notes
PDF Reader 1s 50MB Depends on PDF size
Text Extractor 0.5s 10MB Fast text processing
Web Fetcher 2-3s 20MB Network dependent
RAG Search 2.5s* 200MB *First run (model load)
RAG Search 0.5s 200MB Subsequent runs
Data Visualizer 1.2s 30MB Chart generation
File Converter 1-2s 50MB File size dependent
Email Classifier 0.1s 5MB Very fast
KPI Generator 0.3s 10MB Quick calculations

Bottlenecks

  1. RAG Search - Initial model loading (~2s)

    • Solution: Keep model in memory
  2. Web Fetcher - Network latency

    • Solution: Async requests, caching
  3. PDF Reader - Large files

    • Solution: Stream processing

๐Ÿ”„ State Management

Stateless Design

Each tool request is independent:

  • No session state
  • No user context
  • Pure function design

Benefits:

  • Easy scaling
  • No state synchronization
  • Simple debugging
  • High availability

RAG Store State

Exception: RAG search maintains in-memory vector store:

class SimpleRAGStore:
    def __init__(self):
        self.documents = []
        self.index = None  # FAISS index

Lifecycle:

  • Created on first search
  • Persists during server lifetime
  • Cleared on server restart

๐Ÿงช Testing Architecture

Test Pyramid

         โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
         โ”‚   E2E Tests โ”‚  (MCP integration)
         โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
         โ”‚ Integration โ”‚  (Tool combinations)
         โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
         โ”‚  Unit Tests โ”‚  (Individual functions)
         โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Test Coverage

  • Unit Tests: Test each function independently
  • Integration Tests: Test tool interactions
  • MCP Tests: Test server communication
  • Sample Tests: Test with real data

๐Ÿ“ฆ Dependency Management

Core Dependencies

MCP SDK (>=1.0.0)
โ”œโ”€โ”€ stdio communication
โ””โ”€โ”€ Tool registration

Processing Libraries
โ”œโ”€โ”€ PyPDF2 (PDF reading)
โ”œโ”€โ”€ BeautifulSoup4 (HTML parsing)
โ”œโ”€โ”€ Pandas (Data processing)
โ””โ”€โ”€ Matplotlib (Visualization)

ML/NLP Libraries
โ”œโ”€โ”€ scikit-learn (Text processing)
โ”œโ”€โ”€ NLTK (Keyword extraction)
โ”œโ”€โ”€ sentence-transformers (Embeddings)
โ””โ”€โ”€ FAISS (Vector search)

Optional Dependencies

  • faiss-cpu: Can use faiss-gpu on GPU systems
  • reportlab: Optional for PDF generation

๐Ÿ”ฎ Future Architecture Improvements

Planned Enhancements

  1. Database Integration

    PostgreSQL for persistent storage
    Redis for caching
    
  2. Async Processing

    async def process_pdf(file_path: str):
        # Async PDF processing
        return await asyncio.to_thread(read_pdf, file_path)
    
  3. Microservices

    Each tool as separate service
    API gateway for routing
    Service mesh for communication
    
  4. Monitoring

    Prometheus metrics
    Grafana dashboards
    Error tracking (Sentry)
    

๐Ÿ“ Design Principles

SOLID Principles

  • Single Responsibility: Each tool does one thing
  • Open/Closed: Easy to add new tools
  • Liskov Substitution: Tools are interchangeable
  • Interface Segregation: Minimal tool interfaces
  • Dependency Inversion: Tools depend on abstractions

Clean Architecture

  • Independent of Frameworks: Core logic separate from MCP
  • Testable: Can test without MCP server
  • Independent of UI: Works with any MCP client
  • Independent of Database: No database coupling

๐ŸŽฏ Architectural Goals

โœ… Achieved:

  • Modular design
  • Easy to extend
  • Well-documented
  • Testable
  • Production-ready

๐Ÿ”„ In Progress:

  • Performance optimization
  • Enhanced caching
  • Better error handling

๐ŸŽฏ Future:

  • Multi-tenancy
  • Distributed processing
  • Advanced monitoring
  • Auto-scaling

MissionControlMCP Architecture Documentation v1.0 ๐Ÿ—๏ธ