SETUP.md

Lightweight LLM with MCP Integration

This directory contains a lightweight LLM inference service that can connect to external MCP (Model Context Protocol) servers for enhanced capabilities.

🏗️ Quick Setup

Local Development

# Install dependencies
pip install -r requirements.txt

# Run the service
python app.py

# Or use the start script
./start.sh

Docker Deployment

# Build the image
docker build -t lightweight-llm .

# Run the container
docker run -p 8000:8000 \
  -e MCP_SERVER_URL=https://your-mcp-server.hf.space \
  lightweight-llm

Hugging Face Spaces

1. Create a new Space with Docker SDK
2. Upload all files from this directory
3. Set app_port: 8000 in README.md header
4. Configure environment variables in Space settings

📡 API Usage

Basic Inference

curl -X POST http://localhost:8000/ \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": "Hello, how are you?",
    "parameters": {"max_new_tokens": 100}
  }'

MCP-Enhanced Inference

curl -X POST http://localhost:8000/ \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": "Tell me about product 123",
    "parameters": {
      "max_new_tokens": 200,
      "mcp_server_url": "https://your-mcp-server.hf.space"
    }
  }'

🔧 Configuration

Set these environment variables:

• MCP_SERVER_URL: Default MCP server URL
• MODEL_NAME: Hugging Face model name
• MAX_NEW_TOKENS: Default max tokens
• PORT: Service port (default: 8000)

🧪 Testing

Run the test suite:

python test_service.py

🚀 Production Notes

• The current implementation uses a simple text generation placeholder
• For production, integrate with:
  • Transformers library with a small model
  • vLLM for faster inference
  • TensorRT for optimized inference
  • Any other inference engine

🔗 MCP Integration

This service automatically detects when to use MCP tools based on keywords in the input. It can connect to any MCP server that implements the standard protocol.

Keywords that trigger MCP usage:

• product, price, stock, inventory
• order, customer, database
• search, find, get, fetch, check
• prestashop, shop, cart, purchase

📊 Endpoints

• POST / - Main inference endpoint (HF compatible)
• GET /health - Health check
• GET /info - Service information
• GET /docs - API documentation