SETUP.md

# Lightweight LLM with MCP Integration

This directory contains a lightweight LLM inference service that can connect to external MCP (Model Context Protocol) servers for enhanced capabilities.

## 🏗️ Quick Setup

### Local Development
```bash
# Install dependencies
pip install -r requirements.txt

# Run the service
python app.py

# Or use the start script
./start.sh
```

### Docker Deployment
```bash
# Build the image
docker build -t lightweight-llm .

# Run the container
docker run -p 8000:8000 \
  -e MCP_SERVER_URL=https://your-mcp-server.hf.space \
  lightweight-llm
```

### Hugging Face Spaces
1. Create a new Space with Docker SDK
2. Upload all files from this directory
3. Set `app_port: 8000` in README.md header
4. Configure environment variables in Space settings

## 📡 API Usage

### Basic Inference
```bash
curl -X POST http://localhost:8000/ \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": "Hello, how are you?",
    "parameters": {"max_new_tokens": 100}
  }'
```

### MCP-Enhanced Inference
```bash
curl -X POST http://localhost:8000/ \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": "Tell me about product 123",
    "parameters": {
      "max_new_tokens": 200,
      "mcp_server_url": "https://your-mcp-server.hf.space"
    }
  }'
```

## 🔧 Configuration

Set these environment variables:
- `MCP_SERVER_URL`: Default MCP server URL
- `MODEL_NAME`: Hugging Face model name
- `MAX_NEW_TOKENS`: Default max tokens
- `PORT`: Service port (default: 8000)

## 🧪 Testing

Run the test suite:
```bash
python test_service.py
```

## 🚀 Production Notes

- The current implementation uses a simple text generation placeholder
- For production, integrate with:
  - Transformers library with a small model
  - vLLM for faster inference
  - TensorRT for optimized inference
  - Any other inference engine

## 🔗 MCP Integration

This service automatically detects when to use MCP tools based on keywords in the input. It can connect to any MCP server that implements the standard protocol.

**Keywords that trigger MCP usage:**
- product, price, stock, inventory
- order, customer, database
- search, find, get, fetch, check
- prestashop, shop, cart, purchase

## 📊 Endpoints

- `POST /` - Main inference endpoint (HF compatible)
- `GET /health` - Health check
- `GET /info` - Service information
- `GET /docs` - API documentation