deployment_guide.md

# Hugging Face Spaces Deployment Guide

Complete guide to deploy the IDP API on Hugging Face Spaces.

## Prerequisites

- Hugging Face account
- Trained model weights (classifier and NER)
- Git installed locally

## Step 1: Create a New Space

1. Go to [Hugging Face Spaces](https://huggingface.co/spaces)
2. Click **"Create new Space"**
3. Configure:
   - **Space name**: `idp-api` (or your preferred name)
   - **License**: Apache 2.0 (or your choice)
   - **Select SDK**: Docker
   - **Hardware**: CPU Basic (free) or upgrade to GPU if needed

4. Click **"Create Space"**

## Step 2: Prepare Your Files

Create a `Dockerfile` in your project root:

```dockerfile
FROM python:3.10-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
    poppler-utils \
    libgomp1 \
    libglib2.0-0 \
    libsm6 \
    libxext6 \
    libxrender-dev \
    libgl1-mesa-glx \
    && rm -rf /var/lib/apt/lists/*

# Set working directory
WORKDIR /app

# Copy requirements
COPY requirements.txt .

# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY *.py ./
COPY models/ ./models/

# Expose port 7860 (HF Spaces default)
EXPOSE 7860

# Run the API server
CMD ["uvicorn", "api_server:app", "--host", "0.0.0.0", "--port", "7860"]
```

## Step 3: Train Your Models

Before deployment, train your models locally:

```bash
# Train classifier
python train_classifier.py

# Train NER model
python train_ner.py
```

This will create:
- `models/classifier/best_classifier.pt`
- `models/ner/best_ner.pt`

## Step 4: (Optional) Optimize Models for Faster Inference

Convert to ONNX and quantize:

```bash
# Convert classifier to ONNX
python model_optimizer.py convert_classifier \
    models/classifier/best_classifier.pt \
    models/classifier/classifier.onnx

# Quantize classifier
python model_optimizer.py quantize \
    models/classifier/classifier.onnx \
    models/classifier/classifier_quantized.onnx

# Convert NER to ONNX
python model_optimizer.py convert_ner \
    models/ner/best_ner.pt \
    models/ner/ner.onnx

# Quantize NER
python model_optimizer.py quantize \
    models/ner/ner.onnx \
    models/ner/ner_quantized.onnx
```

If using ONNX models, update `api_server.py` to use ONNX inference sessions.

## Step 5: Push to Hugging Face Space

Clone your Space repository:

```bash
git clone https://huggingface.co/spaces/YOUR_USERNAME/idp-api
cd idp-api
```

Copy your files:

```bash
# Copy Python files
cp /path/to/your/project/*.py .

# Copy models
cp -r /path/to/your/project/models .

# Copy config files
cp /path/to/your/project/requirements.txt .
cp /path/to/your/project/Dockerfile .
```

Create a `README.md` for your Space:

```markdown
---
title: IDP API
emoji: 📄
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
---

# Intelligent Document Processing API

API for extracting structured data from invoices, receipts, and forms.

## Features

- Lightweight OCR with PaddleOCR
- Document classification (Invoice, Receipt, Form)
- Entity extraction (dates, amounts, names, etc.)
- Confidence scoring
- CORS-enabled for web integration

## API Endpoints

### POST /process

Upload a document (PDF or image) and get structured data.

### GET /health

Health check endpoint.

See full documentation at [YOUR_REPO_URL]
```

Commit and push:

```bash
git add .
git commit -m "Initial deployment"
git push
```

## Step 6: Monitor Deployment

1. Go to your Space URL: `https://huggingface.co/spaces/YOUR_USERNAME/idp-api`
2. Watch the build logs in the "Logs" tab
3. Wait for the build to complete (may take 5-10 minutes for first build)
4. Once running, the Space will show "Running" status

## Step 7: Test Your API

Test the health endpoint:

```bash
curl https://YOUR_USERNAME-idp-api.hf.space/health
```

Test document processing:

```bash
curl -X POST \
  https://YOUR_USERNAME-idp-api.hf.space/process \
  -F "file=@sample_invoice.pdf"
```

## Step 8: Configure for Production

### Upgrade Hardware (Optional)

If CPU performance is insufficient:

1. Go to Space settings
2. Under "Hardware", upgrade to:
   - **CPU Upgrade**: 2 vCPUs, 16GB RAM
   - **GPU** T4 Small**: 15GB VRAM (if using GPU)

### Set Secrets (Optional)

If you need API keys or secrets:

1. Go to Space settings
2. Add secrets under "Repository secrets"
3. Access in code via `os.environ.get('SECRET_NAME')`

### Enable Persistent Storage (Optional)

For caching or logging:

1. Go to Space settings
2. Enable "Persistent Storage"
3. Data will persist in `/data` directory

## Troubleshooting

### Build Fails

**Issue**: Docker build fails with package errors

**Solution**:
- Check `requirements.txt` for version conflicts
- Review build logs for specific errors
- Ensure Dockerfile has all system dependencies

### Out of Memory

**Issue**: API crashes with OOM errors

**Solution**:
- Upgrade to higher memory tier
- Reduce batch size
- Use ONNX quantized models
- Disable GPU if not needed

### Slow Inference

**Issue**: Processing takes >5 seconds per document

**Solution**:
- Use ONNX models with INT8 quantization
- Upgrade to GPU hardware
- Reduce OCR DPI (lower quality but faster)
- Cache models in memory (done by default)

## Cost Optimization

**Free Tier**:
- CPU Basic: Free forever
- Suitable for demos and low-traffic apps
- May sleep after inactivity

**Paid Tier** (if needed):
- CPU Upgrade: ~$0.50/hour
- GPU T4: ~$0.60/hour
- Only charged when running

**Tips**:
- Use CPU for development
- Upgrade to GPU only for production with high traffic
- Use ONNX + quantization to maximize CPU performance

## Next Steps

- Set up monitoring with [Hugging Face Analytics](https://huggingface.co/docs/hub/spaces-analytics)
- Add authentication if needed
- Create a custom domain (paid feature)
- Integrate with your Next.js frontend (see `nextjs_integration_guide.md`)

## Support

For issues:
- Check [HF Spaces docs](https://huggingface.co/docs/hub/spaces)
- Ask in [HF Forums](https://discuss.huggingface.co/)
- Review API logs in Space "Logs" tab