| # 🚀 Complete FastAPI Deployment Package |
|
|
| ## 📦 What You've Got |
|
|
| A production-ready FastAPI application for PDF redaction with Named Entity Recognition, ready to deploy on HuggingFace Spaces or any cloud platform. |
|
|
| --- |
|
|
| ## 📁 Directory Structure |
|
|
| ``` |
| pdf-redaction-api/ |
| │ |
| ├── 📄 main.py # FastAPI application |
| ├── 🐳 Dockerfile # Production container |
| ├── 🐳 docker-compose.yml # Local development |
| ├── 📋 requirements.txt # Python dependencies |
| │ |
| ├── 📱 app/ |
| │ ├── __init__.py |
| │ └── redaction.py # Core redaction engine |
| │ |
| ├── 📂 uploads/ # Temporary uploads |
| │ └── .gitkeep |
| │ |
| ├── 📂 outputs/ # Redacted PDFs |
| │ └── .gitkeep |
| │ |
| ├── 🧪 tests/ |
| │ └── test_api.py # API tests |
| │ |
| ├── 📚 Documentation/ |
| │ ├── README.md # Main docs (for HF Spaces) |
| │ ├── DEPLOYMENT.md # Deployment guide |
| │ ├── QUICKSTART.md # Quick start guide |
| │ └── STRUCTURE.md # Project structure |
| │ |
| ├── 🔧 Configuration/ |
| │ ├── .env.example # Environment variables |
| │ ├── .gitignore # Git ignore |
| │ └── .dockerignore # Docker ignore |
| │ |
| ├── 🤖 .github/ |
| │ └── workflows/ |
| │ └── ci-cd.yml # GitHub Actions CI/CD |
| │ |
| ├── 📝 client_example.py # Example API client |
| └── 📜 LICENSE # MIT License |
| ``` |
|
|
| --- |
|
|
| ## ✨ Features |
|
|
| ### Core Functionality |
| ✅ PDF upload and processing |
| ✅ OCR with pytesseract (configurable DPI) |
| ✅ Named Entity Recognition (NER) |
| ✅ Accurate coordinate-based redaction |
| ✅ Multiple entity type support |
| ✅ Downloadable redacted PDFs |
|
|
| ### API Features |
| ✅ RESTful API with FastAPI |
| ✅ Automatic OpenAPI documentation |
| ✅ File upload handling |
| ✅ Background task cleanup |
| ✅ Health checks |
| ✅ Statistics endpoint |
| ✅ CORS support |
|
|
| ### DevOps |
| ✅ Docker containerization |
| ✅ Docker Compose for local dev |
| ✅ GitHub Actions CI/CD |
| ✅ HuggingFace Spaces ready |
| ✅ Comprehensive testing |
| ✅ Logging and monitoring |
|
|
| --- |
|
|
| ## 🎯 Quick Deployment Paths |
|
|
| ### Option 1: HuggingFace Spaces (Recommended for Demo) |
|
|
| **Time: 10 minutes** |
|
|
| ```bash |
| # 1. Create Space on HuggingFace (select Docker SDK) |
| # 2. Clone your space |
| git clone https://huggingface.co/spaces/YOUR_USERNAME/pdf-redaction-api |
| cd pdf-redaction-api |
| |
| # 3. Copy all files |
| cp -r /path/to/pdf-redaction-api/* . |
| |
| # 4. Deploy |
| git add . |
| git commit -m "Initial deployment" |
| git push |
| ``` |
|
|
| **Your API will be at:** `https://YOUR_USERNAME-pdf-redaction-api.hf.space` |
|
|
| **Cost:** FREE (with CPU Basic tier) |
|
|
| --- |
|
|
| ### Option 2: Docker Locally |
|
|
| **Time: 5 minutes** |
|
|
| ```bash |
| # Build |
| docker build -t pdf-redaction-api . |
| |
| # Run |
| docker run -p 7860:7860 pdf-redaction-api |
| |
| # Test |
| curl http://localhost:7860/health |
| ``` |
|
|
| --- |
|
|
| ### Option 3: Direct Python |
|
|
| **Time: 3 minutes** |
|
|
| ```bash |
| # Install dependencies |
| sudo apt-get install tesseract-ocr poppler-utils |
| pip install -r requirements.txt |
| |
| # Run |
| python main.py |
| |
| # Access at http://localhost:7860 |
| ``` |
|
|
| --- |
|
|
| ## 🔌 API Endpoints |
|
|
| ### Core Endpoints |
|
|
| | Method | Endpoint | Description | |
| |--------|----------|-------------| |
| | POST | `/redact` | Upload and redact PDF | |
| | GET | `/download/{job_id}` | Download redacted PDF | |
| | GET | `/health` | Health check | |
| | GET | `/stats` | API statistics | |
| | DELETE | `/cleanup/{job_id}` | Manual cleanup | |
| | GET | `/docs` | Interactive API docs | |
|
|
| ### Example Usage |
|
|
| **cURL:** |
| ```bash |
| curl -X POST "http://localhost:7860/redact" \ |
| -F "file=@document.pdf" \ |
| -F "dpi=300" |
| ``` |
|
|
| **Python:** |
| ```python |
| import requests |
| |
| response = requests.post( |
| "http://localhost:7860/redact", |
| files={"file": open("document.pdf", "rb")}, |
| params={"dpi": 300} |
| ) |
| |
| job_id = response.json()["job_id"] |
| redacted = requests.get(f"http://localhost:7860/download/{job_id}") |
| ``` |
|
|
| --- |
|
|
| ## 🎨 Architecture |
|
|
| ``` |
| ┌─────────────────────────────────────────────────────────┐ |
| │ CLIENT REQUEST │ |
| │ (Upload PDF via POST /redact) │ |
| └─────────────────────────────────────────────────────────┘ |
| ↓ |
| ┌─────────────────────────────────────────────────────────┐ |
| │ FASTAPI (main.py) │ |
| │ • Validate file │ |
| │ • Generate job_id │ |
| │ • Save to uploads/ │ |
| └─────────────────────────────────────────────────────────┘ |
| ↓ |
| ┌─────────────────────────────────────────────────────────┐ |
| │ PDFRedactor (app/redaction.py) │ |
| │ │ |
| │ ┌─────────────────────────────────────────┐ │ |
| │ │ 1. OCR (pytesseract) │ │ |
| │ │ • Convert PDF → Images (pdf2image) │ │ |
| │ │ • Extract text + bounding boxes │ │ |
| │ │ • Store image dimensions │ │ |
| │ └─────────────────────────────────────────┘ │ |
| │ ↓ │ |
| │ ┌─────────────────────────────────────────┐ │ |
| │ │ 2. NER (HuggingFace Transformers) │ │ |
| │ │ • Load model │ │ |
| │ │ • Identify entities in text │ │ |
| │ │ • Return entity types + positions │ │ |
| │ └─────────────────────────────────────────┘ │ |
| │ ↓ │ |
| │ ┌─────────────────────────────────────────┐ │ |
| │ │ 3. Mapping │ │ |
| │ │ • Create character span index │ │ |
| │ │ • Match NER entities to OCR boxes │ │ |
| │ └─────────────────────────────────────────┘ │ |
| │ ↓ │ |
| │ ┌─────────────────────────────────────────┐ │ |
| │ │ 4. Redaction (pypdf) │ │ |
| │ │ • Scale image coords → PDF coords │ │ |
| │ │ • Create black rectangle annotations │ │ |
| │ │ • Write redacted PDF │ │ |
| │ └─────────────────────────────────────────┘ │ |
| └─────────────────────────────────────────────────────────┘ |
| ↓ |
| ┌─────────────────────────────────────────────────────────┐ |
| │ RESPONSE │ |
| │ • job_id │ |
| │ • List of entities │ |
| │ • Download URL │ |
| └─────────────────────────────────────────────────────────┘ |
| ``` |
|
|
| --- |
|
|
| ## 🔐 Security Considerations |
|
|
| ### Current Implementation |
| - ✅ File validation (PDF only) |
| - ✅ Temporary file cleanup |
| - ✅ CORS middleware |
| - ✅ Error handling |
|
|
| ### For Production (TODO) |
| - ⚠️ Add API key authentication |
| - ⚠️ Implement rate limiting |
| - ⚠️ Add file size limits |
| - ⚠️ Use HTTPS only |
| - ⚠️ Implement user quotas |
| - ⚠️ Add input sanitization |
|
|
| **Example API Key Auth:** |
| ```python |
| # Add to main.py |
| from fastapi import Security, HTTPException |
| from fastapi.security import APIKeyHeader |
| |
| API_KEY = "your-secret-key" |
| api_key_header = APIKeyHeader(name="X-API-Key") |
| |
| def verify_api_key(key: str = Security(api_key_header)): |
| if key != API_KEY: |
| raise HTTPException(401, "Invalid API Key") |
| ``` |
|
|
| --- |
|
|
| ## 📊 Performance Tuning |
|
|
| ### DPI Settings |
|
|
| | DPI | Quality | Speed | Use Case | |
| |-----|---------|-------|----------| |
| | 150 | Low | Fast | Quick previews | |
| | 200 | Medium | Medium | General use | |
| | 300 | High | Slow | **Recommended** | |
| | 600 | Very High | Very Slow | Critical documents | |
|
|
| ### Hardware Requirements |
|
|
| **Minimum (Free Tier):** |
| - CPU: 2 cores |
| - RAM: 2GB |
| - Storage: 1GB |
|
|
| **Recommended (Production):** |
| - CPU: 4+ cores |
| - RAM: 8GB |
| - Storage: 10GB |
| - GPU: Optional (speeds up NER) |
|
|
| --- |
|
|
| ## 🧪 Testing |
|
|
| ```bash |
| # Install test dependencies |
| pip install pytest pytest-cov httpx |
| |
| # Run tests |
| pytest tests/ -v |
| |
| # With coverage |
| pytest tests/ --cov=app --cov-report=html |
| |
| # View coverage report |
| open htmlcov/index.html |
| ``` |
|
|
| --- |
|
|
| ## 📈 Monitoring |
|
|
| ### Built-in Endpoints |
|
|
| **Health Check:** |
| ```bash |
| curl http://localhost:7860/health |
| ``` |
|
|
| **Statistics:** |
| ```bash |
| curl http://localhost:7860/stats |
| ``` |
|
|
| ### Logs |
|
|
| **Development:** |
| ```bash |
| python main.py |
| # Logs appear in console |
| ``` |
|
|
| **Docker:** |
| ```bash |
| docker logs -f container_name |
| ``` |
|
|
| **HuggingFace Spaces:** |
| - View in Space dashboard → Logs tab |
|
|
| --- |
|
|
| ## 💰 Cost Estimation |
|
|
| ### HuggingFace Spaces |
|
|
| | Tier | CPU | RAM | Price | Use Case | |
| |------|-----|-----|-------|----------| |
| | Basic | 2 | 16GB | **FREE** | Demo, testing | |
| | CPU Upgrade | 4 | 32GB | $0.50/hr | Production | |
| | GPU T4 | - | - | $0.60/hr | Heavy load | |
| | GPU A10G | - | - | $1.50/hr | Enterprise | |
|
|
| **Monthly Costs (if always on):** |
| - Free: $0 |
| - CPU Upgrade: ~$360/month |
| - GPU T4: ~$432/month |
|
|
| **Recommendation:** Start free, upgrade based on usage |
|
|
| ### Alternatives |
|
|
| **AWS ECS Fargate:** ~$30-100/month |
| **Google Cloud Run:** Pay per request (~$10-50/month) |
| **DigitalOcean App:** $12-24/month |
| **Self-hosted VPS:** $5-20/month |
|
|
| --- |
|
|
| ## 🔄 CI/CD Pipeline |
|
|
| ### Automated with GitHub Actions |
|
|
| ``` |
| Push to GitHub |
| ↓ |
| [Run Tests] |
| ↓ |
| [Build Docker] |
| ↓ |
| [Test Container] |
| ↓ |
| [Deploy to HuggingFace] |
| ``` |
|
|
| **Setup:** |
| 1. Add secrets in GitHub repo settings: |
| - `HF_TOKEN`: HuggingFace access token |
| - `HF_SPACE`: Your space name (username/space-name) |
|
|
| 2. Push to main branch → Auto-deploy! ✨ |
|
|
| --- |
|
|
| ## 📚 Documentation Access |
|
|
| | Document | Purpose | |
| |----------|---------| |
| | `README.md` | Overview, API docs, usage examples | |
| | `QUICKSTART.md` | 5-minute setup guide | |
| | `DEPLOYMENT.md` | Production deployment | |
| | `STRUCTURE.md` | Code organization | |
| | `/docs` endpoint | Interactive API documentation | |
|
|
| --- |
|
|
| ## 🎓 Learning Resources |
|
|
| ### FastAPI |
| - Docs: https://fastapi.tiangolo.com |
| - Tutorial: https://fastapi.tiangolo.com/tutorial |
|
|
| ### HuggingFace |
| - Spaces: https://huggingface.co/docs/hub/spaces |
| - Transformers: https://huggingface.co/docs/transformers |
|
|
| ### Docker |
| - Getting Started: https://docs.docker.com/get-started |
|
|
| --- |
|
|
| ## 🐛 Troubleshooting |
|
|
| ### Common Issues |
|
|
| **Problem:** "Tesseract not found" |
| **Solution:** `apt-get install tesseract-ocr` |
|
|
| **Problem:** "Poppler not found" |
| **Solution:** `apt-get install poppler-utils` |
|
|
| **Problem:** Slow processing |
| **Solution:** Lower DPI to 150-200 |
|
|
| **Problem:** Out of memory |
| **Solution:** Upgrade hardware or reduce DPI |
|
|
| **Problem:** Model not loading |
| **Solution:** Check internet, wait for download |
|
|
| ### Debug Mode |
|
|
| ```python |
| # In main.py, add debug mode |
| if __name__ == "__main__": |
| uvicorn.run("main:app", host="0.0.0.0", port=7860, reload=True, log_level="debug") |
| ``` |
|
|
| --- |
|
|
| ## ✅ Checklist for Production |
|
|
| - [ ] Test all endpoints thoroughly |
| - [ ] Add API key authentication |
| - [ ] Implement rate limiting |
| - [ ] Set up monitoring (Sentry, DataDog, etc.) |
| - [ ] Configure auto-scaling |
| - [ ] Set up backups |
| - [ ] Add usage analytics |
| - [ ] Create user documentation |
| - [ ] Set up SSL/TLS (HF provides by default) |
| - [ ] Test with large files |
| - [ ] Load testing |
| - [ ] Security audit |
| - [ ] Legal compliance (GDPR, etc.) |
|
|
| --- |
|
|
| ## 🎉 You're Ready! |
|
|
| Your FastAPI PDF Redaction application is complete and ready to deploy! |
|
|
| ### Next Steps: |
| 1. ✨ Deploy to HuggingFace Spaces (easiest) |
| 2. 🧪 Test with real PDFs |
| 3. 📊 Monitor usage |
| 4. 🔒 Add security for production |
| 5. 🚀 Scale as needed |
|
|
| ### Support: |
| - 📖 Read the documentation |
| - 🐛 Check troubleshooting guide |
| - 💬 HuggingFace community forums |
| - 📧 Create issues on your repo |
|
|
| **Happy Deploying! 🚀** |
|
|
