Spaces:
Sleeping
Sleeping
GitHub Action commited on
Commit ·
e710d79
1
Parent(s): b63c95c
Sync from GitHub: 12072c8faa1a2b890eeca75571a82673d6b429db
Browse files- DEPLOYMENT_GUIDE.md +0 -419
- HUGGINGFACE_DEPLOYMENT.md +0 -123
- QUICK_START.md +0 -92
- README.md +0 -310
- railway.json +0 -13
- render.yaml +0 -26
- vercel.json +0 -20
DEPLOYMENT_GUIDE.md
DELETED
|
@@ -1,419 +0,0 @@
|
|
| 1 |
-
# CloverShield ML API Deployment Guide
|
| 2 |
-
|
| 3 |
-
Complete guide for deploying the ML Inference API to various platforms.
|
| 4 |
-
|
| 5 |
-
## 📋 Prerequisites
|
| 6 |
-
|
| 7 |
-
- Docker installed (for containerized deployment)
|
| 8 |
-
- Model file: `Models/fraud_pipeline_final.pkl`
|
| 9 |
-
- Environment variables configured
|
| 10 |
-
- Platform account (Vercel/Render/Railway)
|
| 11 |
-
|
| 12 |
-
## 🐳 Docker Deployment
|
| 13 |
-
|
| 14 |
-
### Local Docker
|
| 15 |
-
|
| 16 |
-
```bash
|
| 17 |
-
# Build image
|
| 18 |
-
docker build -t clovershield-ml-api .
|
| 19 |
-
|
| 20 |
-
# Run container
|
| 21 |
-
docker run -d \
|
| 22 |
-
--name clovershield-api \
|
| 23 |
-
-p 8000:8000 \
|
| 24 |
-
-v $(pwd)/Models:/app/Models \
|
| 25 |
-
-e GROQ_API_KEY=your-key \
|
| 26 |
-
clovershield-ml-api
|
| 27 |
-
|
| 28 |
-
# Check logs
|
| 29 |
-
docker logs clovershield-api
|
| 30 |
-
|
| 31 |
-
# Test
|
| 32 |
-
curl http://localhost:8000/health
|
| 33 |
-
```
|
| 34 |
-
|
| 35 |
-
### Docker Compose
|
| 36 |
-
|
| 37 |
-
Create `docker-compose.yml`:
|
| 38 |
-
|
| 39 |
-
```yaml
|
| 40 |
-
version: '3.8'
|
| 41 |
-
|
| 42 |
-
services:
|
| 43 |
-
ml-api:
|
| 44 |
-
build: .
|
| 45 |
-
ports:
|
| 46 |
-
- "8000:8000"
|
| 47 |
-
volumes:
|
| 48 |
-
- ./Models:/app/Models
|
| 49 |
-
environment:
|
| 50 |
-
- MODEL_PATH=Models/fraud_pipeline_final.pkl
|
| 51 |
-
- GROQ_API_KEY=${GROQ_API_KEY}
|
| 52 |
-
healthcheck:
|
| 53 |
-
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
|
| 54 |
-
interval: 30s
|
| 55 |
-
timeout: 10s
|
| 56 |
-
retries: 3
|
| 57 |
-
```
|
| 58 |
-
|
| 59 |
-
Run:
|
| 60 |
-
```bash
|
| 61 |
-
docker compose up -d
|
| 62 |
-
```
|
| 63 |
-
|
| 64 |
-
## ☁️ Platform-Specific Deployment
|
| 65 |
-
|
| 66 |
-
### Option 1: Render.com (Recommended)
|
| 67 |
-
|
| 68 |
-
**Pros:**
|
| 69 |
-
- Easy setup
|
| 70 |
-
- Good for ML workloads
|
| 71 |
-
- Free tier available
|
| 72 |
-
- Supports Docker
|
| 73 |
-
|
| 74 |
-
**Steps:**
|
| 75 |
-
|
| 76 |
-
1. **Create Account**: Sign up at [render.com](https://render.com)
|
| 77 |
-
|
| 78 |
-
2. **Create New Web Service**:
|
| 79 |
-
- Connect GitHub repository
|
| 80 |
-
- Select "Docker" as environment
|
| 81 |
-
- Use `render.yaml` configuration
|
| 82 |
-
|
| 83 |
-
3. **Configure Environment Variables**:
|
| 84 |
-
```
|
| 85 |
-
MODEL_PATH=Models/fraud_pipeline_final.pkl
|
| 86 |
-
MODEL_THRESHOLD=0.00754482
|
| 87 |
-
GROQ_API_KEY=your-key-here
|
| 88 |
-
```
|
| 89 |
-
|
| 90 |
-
4. **Deploy**:
|
| 91 |
-
- Render will auto-detect `render.yaml`
|
| 92 |
-
- Build and deploy automatically
|
| 93 |
-
- Get public URL: `https://clovershield-ml-api.onrender.com`
|
| 94 |
-
|
| 95 |
-
5. **Update Model File**:
|
| 96 |
-
- Upload model to cloud storage (S3, etc.)
|
| 97 |
-
- Modify Dockerfile to download on startup
|
| 98 |
-
- Or use Render's persistent disk
|
| 99 |
-
|
| 100 |
-
**Model File Options:**
|
| 101 |
-
- **Option A**: Include in Docker image (not recommended - large)
|
| 102 |
-
- **Option B**: Download from S3/GCS on startup
|
| 103 |
-
- **Option C**: Use Render persistent disk
|
| 104 |
-
|
| 105 |
-
### Option 2: Railway.app
|
| 106 |
-
|
| 107 |
-
**Pros:**
|
| 108 |
-
- Simple deployment
|
| 109 |
-
- Good developer experience
|
| 110 |
-
- Automatic HTTPS
|
| 111 |
-
|
| 112 |
-
**Steps:**
|
| 113 |
-
|
| 114 |
-
1. **Create Account**: Sign up at [railway.app](https://railway.app)
|
| 115 |
-
|
| 116 |
-
2. **Create New Project**:
|
| 117 |
-
- Connect GitHub repository
|
| 118 |
-
- Railway auto-detects `railway.json`
|
| 119 |
-
|
| 120 |
-
3. **Configure Environment Variables**:
|
| 121 |
-
- Go to Variables tab
|
| 122 |
-
- Add required variables
|
| 123 |
-
|
| 124 |
-
4. **Deploy**:
|
| 125 |
-
- Railway builds and deploys automatically
|
| 126 |
-
- Get public URL: `https://clovershield-ml-api.up.railway.app`
|
| 127 |
-
|
| 128 |
-
### Option 3: Vercel (Limited)
|
| 129 |
-
|
| 130 |
-
**Pros:**
|
| 131 |
-
- Fast deployment
|
| 132 |
-
- Great for frontend
|
| 133 |
-
|
| 134 |
-
**Cons:**
|
| 135 |
-
- Limited for ML workloads
|
| 136 |
-
- Serverless function timeout limits
|
| 137 |
-
- Model size restrictions
|
| 138 |
-
|
| 139 |
-
**Steps:**
|
| 140 |
-
|
| 141 |
-
1. **Install Vercel CLI**: `npm i -g vercel`
|
| 142 |
-
|
| 143 |
-
2. **Deploy**:
|
| 144 |
-
```bash
|
| 145 |
-
vercel
|
| 146 |
-
```
|
| 147 |
-
|
| 148 |
-
3. **Configure**:
|
| 149 |
-
- Set environment variables in dashboard
|
| 150 |
-
- Note: Model file must be < 50MB
|
| 151 |
-
|
| 152 |
-
**Note**: Vercel is not ideal for ML APIs. Use Render or Railway instead.
|
| 153 |
-
|
| 154 |
-
### Option 4: AWS/GCP/Azure
|
| 155 |
-
|
| 156 |
-
For production deployments, consider:
|
| 157 |
-
|
| 158 |
-
- **AWS**: ECS, EKS, or Lambda (with container support)
|
| 159 |
-
- **GCP**: Cloud Run or GKE
|
| 160 |
-
- **Azure**: Container Instances or AKS
|
| 161 |
-
|
| 162 |
-
## 🔧 Model File Management
|
| 163 |
-
|
| 164 |
-
### Option 1: Include in Docker Image
|
| 165 |
-
|
| 166 |
-
**Pros**: Simple
|
| 167 |
-
**Cons**: Large image size, slower builds
|
| 168 |
-
|
| 169 |
-
```dockerfile
|
| 170 |
-
COPY Models/fraud_pipeline_final.pkl Models/
|
| 171 |
-
```
|
| 172 |
-
|
| 173 |
-
### Option 2: Download from Cloud Storage
|
| 174 |
-
|
| 175 |
-
**Pros**: Smaller image, flexible
|
| 176 |
-
**Cons**: Requires cloud storage setup
|
| 177 |
-
|
| 178 |
-
Modify `main.py` startup:
|
| 179 |
-
|
| 180 |
-
```python
|
| 181 |
-
import boto3
|
| 182 |
-
import os
|
| 183 |
-
|
| 184 |
-
def download_model():
|
| 185 |
-
s3 = boto3.client('s3')
|
| 186 |
-
s3.download_file('your-bucket', 'fraud_pipeline_final.pkl', 'Models/fraud_pipeline_final.pkl')
|
| 187 |
-
```
|
| 188 |
-
|
| 189 |
-
### Option 3: Environment Variable Path
|
| 190 |
-
|
| 191 |
-
**Pros**: Flexible
|
| 192 |
-
**Cons**: Requires manual setup
|
| 193 |
-
|
| 194 |
-
Set `MODEL_PATH` to point to mounted volume or cloud path.
|
| 195 |
-
|
| 196 |
-
## 🔒 Production Security
|
| 197 |
-
|
| 198 |
-
### 1. API Authentication
|
| 199 |
-
|
| 200 |
-
Add API key middleware:
|
| 201 |
-
|
| 202 |
-
```python
|
| 203 |
-
from fastapi import Header, HTTPException
|
| 204 |
-
|
| 205 |
-
API_KEY = os.getenv("API_KEY")
|
| 206 |
-
|
| 207 |
-
@app.middleware("http")
|
| 208 |
-
async def verify_api_key(request: Request, call_next):
|
| 209 |
-
if request.url.path.startswith("/docs"):
|
| 210 |
-
return await call_next(request)
|
| 211 |
-
|
| 212 |
-
api_key = request.headers.get("X-API-Key")
|
| 213 |
-
if api_key != API_KEY:
|
| 214 |
-
raise HTTPException(status_code=401, detail="Invalid API key")
|
| 215 |
-
|
| 216 |
-
return await call_next(request)
|
| 217 |
-
```
|
| 218 |
-
|
| 219 |
-
### 2. Rate Limiting
|
| 220 |
-
|
| 221 |
-
Install: `pip install slowapi`
|
| 222 |
-
|
| 223 |
-
```python
|
| 224 |
-
from slowapi import Limiter, _rate_limit_exceeded_handler
|
| 225 |
-
from slowapi.util import get_remote_address
|
| 226 |
-
|
| 227 |
-
limiter = Limiter(key_func=get_remote_address)
|
| 228 |
-
app.state.limiter = limiter
|
| 229 |
-
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
|
| 230 |
-
|
| 231 |
-
@app.post("/predict")
|
| 232 |
-
@limiter.limit("10/minute")
|
| 233 |
-
async def predict(...):
|
| 234 |
-
...
|
| 235 |
-
```
|
| 236 |
-
|
| 237 |
-
### 3. HTTPS
|
| 238 |
-
|
| 239 |
-
- Use platform's built-in HTTPS (Render, Railway)
|
| 240 |
-
- Or configure reverse proxy (nginx, Traefik)
|
| 241 |
-
|
| 242 |
-
### 4. Monitoring
|
| 243 |
-
|
| 244 |
-
Add logging and monitoring:
|
| 245 |
-
|
| 246 |
-
```python
|
| 247 |
-
import logging
|
| 248 |
-
from prometheus_client import Counter, Histogram
|
| 249 |
-
|
| 250 |
-
request_count = Counter('requests_total', 'Total requests')
|
| 251 |
-
request_duration = Histogram('request_duration_seconds', 'Request duration')
|
| 252 |
-
|
| 253 |
-
@app.middleware("http")
|
| 254 |
-
async def log_requests(request: Request, call_next):
|
| 255 |
-
start_time = time.time()
|
| 256 |
-
response = await call_next(request)
|
| 257 |
-
duration = time.time() - start_time
|
| 258 |
-
request_duration.observe(duration)
|
| 259 |
-
request_count.inc()
|
| 260 |
-
return response
|
| 261 |
-
```
|
| 262 |
-
|
| 263 |
-
## 📊 Health Checks
|
| 264 |
-
|
| 265 |
-
All platforms support health checks:
|
| 266 |
-
|
| 267 |
-
```bash
|
| 268 |
-
# Test health endpoint
|
| 269 |
-
curl https://your-api-url.com/health
|
| 270 |
-
|
| 271 |
-
# Expected response
|
| 272 |
-
{
|
| 273 |
-
"status": "healthy",
|
| 274 |
-
"model_loaded": true,
|
| 275 |
-
"model_version": "1.0.0"
|
| 276 |
-
}
|
| 277 |
-
```
|
| 278 |
-
|
| 279 |
-
## 🧪 Testing Deployment
|
| 280 |
-
|
| 281 |
-
### 1. Health Check
|
| 282 |
-
|
| 283 |
-
```bash
|
| 284 |
-
curl https://your-api-url.com/health
|
| 285 |
-
```
|
| 286 |
-
|
| 287 |
-
### 2. Prediction Test
|
| 288 |
-
|
| 289 |
-
```bash
|
| 290 |
-
curl -X POST https://your-api-url.com/predict \
|
| 291 |
-
-H "Content-Type: application/json" \
|
| 292 |
-
-d '{
|
| 293 |
-
"transaction": {
|
| 294 |
-
"step": 1,
|
| 295 |
-
"type": "TRANSFER",
|
| 296 |
-
"amount": 5000.00,
|
| 297 |
-
"nameOrig": "C123456789",
|
| 298 |
-
"oldBalanceOrig": 50000.00,
|
| 299 |
-
"newBalanceOrig": 45000.00,
|
| 300 |
-
"nameDest": "C234567890",
|
| 301 |
-
"oldBalanceDest": 25000.00,
|
| 302 |
-
"newBalanceDest": 30000.00
|
| 303 |
-
}
|
| 304 |
-
}'
|
| 305 |
-
```
|
| 306 |
-
|
| 307 |
-
### 3. Load Testing
|
| 308 |
-
|
| 309 |
-
Use `ab` (Apache Bench) or `wrk`:
|
| 310 |
-
|
| 311 |
-
```bash
|
| 312 |
-
# Install
|
| 313 |
-
brew install wrk # macOS
|
| 314 |
-
# or
|
| 315 |
-
apt-get install apache2-utils # Linux
|
| 316 |
-
|
| 317 |
-
# Test
|
| 318 |
-
wrk -t4 -c100 -d30s --script=test.lua https://your-api-url.com/predict
|
| 319 |
-
```
|
| 320 |
-
|
| 321 |
-
## 🐛 Troubleshooting
|
| 322 |
-
|
| 323 |
-
### Model Not Loading
|
| 324 |
-
|
| 325 |
-
**Symptoms**: `503 Model not loaded` error
|
| 326 |
-
|
| 327 |
-
**Solutions**:
|
| 328 |
-
1. Check `MODEL_PATH` environment variable
|
| 329 |
-
2. Verify model file exists in container
|
| 330 |
-
3. Check file permissions
|
| 331 |
-
4. Review container logs
|
| 332 |
-
|
| 333 |
-
### Out of Memory
|
| 334 |
-
|
| 335 |
-
**Symptoms**: Container crashes, OOM errors
|
| 336 |
-
|
| 337 |
-
**Solutions**:
|
| 338 |
-
1. Increase container memory limit
|
| 339 |
-
2. Use model quantization
|
| 340 |
-
3. Implement model caching
|
| 341 |
-
4. Scale horizontally
|
| 342 |
-
|
| 343 |
-
### Slow Response Times
|
| 344 |
-
|
| 345 |
-
**Symptoms**: High latency
|
| 346 |
-
|
| 347 |
-
**Solutions**:
|
| 348 |
-
1. Optimize model (quantization, pruning)
|
| 349 |
-
2. Use faster hardware (GPU if available)
|
| 350 |
-
3. Implement caching
|
| 351 |
-
4. Optimize feature engineering
|
| 352 |
-
|
| 353 |
-
### CORS Errors
|
| 354 |
-
|
| 355 |
-
**Symptoms**: Frontend can't call API
|
| 356 |
-
|
| 357 |
-
**Solutions**:
|
| 358 |
-
1. Configure CORS in `main.py`
|
| 359 |
-
2. Add frontend domain to allowed origins
|
| 360 |
-
3. Check platform CORS settings
|
| 361 |
-
|
| 362 |
-
## 📈 Scaling
|
| 363 |
-
|
| 364 |
-
### Horizontal Scaling
|
| 365 |
-
|
| 366 |
-
- Use load balancer
|
| 367 |
-
- Deploy multiple instances
|
| 368 |
-
- Use platform auto-scaling
|
| 369 |
-
|
| 370 |
-
### Vertical Scaling
|
| 371 |
-
|
| 372 |
-
- Increase container resources
|
| 373 |
-
- Use faster CPU/GPU
|
| 374 |
-
- Optimize model
|
| 375 |
-
|
| 376 |
-
## ✅ Deployment Checklist
|
| 377 |
-
|
| 378 |
-
- [ ] Model file accessible
|
| 379 |
-
- [ ] Environment variables set
|
| 380 |
-
- [ ] Health check working
|
| 381 |
-
- [ ] API endpoints responding
|
| 382 |
-
- [ ] CORS configured
|
| 383 |
-
- [ ] Authentication enabled (production)
|
| 384 |
-
- [ ] Rate limiting configured
|
| 385 |
-
- [ ] Monitoring set up
|
| 386 |
-
- [ ] Logs accessible
|
| 387 |
-
- [ ] HTTPS enabled
|
| 388 |
-
- [ ] Backup strategy in place
|
| 389 |
-
|
| 390 |
-
## 🚀 Quick Deploy Commands
|
| 391 |
-
|
| 392 |
-
### Render
|
| 393 |
-
|
| 394 |
-
```bash
|
| 395 |
-
# Using Render CLI
|
| 396 |
-
render deploy
|
| 397 |
-
```
|
| 398 |
-
|
| 399 |
-
### Railway
|
| 400 |
-
|
| 401 |
-
```bash
|
| 402 |
-
# Using Railway CLI
|
| 403 |
-
railway up
|
| 404 |
-
```
|
| 405 |
-
|
| 406 |
-
### Docker Hub + Any Platform
|
| 407 |
-
|
| 408 |
-
```bash
|
| 409 |
-
# Build and push
|
| 410 |
-
docker build -t your-username/clovershield-ml-api .
|
| 411 |
-
docker push your-username/clovershield-ml-api
|
| 412 |
-
|
| 413 |
-
# Deploy to platform using image
|
| 414 |
-
```
|
| 415 |
-
|
| 416 |
-
---
|
| 417 |
-
|
| 418 |
-
**Status**: Ready for deployment! 🚀
|
| 419 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
HUGGINGFACE_DEPLOYMENT.md
DELETED
|
@@ -1,123 +0,0 @@
|
|
| 1 |
-
# Deploying CloverShield ML API to Hugging Face Spaces
|
| 2 |
-
|
| 3 |
-
This guide outlines the steps to migrate the backend ML API from Render to Hugging Face Spaces using the Docker SDK.
|
| 4 |
-
|
| 5 |
-
## Prerequisites
|
| 6 |
-
|
| 7 |
-
- A [Hugging Face account](https://huggingface.co/join).
|
| 8 |
-
- The `ml-api` folder ready with the updated `Dockerfile` and `README.md`.
|
| 9 |
-
|
| 10 |
-
## Step 1: Create a New Space
|
| 11 |
-
|
| 12 |
-
1. Log in to your Hugging Face account.
|
| 13 |
-
2. Click on your profile picture in the top right and select **"New Space"**.
|
| 14 |
-
3. Fill in the details:
|
| 15 |
-
- **Space Name:** `clovershield-ml-api` (or similar).
|
| 16 |
-
- **License:** MIT or Apache 2.0 (matches your project).
|
| 17 |
-
- **Space SDK:** Select **Docker**.
|
| 18 |
-
- **Space Hardware:** **CPU Basic (Free)** is usually sufficient for this model size (2MB). If performance is slow, you can upgrade later.
|
| 19 |
-
- **Visibility:** **Public** or **Private** (Private is safer for internal testing, but you might need a token to access it from the frontend). For the hackathon demo, **Public** is easiest but be careful with keys. **Private** is recommended if you can handle authentication headers in your frontend.
|
| 20 |
-
|
| 21 |
-
4. Click **"Create Space"**.
|
| 22 |
-
|
| 23 |
-
## Step 2: Upload Files
|
| 24 |
-
|
| 25 |
-
You have two options to upload the code:
|
| 26 |
-
|
| 27 |
-
### Option A: Web Interface (Easiest)
|
| 28 |
-
|
| 29 |
-
1. Once the Space is created, you will see the instructions page. Click on the **"Files"** tab.
|
| 30 |
-
2. Click **"Add file"** > **"Upload files"**.
|
| 31 |
-
3. Drag and drop **ALL** files and folders from your local `ml-api` directory into the upload area.
|
| 32 |
-
- **Important:** Ensure you include:
|
| 33 |
-
- `Dockerfile`
|
| 34 |
-
- `requirements.txt`
|
| 35 |
-
- `main.py`
|
| 36 |
-
- `inference.py`
|
| 37 |
-
- `feature_engineering.py`
|
| 38 |
-
- `entrypoint.sh`
|
| 39 |
-
- `README.md`
|
| 40 |
-
- `Models/fraud_pipeline_final.pkl` (The folder structure `Models/` must be preserved).
|
| 41 |
-
- `dataset/test_dataset.csv` (If used).
|
| 42 |
-
4. In the "Commit changes" box, type "Initial commit" and click **"Commit changes to main"**.
|
| 43 |
-
|
| 44 |
-
### Option B: Git Command Line (Manual)
|
| 45 |
-
|
| 46 |
-
1. Clone the Space repository locally.
|
| 47 |
-
2. Copy files and push manually.
|
| 48 |
-
|
| 49 |
-
### Option C: Automatic Sync with GitHub (Recommended)
|
| 50 |
-
|
| 51 |
-
This method automatically updates your Space whenever you push code to GitHub.
|
| 52 |
-
|
| 53 |
-
1. **Get a Hugging Face Access Token:**
|
| 54 |
-
* Go to [Hugging Face Settings > Access Tokens](https://huggingface.co/settings/tokens).
|
| 55 |
-
* Click "New token".
|
| 56 |
-
* Name it `GITHUB_ACTION` (or similar).
|
| 57 |
-
* **Permissions:** Select **"Write"** access.
|
| 58 |
-
* Copy the token (starts with `hf_...`).
|
| 59 |
-
|
| 60 |
-
2. **Add Token to GitHub Secrets:**
|
| 61 |
-
* Go to your GitHub repository.
|
| 62 |
-
* Navigate to **Settings** > **Secrets and variables** > **Actions**.
|
| 63 |
-
* Click **"New repository secret"**.
|
| 64 |
-
* **Name:** `HF_TOKEN`
|
| 65 |
-
* **Value:** (Paste your Hugging Face token here).
|
| 66 |
-
* Click "Add secret".
|
| 67 |
-
|
| 68 |
-
3. **Configure the Workflow:**
|
| 69 |
-
* Open the file `.github/workflows/deploy-ml-api.yml` in your project.
|
| 70 |
-
* Edit the `env` section to match your details:
|
| 71 |
-
```yaml
|
| 72 |
-
HF_SPACE_USERNAME: "your-hf-username"
|
| 73 |
-
HF_SPACE_NAME: "clovershield-ml-api"
|
| 74 |
-
```
|
| 75 |
-
* Commit and push this change to GitHub.
|
| 76 |
-
|
| 77 |
-
Now, whenever you push changes to the `ml-api` folder in GitHub, the Action will run and update your Hugging Face Space automatically.
|
| 78 |
-
|
| 79 |
-
## Step 3: Configure Environment Variables (Secrets)
|
| 80 |
-
|
| 81 |
-
Your API needs the `GROQ_API_KEY` to function if LLM explanations are enabled.
|
| 82 |
-
|
| 83 |
-
1. Go to the **"Settings"** tab of your Space.
|
| 84 |
-
2. Scroll down to the **"Variables and secrets"** section.
|
| 85 |
-
3. Click **"New secret"**.
|
| 86 |
-
- **Name:** `GROQ_API_KEY`
|
| 87 |
-
- **Value:** (Paste your Groq API key here)
|
| 88 |
-
4. (Optional) Add other variables if you changed defaults:
|
| 89 |
-
- `MODEL_THRESHOLD`
|
| 90 |
-
- `PAGERANK_LIMIT`
|
| 91 |
-
|
| 92 |
-
**Note:** Hugging Face automatically sets `PORT` to `7860`. Our updated `Dockerfile` and `entrypoint.sh` are configured to respect this.
|
| 93 |
-
|
| 94 |
-
## Step 4: Build and Verify
|
| 95 |
-
|
| 96 |
-
1. Go to the **"App"** tab.
|
| 97 |
-
2. You will see a "Building" status. Click "Logs" to watch the build process.
|
| 98 |
-
3. Once "Running", you should see the Swagger UI or the JSON response from the root endpoint in the preview window.
|
| 99 |
-
4. Your API URL will be: `https://<your-username>-clovershield-ml-api.hf.space`
|
| 100 |
-
|
| 101 |
-
## Step 5: Update Frontend Connection (Vercel)
|
| 102 |
-
|
| 103 |
-
Now that your backend is on HF Spaces, you need to update your Vercel deployment to point to the new URL.
|
| 104 |
-
|
| 105 |
-
1. **Go to Vercel Dashboard:**
|
| 106 |
-
* Navigate to your project on [Vercel](https://vercel.com).
|
| 107 |
-
* Go to **Settings** > **Environment Variables**.
|
| 108 |
-
|
| 109 |
-
2. **Update the Variable:**
|
| 110 |
-
* Find the existing `NEXT_PUBLIC_ML_API_URL` or create a new one.
|
| 111 |
-
* **Value:** `https://<your-username>-<your-space-name>.hf.space`
|
| 112 |
-
* *Important:* Do NOT include a trailing slash (/) at the end.
|
| 113 |
-
|
| 114 |
-
3. **Redeploy:**
|
| 115 |
-
* Go to the **Deployments** tab.
|
| 116 |
-
* Click on the three dots (...) of your latest deployment and select **Redeploy**. This is necessary for Vercel to pick up the new environment variable value in the client-side code.
|
| 117 |
-
|
| 118 |
-
## Troubleshooting
|
| 119 |
-
|
| 120 |
-
- **Build Fails:** Check the "Logs". Common issues are missing dependencies in `requirements.txt` or file path errors.
|
| 121 |
-
- **"403 Forbidden":** If your Space is **Private**, you cannot access it directly from a public frontend without passing an authorization header (Bearer token). For this hackathon/demo, switch the Space Visibility to **Public** in Settings > General.
|
| 122 |
-
- **Model Load Error:** Ensure `Models/fraud_pipeline_final.pkl` was actually uploaded. Check the "Files" tab.
|
| 123 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
QUICK_START.md
DELETED
|
@@ -1,92 +0,0 @@
|
|
| 1 |
-
# ML API Quick Start
|
| 2 |
-
|
| 3 |
-
Quick reference for deploying CloverShield ML Inference API.
|
| 4 |
-
|
| 5 |
-
## ✅ What's Included
|
| 6 |
-
|
| 7 |
-
- ✅ FastAPI service with `/predict` endpoint
|
| 8 |
-
- ✅ Docker containerization
|
| 9 |
-
- ✅ Deployment configs for Render/Railway/Vercel
|
| 10 |
-
- ✅ Complete documentation
|
| 11 |
-
|
| 12 |
-
## 🚀 Deploy in 5 Minutes
|
| 13 |
-
|
| 14 |
-
### Option 1: Render.com (Recommended)
|
| 15 |
-
|
| 16 |
-
1. **Sign up** at [render.com](https://render.com)
|
| 17 |
-
2. **Create Web Service** → Connect GitHub repo
|
| 18 |
-
3. **Configure**:
|
| 19 |
-
- Environment: Docker
|
| 20 |
-
- Build Command: (auto-detected)
|
| 21 |
-
- Start Command: (auto-detected)
|
| 22 |
-
4. **Set Environment Variables**:
|
| 23 |
-
```
|
| 24 |
-
MODEL_PATH=Models/fraud_pipeline_final.pkl
|
| 25 |
-
GROQ_API_KEY=your-key (optional)
|
| 26 |
-
```
|
| 27 |
-
5. **Deploy** → Get URL: `https://your-app.onrender.com`
|
| 28 |
-
|
| 29 |
-
### Option 2: Railway.app
|
| 30 |
-
|
| 31 |
-
1. **Sign up** at [railway.app](https://railway.app)
|
| 32 |
-
2. **New Project** → Connect GitHub
|
| 33 |
-
3. **Railway auto-detects** `railway.json`
|
| 34 |
-
4. **Set environment variables** in dashboard
|
| 35 |
-
5. **Deploy** → Get URL automatically
|
| 36 |
-
|
| 37 |
-
### Option 3: Local Docker
|
| 38 |
-
|
| 39 |
-
```bash
|
| 40 |
-
# Build
|
| 41 |
-
docker build -t clovershield-ml-api .
|
| 42 |
-
|
| 43 |
-
# Run
|
| 44 |
-
docker run -p 8000:8000 \
|
| 45 |
-
-v $(pwd)/Models:/app/Models \
|
| 46 |
-
clovershield-ml-api
|
| 47 |
-
|
| 48 |
-
# Test
|
| 49 |
-
curl http://localhost:8000/health
|
| 50 |
-
```
|
| 51 |
-
|
| 52 |
-
## 📝 API Usage
|
| 53 |
-
|
| 54 |
-
### Health Check
|
| 55 |
-
```bash
|
| 56 |
-
curl https://your-api-url.com/health
|
| 57 |
-
```
|
| 58 |
-
|
| 59 |
-
### Predict Fraud
|
| 60 |
-
```bash
|
| 61 |
-
curl -X POST https://your-api-url.com/predict \
|
| 62 |
-
-H "Content-Type: application/json" \
|
| 63 |
-
-d '{
|
| 64 |
-
"transaction": {
|
| 65 |
-
"step": 1,
|
| 66 |
-
"type": "TRANSFER",
|
| 67 |
-
"amount": 5000.00,
|
| 68 |
-
"nameOrig": "C123456789",
|
| 69 |
-
"oldBalanceOrig": 50000.00,
|
| 70 |
-
"newBalanceOrig": 45000.00,
|
| 71 |
-
"nameDest": "C234567890",
|
| 72 |
-
"oldBalanceDest": 25000.00,
|
| 73 |
-
"newBalanceDest": 30000.00
|
| 74 |
-
}
|
| 75 |
-
}'
|
| 76 |
-
```
|
| 77 |
-
|
| 78 |
-
## 📚 Full Documentation
|
| 79 |
-
|
| 80 |
-
- [README.md](./README.md) - Complete API documentation
|
| 81 |
-
- [DEPLOYMENT_GUIDE.md](./DEPLOYMENT_GUIDE.md) - Detailed deployment guide
|
| 82 |
-
|
| 83 |
-
## ✅ Checklist
|
| 84 |
-
|
| 85 |
-
- [ ] Model file available (`Models/fraud_pipeline_final.pkl`)
|
| 86 |
-
- [ ] Environment variables configured
|
| 87 |
-
- [ ] API deployed and accessible
|
| 88 |
-
- [ ] Health check passing
|
| 89 |
-
- [ ] Test prediction working
|
| 90 |
-
|
| 91 |
-
**Ready to deploy!** 🚀
|
| 92 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
DELETED
|
@@ -1,310 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
title: CloverShield ML API
|
| 3 |
-
emoji: 🛡️
|
| 4 |
-
colorFrom: green
|
| 5 |
-
colorTo: blue
|
| 6 |
-
sdk: docker
|
| 7 |
-
app_port: 7860
|
| 8 |
-
---
|
| 9 |
-
|
| 10 |
-
# CloverShield ML Inference API
|
| 11 |
-
|
| 12 |
-
FastAPI microservice for fraud detection predictions. This service replaces the local model loading mechanism in the Streamlit app.
|
| 13 |
-
|
| 14 |
-
## 🚀 Quick Start
|
| 15 |
-
|
| 16 |
-
### Local Development
|
| 17 |
-
|
| 18 |
-
1. **Install dependencies:**
|
| 19 |
-
```bash
|
| 20 |
-
pip install -r requirements.txt
|
| 21 |
-
```
|
| 22 |
-
|
| 23 |
-
2. **Copy model file:**
|
| 24 |
-
```bash
|
| 25 |
-
cp ../Models/fraud_pipeline_final.pkl Models/
|
| 26 |
-
```
|
| 27 |
-
|
| 28 |
-
3. **Set environment variables:**
|
| 29 |
-
```bash
|
| 30 |
-
cp env.template .env
|
| 31 |
-
# Edit .env with your values
|
| 32 |
-
```
|
| 33 |
-
|
| 34 |
-
4. **Run the API:**
|
| 35 |
-
```bash
|
| 36 |
-
python main.py
|
| 37 |
-
# Or
|
| 38 |
-
uvicorn main:app --reload
|
| 39 |
-
```
|
| 40 |
-
|
| 41 |
-
5. **Test the API:**
|
| 42 |
-
```bash
|
| 43 |
-
curl http://localhost:8000/health
|
| 44 |
-
```
|
| 45 |
-
|
| 46 |
-
### Docker
|
| 47 |
-
|
| 48 |
-
1. **Build the image:**
|
| 49 |
-
```bash
|
| 50 |
-
docker build -t clovershield-ml-api .
|
| 51 |
-
```
|
| 52 |
-
|
| 53 |
-
2. **Run the container:**
|
| 54 |
-
```bash
|
| 55 |
-
docker run -p 8000:8000 \
|
| 56 |
-
-v $(pwd)/Models:/app/Models \
|
| 57 |
-
-e GROQ_API_KEY=your-key \
|
| 58 |
-
clovershield-ml-api
|
| 59 |
-
```
|
| 60 |
-
|
| 61 |
-
## 📚 API Endpoints
|
| 62 |
-
|
| 63 |
-
### `GET /`
|
| 64 |
-
Root endpoint - service information
|
| 65 |
-
|
| 66 |
-
### `GET /health`
|
| 67 |
-
Health check endpoint
|
| 68 |
-
|
| 69 |
-
**Response:**
|
| 70 |
-
```json
|
| 71 |
-
{
|
| 72 |
-
"status": "healthy",
|
| 73 |
-
"model_loaded": true,
|
| 74 |
-
"model_version": "1.0.0",
|
| 75 |
-
"shap_available": true,
|
| 76 |
-
"llm_available": true
|
| 77 |
-
}
|
| 78 |
-
```
|
| 79 |
-
|
| 80 |
-
### `GET /model/info`
|
| 81 |
-
Get model metadata
|
| 82 |
-
|
| 83 |
-
### `POST /predict`
|
| 84 |
-
Predict fraud probability for a single transaction
|
| 85 |
-
|
| 86 |
-
**Request:**
|
| 87 |
-
```json
|
| 88 |
-
{
|
| 89 |
-
"transaction": {
|
| 90 |
-
"step": 1,
|
| 91 |
-
"type": "CASH_OUT",
|
| 92 |
-
"amount": 5000.00,
|
| 93 |
-
"nameOrig": "C123456789",
|
| 94 |
-
"oldBalanceOrig": 50000.00,
|
| 95 |
-
"newBalanceOrig": 45000.00,
|
| 96 |
-
"nameDest": "C234567890",
|
| 97 |
-
"oldBalanceDest": 25000.00,
|
| 98 |
-
"newBalanceDest": 30000.00
|
| 99 |
-
},
|
| 100 |
-
"options": {
|
| 101 |
-
"include_shap": true,
|
| 102 |
-
"include_llm_explanation": false,
|
| 103 |
-
"language": "en",
|
| 104 |
-
"topk": 10
|
| 105 |
-
}
|
| 106 |
-
}
|
| 107 |
-
```
|
| 108 |
-
|
| 109 |
-
**Response:**
|
| 110 |
-
```json
|
| 111 |
-
{
|
| 112 |
-
"transaction_id": "550e8400-e29b-41d4-a716-446655440000",
|
| 113 |
-
"prediction": {
|
| 114 |
-
"fraud_probability": 0.2345,
|
| 115 |
-
"decision": "pass",
|
| 116 |
-
"risk_level": "low",
|
| 117 |
-
"confidence": 0.85
|
| 118 |
-
},
|
| 119 |
-
"shap_explanations": [
|
| 120 |
-
{
|
| 121 |
-
"feature": "amount_over_oldBalanceOrig",
|
| 122 |
-
"value": 0.1,
|
| 123 |
-
"shap": 0.0234,
|
| 124 |
-
"shap_abs": 0.0234,
|
| 125 |
-
"rank": 1
|
| 126 |
-
}
|
| 127 |
-
],
|
| 128 |
-
"llm_explanation": null,
|
| 129 |
-
"processing_time_ms": 145,
|
| 130 |
-
"model_version": "1.0.0",
|
| 131 |
-
"timestamp": "2024-01-15T10:30:00Z"
|
| 132 |
-
}
|
| 133 |
-
```
|
| 134 |
-
|
| 135 |
-
### `POST /predict/batch`
|
| 136 |
-
Predict fraud probability for multiple transactions
|
| 137 |
-
|
| 138 |
-
## 🐳 Docker Deployment
|
| 139 |
-
|
| 140 |
-
### Build and Push to Registry
|
| 141 |
-
|
| 142 |
-
```bash
|
| 143 |
-
# Build
|
| 144 |
-
docker build -t clovershield-ml-api:latest .
|
| 145 |
-
|
| 146 |
-
# Tag for registry
|
| 147 |
-
docker tag clovershield-ml-api:latest your-registry/clovershield-ml-api:latest
|
| 148 |
-
|
| 149 |
-
# Push
|
| 150 |
-
docker push your-registry/clovershield-ml-api:latest
|
| 151 |
-
```
|
| 152 |
-
|
| 153 |
-
## ☁️ Platform Deployment
|
| 154 |
-
|
| 155 |
-
### Hugging Face Spaces (Recommended)
|
| 156 |
-
|
| 157 |
-
1. Create a new Space on Hugging Face.
|
| 158 |
-
2. Select "Docker" as the SDK.
|
| 159 |
-
3. Upload the contents of this directory to the Space.
|
| 160 |
-
4. Set the `GROQ_API_KEY` secret in the Space settings.
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
### Vercel
|
| 164 |
-
|
| 165 |
-
1. Install Vercel CLI: `npm i -g vercel`
|
| 166 |
-
2. Deploy: `vercel`
|
| 167 |
-
3. Set environment variables in Vercel dashboard
|
| 168 |
-
|
| 169 |
-
**Note:** Vercel has limitations for ML models. Consider Render or Railway for better performance.
|
| 170 |
-
|
| 171 |
-
### Render
|
| 172 |
-
|
| 173 |
-
1. Connect your GitHub repository
|
| 174 |
-
2. Create new Web Service
|
| 175 |
-
3. Use `render.yaml` configuration
|
| 176 |
-
4. Set environment variables in dashboard
|
| 177 |
-
5. Deploy
|
| 178 |
-
|
| 179 |
-
### Railway
|
| 180 |
-
|
| 181 |
-
1. Connect your GitHub repository
|
| 182 |
-
2. Create new project
|
| 183 |
-
3. Railway will auto-detect `railway.json`
|
| 184 |
-
4. Set environment variables
|
| 185 |
-
5. Deploy
|
| 186 |
-
|
| 187 |
-
## 🔧 Configuration
|
| 188 |
-
|
| 189 |
-
### Environment Variables
|
| 190 |
-
|
| 191 |
-
- `MODEL_PATH` - Path to model file (default: `Models/fraud_pipeline_final.pkl`)
|
| 192 |
-
- `TEST_DATASET_PATH` - Path to test dataset CSV for feature engineering (optional, auto-detected if not provided)
|
| 193 |
-
- `MODEL_THRESHOLD` - Decision threshold (default: `0.00754482` - optimized for 99% recall)
|
| 194 |
-
- `PORT` - Server port (default: `8000`)
|
| 195 |
-
- `HOST` - Server host (default: `0.0.0.0`)
|
| 196 |
-
- `GROQ_API_KEY` - Optional Groq API key for LLM explanations
|
| 197 |
-
- `PAGERANK_LIMIT` - Optional limit on nodes for PageRank computation (for memory optimization)
|
| 198 |
-
- `MAX_FIT_ROWS` - Maximum rows to use for feature engineering fitting (default: `50000`)
|
| 199 |
-
|
| 200 |
-
### Model File
|
| 201 |
-
|
| 202 |
-
The model file (`fraud_pipeline_final.pkl`) must be available at runtime. Options:
|
| 203 |
-
|
| 204 |
-
1. **Include in Docker image** (not recommended for large files)
|
| 205 |
-
2. **Mount as volume** (Docker)
|
| 206 |
-
3. **Download from cloud storage** (S3, GCS, etc.)
|
| 207 |
-
4. **Use environment variable** to specify path
|
| 208 |
-
|
| 209 |
-
## 📊 API Documentation
|
| 210 |
-
|
| 211 |
-
Interactive API documentation available at:
|
| 212 |
-
- Swagger UI: `http://localhost:8000/docs`
|
| 213 |
-
- ReDoc: `http://localhost:8000/redoc`
|
| 214 |
-
|
| 215 |
-
## 🧪 Testing
|
| 216 |
-
|
| 217 |
-
### Test with curl
|
| 218 |
-
|
| 219 |
-
```bash
|
| 220 |
-
# Health check
|
| 221 |
-
curl http://localhost:8000/health
|
| 222 |
-
|
| 223 |
-
# Prediction
|
| 224 |
-
curl -X POST http://localhost:8000/predict \
|
| 225 |
-
-H "Content-Type: application/json" \
|
| 226 |
-
-d '{
|
| 227 |
-
"transaction": {
|
| 228 |
-
"step": 1,
|
| 229 |
-
"type": "TRANSFER",
|
| 230 |
-
"amount": 5000.00,
|
| 231 |
-
"nameOrig": "C123456789",
|
| 232 |
-
"oldBalanceOrig": 50000.00,
|
| 233 |
-
"newBalanceOrig": 45000.00,
|
| 234 |
-
"nameDest": "C234567890",
|
| 235 |
-
"oldBalanceDest": 25000.00,
|
| 236 |
-
"newBalanceDest": 30000.00
|
| 237 |
-
}
|
| 238 |
-
}'
|
| 239 |
-
```
|
| 240 |
-
|
| 241 |
-
### Test with Python
|
| 242 |
-
|
| 243 |
-
```python
|
| 244 |
-
import requests
|
| 245 |
-
|
| 246 |
-
response = requests.post(
|
| 247 |
-
"http://localhost:8000/predict",
|
| 248 |
-
json={
|
| 249 |
-
"transaction": {
|
| 250 |
-
"step": 1,
|
| 251 |
-
"type": "TRANSFER",
|
| 252 |
-
"amount": 5000.00,
|
| 253 |
-
"nameOrig": "C123456789",
|
| 254 |
-
"oldBalanceOrig": 50000.00,
|
| 255 |
-
"newBalanceOrig": 45000.00,
|
| 256 |
-
"nameDest": "C234567890",
|
| 257 |
-
"oldBalanceDest": 25000.00,
|
| 258 |
-
"newBalanceDest": 30000.00
|
| 259 |
-
}
|
| 260 |
-
}
|
| 261 |
-
)
|
| 262 |
-
|
| 263 |
-
print(response.json())
|
| 264 |
-
```
|
| 265 |
-
|
| 266 |
-
## 🔒 Security
|
| 267 |
-
|
| 268 |
-
**Production Recommendations:**
|
| 269 |
-
- Add API key authentication for production
|
| 270 |
-
- Use HTTPS in production
|
| 271 |
-
- Implement rate limiting
|
| 272 |
-
- Validate all inputs
|
| 273 |
-
- Monitor for abuse
|
| 274 |
-
- Restrict CORS origins (currently set to `["*"` for development)
|
| 275 |
-
- Use environment variables for sensitive data
|
| 276 |
-
|
| 277 |
-
## 📈 Performance
|
| 278 |
-
|
| 279 |
-
**Optimization Tips:**
|
| 280 |
-
- Model loads once on startup (lazy loading supported for serverless)
|
| 281 |
-
- Single worker recommended (ML models are not thread-safe)
|
| 282 |
-
- Consider caching for frequently accessed data
|
| 283 |
-
- Monitor memory usage (XGBoost models can be large)
|
| 284 |
-
- Feature engineering uses sampled dataset (configurable via `MAX_FIT_ROWS`)
|
| 285 |
-
- SHAP background uses minimal sample size (100 rows) for efficiency
|
| 286 |
-
|
| 287 |
-
## 🐛 Troubleshooting
|
| 288 |
-
|
| 289 |
-
### Model not loading
|
| 290 |
-
|
| 291 |
-
- Check `MODEL_PATH` environment variable
|
| 292 |
-
- Verify model file exists
|
| 293 |
-
- Check file permissions
|
| 294 |
-
- Review logs for error messages
|
| 295 |
-
|
| 296 |
-
### SHAP not working
|
| 297 |
-
|
| 298 |
-
- Ensure XGBoost is installed
|
| 299 |
-
- Check model type compatibility
|
| 300 |
-
- Review SHAP initialization logs
|
| 301 |
-
|
| 302 |
-
### LLM explanations failing
|
| 303 |
-
|
| 304 |
-
- Verify `GROQ_API_KEY` is set
|
| 305 |
-
- Check API key validity
|
| 306 |
-
- Review Groq API status
|
| 307 |
-
|
| 308 |
-
## 📝 License
|
| 309 |
-
|
| 310 |
-
See main project LICENSE file.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
railway.json
DELETED
|
@@ -1,13 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"$schema": "https://railway.app/railway.schema.json",
|
| 3 |
-
"build": {
|
| 4 |
-
"builder": "DOCKERFILE",
|
| 5 |
-
"dockerfilePath": "Dockerfile"
|
| 6 |
-
},
|
| 7 |
-
"deploy": {
|
| 8 |
-
"startCommand": "uvicorn main:app --host 0.0.0.0 --port $PORT",
|
| 9 |
-
"restartPolicyType": "ON_FAILURE",
|
| 10 |
-
"restartPolicyMaxRetries": 10
|
| 11 |
-
}
|
| 12 |
-
}
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
render.yaml
DELETED
|
@@ -1,26 +0,0 @@
|
|
| 1 |
-
# Render.com deployment configuration for CloverShield ML API
|
| 2 |
-
|
| 3 |
-
services:
|
| 4 |
-
- type: web
|
| 5 |
-
name: clovershield-ml-api
|
| 6 |
-
runtime: docker
|
| 7 |
-
dockerfilePath: ./Dockerfile
|
| 8 |
-
dockerContext: .
|
| 9 |
-
envVars:
|
| 10 |
-
- key: MODEL_PATH
|
| 11 |
-
value: Models/fraud_pipeline_final.pkl
|
| 12 |
-
- key: MODEL_THRESHOLD
|
| 13 |
-
value: 0.0793
|
| 14 |
-
- key: PORT
|
| 15 |
-
value: 8000
|
| 16 |
-
- key: HOST
|
| 17 |
-
value: 0.0.0.0
|
| 18 |
-
- key: MAX_FIT_ROWS
|
| 19 |
-
value: "30000" # Reduced for Render free tier (512MB RAM)
|
| 20 |
-
- key: PAGERANK_LIMIT
|
| 21 |
-
value: "5000" # Reduced for memory efficiency
|
| 22 |
-
- key: GROQ_API_KEY
|
| 23 |
-
sync: false # Set in Render dashboard
|
| 24 |
-
healthCheckPath: /health
|
| 25 |
-
plan: starter # Upgrade for production
|
| 26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
vercel.json
DELETED
|
@@ -1,20 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"version": 2,
|
| 3 |
-
"builds": [
|
| 4 |
-
{
|
| 5 |
-
"src": "main.py",
|
| 6 |
-
"use": "@vercel/python"
|
| 7 |
-
}
|
| 8 |
-
],
|
| 9 |
-
"routes": [
|
| 10 |
-
{
|
| 11 |
-
"src": "/(.*)",
|
| 12 |
-
"dest": "main.py"
|
| 13 |
-
}
|
| 14 |
-
],
|
| 15 |
-
"env": {
|
| 16 |
-
"MODEL_PATH": "Models/fraud_pipeline_final.pkl",
|
| 17 |
-
"MODEL_THRESHOLD": "0.0793"
|
| 18 |
-
}
|
| 19 |
-
}
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|