GitHub Action commited on
Commit
e710d79
·
1 Parent(s): b63c95c

Sync from GitHub: 12072c8faa1a2b890eeca75571a82673d6b429db

Browse files
Files changed (7) hide show
  1. DEPLOYMENT_GUIDE.md +0 -419
  2. HUGGINGFACE_DEPLOYMENT.md +0 -123
  3. QUICK_START.md +0 -92
  4. README.md +0 -310
  5. railway.json +0 -13
  6. render.yaml +0 -26
  7. vercel.json +0 -20
DEPLOYMENT_GUIDE.md DELETED
@@ -1,419 +0,0 @@
1
- # CloverShield ML API Deployment Guide
2
-
3
- Complete guide for deploying the ML Inference API to various platforms.
4
-
5
- ## 📋 Prerequisites
6
-
7
- - Docker installed (for containerized deployment)
8
- - Model file: `Models/fraud_pipeline_final.pkl`
9
- - Environment variables configured
10
- - Platform account (Vercel/Render/Railway)
11
-
12
- ## 🐳 Docker Deployment
13
-
14
- ### Local Docker
15
-
16
- ```bash
17
- # Build image
18
- docker build -t clovershield-ml-api .
19
-
20
- # Run container
21
- docker run -d \
22
- --name clovershield-api \
23
- -p 8000:8000 \
24
- -v $(pwd)/Models:/app/Models \
25
- -e GROQ_API_KEY=your-key \
26
- clovershield-ml-api
27
-
28
- # Check logs
29
- docker logs clovershield-api
30
-
31
- # Test
32
- curl http://localhost:8000/health
33
- ```
34
-
35
- ### Docker Compose
36
-
37
- Create `docker-compose.yml`:
38
-
39
- ```yaml
40
- version: '3.8'
41
-
42
- services:
43
- ml-api:
44
- build: .
45
- ports:
46
- - "8000:8000"
47
- volumes:
48
- - ./Models:/app/Models
49
- environment:
50
- - MODEL_PATH=Models/fraud_pipeline_final.pkl
51
- - GROQ_API_KEY=${GROQ_API_KEY}
52
- healthcheck:
53
- test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
54
- interval: 30s
55
- timeout: 10s
56
- retries: 3
57
- ```
58
-
59
- Run:
60
- ```bash
61
- docker compose up -d
62
- ```
63
-
64
- ## ☁️ Platform-Specific Deployment
65
-
66
- ### Option 1: Render.com (Recommended)
67
-
68
- **Pros:**
69
- - Easy setup
70
- - Good for ML workloads
71
- - Free tier available
72
- - Supports Docker
73
-
74
- **Steps:**
75
-
76
- 1. **Create Account**: Sign up at [render.com](https://render.com)
77
-
78
- 2. **Create New Web Service**:
79
- - Connect GitHub repository
80
- - Select "Docker" as environment
81
- - Use `render.yaml` configuration
82
-
83
- 3. **Configure Environment Variables**:
84
- ```
85
- MODEL_PATH=Models/fraud_pipeline_final.pkl
86
- MODEL_THRESHOLD=0.00754482
87
- GROQ_API_KEY=your-key-here
88
- ```
89
-
90
- 4. **Deploy**:
91
- - Render will auto-detect `render.yaml`
92
- - Build and deploy automatically
93
- - Get public URL: `https://clovershield-ml-api.onrender.com`
94
-
95
- 5. **Update Model File**:
96
- - Upload model to cloud storage (S3, etc.)
97
- - Modify Dockerfile to download on startup
98
- - Or use Render's persistent disk
99
-
100
- **Model File Options:**
101
- - **Option A**: Include in Docker image (not recommended - large)
102
- - **Option B**: Download from S3/GCS on startup
103
- - **Option C**: Use Render persistent disk
104
-
105
- ### Option 2: Railway.app
106
-
107
- **Pros:**
108
- - Simple deployment
109
- - Good developer experience
110
- - Automatic HTTPS
111
-
112
- **Steps:**
113
-
114
- 1. **Create Account**: Sign up at [railway.app](https://railway.app)
115
-
116
- 2. **Create New Project**:
117
- - Connect GitHub repository
118
- - Railway auto-detects `railway.json`
119
-
120
- 3. **Configure Environment Variables**:
121
- - Go to Variables tab
122
- - Add required variables
123
-
124
- 4. **Deploy**:
125
- - Railway builds and deploys automatically
126
- - Get public URL: `https://clovershield-ml-api.up.railway.app`
127
-
128
- ### Option 3: Vercel (Limited)
129
-
130
- **Pros:**
131
- - Fast deployment
132
- - Great for frontend
133
-
134
- **Cons:**
135
- - Limited for ML workloads
136
- - Serverless function timeout limits
137
- - Model size restrictions
138
-
139
- **Steps:**
140
-
141
- 1. **Install Vercel CLI**: `npm i -g vercel`
142
-
143
- 2. **Deploy**:
144
- ```bash
145
- vercel
146
- ```
147
-
148
- 3. **Configure**:
149
- - Set environment variables in dashboard
150
- - Note: Model file must be < 50MB
151
-
152
- **Note**: Vercel is not ideal for ML APIs. Use Render or Railway instead.
153
-
154
- ### Option 4: AWS/GCP/Azure
155
-
156
- For production deployments, consider:
157
-
158
- - **AWS**: ECS, EKS, or Lambda (with container support)
159
- - **GCP**: Cloud Run or GKE
160
- - **Azure**: Container Instances or AKS
161
-
162
- ## 🔧 Model File Management
163
-
164
- ### Option 1: Include in Docker Image
165
-
166
- **Pros**: Simple
167
- **Cons**: Large image size, slower builds
168
-
169
- ```dockerfile
170
- COPY Models/fraud_pipeline_final.pkl Models/
171
- ```
172
-
173
- ### Option 2: Download from Cloud Storage
174
-
175
- **Pros**: Smaller image, flexible
176
- **Cons**: Requires cloud storage setup
177
-
178
- Modify `main.py` startup:
179
-
180
- ```python
181
- import boto3
182
- import os
183
-
184
- def download_model():
185
- s3 = boto3.client('s3')
186
- s3.download_file('your-bucket', 'fraud_pipeline_final.pkl', 'Models/fraud_pipeline_final.pkl')
187
- ```
188
-
189
- ### Option 3: Environment Variable Path
190
-
191
- **Pros**: Flexible
192
- **Cons**: Requires manual setup
193
-
194
- Set `MODEL_PATH` to point to mounted volume or cloud path.
195
-
196
- ## 🔒 Production Security
197
-
198
- ### 1. API Authentication
199
-
200
- Add API key middleware:
201
-
202
- ```python
203
- from fastapi import Header, HTTPException
204
-
205
- API_KEY = os.getenv("API_KEY")
206
-
207
- @app.middleware("http")
208
- async def verify_api_key(request: Request, call_next):
209
- if request.url.path.startswith("/docs"):
210
- return await call_next(request)
211
-
212
- api_key = request.headers.get("X-API-Key")
213
- if api_key != API_KEY:
214
- raise HTTPException(status_code=401, detail="Invalid API key")
215
-
216
- return await call_next(request)
217
- ```
218
-
219
- ### 2. Rate Limiting
220
-
221
- Install: `pip install slowapi`
222
-
223
- ```python
224
- from slowapi import Limiter, _rate_limit_exceeded_handler
225
- from slowapi.util import get_remote_address
226
-
227
- limiter = Limiter(key_func=get_remote_address)
228
- app.state.limiter = limiter
229
- app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
230
-
231
- @app.post("/predict")
232
- @limiter.limit("10/minute")
233
- async def predict(...):
234
- ...
235
- ```
236
-
237
- ### 3. HTTPS
238
-
239
- - Use platform's built-in HTTPS (Render, Railway)
240
- - Or configure reverse proxy (nginx, Traefik)
241
-
242
- ### 4. Monitoring
243
-
244
- Add logging and monitoring:
245
-
246
- ```python
247
- import logging
248
- from prometheus_client import Counter, Histogram
249
-
250
- request_count = Counter('requests_total', 'Total requests')
251
- request_duration = Histogram('request_duration_seconds', 'Request duration')
252
-
253
- @app.middleware("http")
254
- async def log_requests(request: Request, call_next):
255
- start_time = time.time()
256
- response = await call_next(request)
257
- duration = time.time() - start_time
258
- request_duration.observe(duration)
259
- request_count.inc()
260
- return response
261
- ```
262
-
263
- ## 📊 Health Checks
264
-
265
- All platforms support health checks:
266
-
267
- ```bash
268
- # Test health endpoint
269
- curl https://your-api-url.com/health
270
-
271
- # Expected response
272
- {
273
- "status": "healthy",
274
- "model_loaded": true,
275
- "model_version": "1.0.0"
276
- }
277
- ```
278
-
279
- ## 🧪 Testing Deployment
280
-
281
- ### 1. Health Check
282
-
283
- ```bash
284
- curl https://your-api-url.com/health
285
- ```
286
-
287
- ### 2. Prediction Test
288
-
289
- ```bash
290
- curl -X POST https://your-api-url.com/predict \
291
- -H "Content-Type: application/json" \
292
- -d '{
293
- "transaction": {
294
- "step": 1,
295
- "type": "TRANSFER",
296
- "amount": 5000.00,
297
- "nameOrig": "C123456789",
298
- "oldBalanceOrig": 50000.00,
299
- "newBalanceOrig": 45000.00,
300
- "nameDest": "C234567890",
301
- "oldBalanceDest": 25000.00,
302
- "newBalanceDest": 30000.00
303
- }
304
- }'
305
- ```
306
-
307
- ### 3. Load Testing
308
-
309
- Use `ab` (Apache Bench) or `wrk`:
310
-
311
- ```bash
312
- # Install
313
- brew install wrk # macOS
314
- # or
315
- apt-get install apache2-utils # Linux
316
-
317
- # Test
318
- wrk -t4 -c100 -d30s --script=test.lua https://your-api-url.com/predict
319
- ```
320
-
321
- ## 🐛 Troubleshooting
322
-
323
- ### Model Not Loading
324
-
325
- **Symptoms**: `503 Model not loaded` error
326
-
327
- **Solutions**:
328
- 1. Check `MODEL_PATH` environment variable
329
- 2. Verify model file exists in container
330
- 3. Check file permissions
331
- 4. Review container logs
332
-
333
- ### Out of Memory
334
-
335
- **Symptoms**: Container crashes, OOM errors
336
-
337
- **Solutions**:
338
- 1. Increase container memory limit
339
- 2. Use model quantization
340
- 3. Implement model caching
341
- 4. Scale horizontally
342
-
343
- ### Slow Response Times
344
-
345
- **Symptoms**: High latency
346
-
347
- **Solutions**:
348
- 1. Optimize model (quantization, pruning)
349
- 2. Use faster hardware (GPU if available)
350
- 3. Implement caching
351
- 4. Optimize feature engineering
352
-
353
- ### CORS Errors
354
-
355
- **Symptoms**: Frontend can't call API
356
-
357
- **Solutions**:
358
- 1. Configure CORS in `main.py`
359
- 2. Add frontend domain to allowed origins
360
- 3. Check platform CORS settings
361
-
362
- ## 📈 Scaling
363
-
364
- ### Horizontal Scaling
365
-
366
- - Use load balancer
367
- - Deploy multiple instances
368
- - Use platform auto-scaling
369
-
370
- ### Vertical Scaling
371
-
372
- - Increase container resources
373
- - Use faster CPU/GPU
374
- - Optimize model
375
-
376
- ## ✅ Deployment Checklist
377
-
378
- - [ ] Model file accessible
379
- - [ ] Environment variables set
380
- - [ ] Health check working
381
- - [ ] API endpoints responding
382
- - [ ] CORS configured
383
- - [ ] Authentication enabled (production)
384
- - [ ] Rate limiting configured
385
- - [ ] Monitoring set up
386
- - [ ] Logs accessible
387
- - [ ] HTTPS enabled
388
- - [ ] Backup strategy in place
389
-
390
- ## 🚀 Quick Deploy Commands
391
-
392
- ### Render
393
-
394
- ```bash
395
- # Using Render CLI
396
- render deploy
397
- ```
398
-
399
- ### Railway
400
-
401
- ```bash
402
- # Using Railway CLI
403
- railway up
404
- ```
405
-
406
- ### Docker Hub + Any Platform
407
-
408
- ```bash
409
- # Build and push
410
- docker build -t your-username/clovershield-ml-api .
411
- docker push your-username/clovershield-ml-api
412
-
413
- # Deploy to platform using image
414
- ```
415
-
416
- ---
417
-
418
- **Status**: Ready for deployment! 🚀
419
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
HUGGINGFACE_DEPLOYMENT.md DELETED
@@ -1,123 +0,0 @@
1
- # Deploying CloverShield ML API to Hugging Face Spaces
2
-
3
- This guide outlines the steps to migrate the backend ML API from Render to Hugging Face Spaces using the Docker SDK.
4
-
5
- ## Prerequisites
6
-
7
- - A [Hugging Face account](https://huggingface.co/join).
8
- - The `ml-api` folder ready with the updated `Dockerfile` and `README.md`.
9
-
10
- ## Step 1: Create a New Space
11
-
12
- 1. Log in to your Hugging Face account.
13
- 2. Click on your profile picture in the top right and select **"New Space"**.
14
- 3. Fill in the details:
15
- - **Space Name:** `clovershield-ml-api` (or similar).
16
- - **License:** MIT or Apache 2.0 (matches your project).
17
- - **Space SDK:** Select **Docker**.
18
- - **Space Hardware:** **CPU Basic (Free)** is usually sufficient for this model size (2MB). If performance is slow, you can upgrade later.
19
- - **Visibility:** **Public** or **Private** (Private is safer for internal testing, but you might need a token to access it from the frontend). For the hackathon demo, **Public** is easiest but be careful with keys. **Private** is recommended if you can handle authentication headers in your frontend.
20
-
21
- 4. Click **"Create Space"**.
22
-
23
- ## Step 2: Upload Files
24
-
25
- You have two options to upload the code:
26
-
27
- ### Option A: Web Interface (Easiest)
28
-
29
- 1. Once the Space is created, you will see the instructions page. Click on the **"Files"** tab.
30
- 2. Click **"Add file"** > **"Upload files"**.
31
- 3. Drag and drop **ALL** files and folders from your local `ml-api` directory into the upload area.
32
- - **Important:** Ensure you include:
33
- - `Dockerfile`
34
- - `requirements.txt`
35
- - `main.py`
36
- - `inference.py`
37
- - `feature_engineering.py`
38
- - `entrypoint.sh`
39
- - `README.md`
40
- - `Models/fraud_pipeline_final.pkl` (The folder structure `Models/` must be preserved).
41
- - `dataset/test_dataset.csv` (If used).
42
- 4. In the "Commit changes" box, type "Initial commit" and click **"Commit changes to main"**.
43
-
44
- ### Option B: Git Command Line (Manual)
45
-
46
- 1. Clone the Space repository locally.
47
- 2. Copy files and push manually.
48
-
49
- ### Option C: Automatic Sync with GitHub (Recommended)
50
-
51
- This method automatically updates your Space whenever you push code to GitHub.
52
-
53
- 1. **Get a Hugging Face Access Token:**
54
- * Go to [Hugging Face Settings > Access Tokens](https://huggingface.co/settings/tokens).
55
- * Click "New token".
56
- * Name it `GITHUB_ACTION` (or similar).
57
- * **Permissions:** Select **"Write"** access.
58
- * Copy the token (starts with `hf_...`).
59
-
60
- 2. **Add Token to GitHub Secrets:**
61
- * Go to your GitHub repository.
62
- * Navigate to **Settings** > **Secrets and variables** > **Actions**.
63
- * Click **"New repository secret"**.
64
- * **Name:** `HF_TOKEN`
65
- * **Value:** (Paste your Hugging Face token here).
66
- * Click "Add secret".
67
-
68
- 3. **Configure the Workflow:**
69
- * Open the file `.github/workflows/deploy-ml-api.yml` in your project.
70
- * Edit the `env` section to match your details:
71
- ```yaml
72
- HF_SPACE_USERNAME: "your-hf-username"
73
- HF_SPACE_NAME: "clovershield-ml-api"
74
- ```
75
- * Commit and push this change to GitHub.
76
-
77
- Now, whenever you push changes to the `ml-api` folder in GitHub, the Action will run and update your Hugging Face Space automatically.
78
-
79
- ## Step 3: Configure Environment Variables (Secrets)
80
-
81
- Your API needs the `GROQ_API_KEY` to function if LLM explanations are enabled.
82
-
83
- 1. Go to the **"Settings"** tab of your Space.
84
- 2. Scroll down to the **"Variables and secrets"** section.
85
- 3. Click **"New secret"**.
86
- - **Name:** `GROQ_API_KEY`
87
- - **Value:** (Paste your Groq API key here)
88
- 4. (Optional) Add other variables if you changed defaults:
89
- - `MODEL_THRESHOLD`
90
- - `PAGERANK_LIMIT`
91
-
92
- **Note:** Hugging Face automatically sets `PORT` to `7860`. Our updated `Dockerfile` and `entrypoint.sh` are configured to respect this.
93
-
94
- ## Step 4: Build and Verify
95
-
96
- 1. Go to the **"App"** tab.
97
- 2. You will see a "Building" status. Click "Logs" to watch the build process.
98
- 3. Once "Running", you should see the Swagger UI or the JSON response from the root endpoint in the preview window.
99
- 4. Your API URL will be: `https://<your-username>-clovershield-ml-api.hf.space`
100
-
101
- ## Step 5: Update Frontend Connection (Vercel)
102
-
103
- Now that your backend is on HF Spaces, you need to update your Vercel deployment to point to the new URL.
104
-
105
- 1. **Go to Vercel Dashboard:**
106
- * Navigate to your project on [Vercel](https://vercel.com).
107
- * Go to **Settings** > **Environment Variables**.
108
-
109
- 2. **Update the Variable:**
110
- * Find the existing `NEXT_PUBLIC_ML_API_URL` or create a new one.
111
- * **Value:** `https://<your-username>-<your-space-name>.hf.space`
112
- * *Important:* Do NOT include a trailing slash (/) at the end.
113
-
114
- 3. **Redeploy:**
115
- * Go to the **Deployments** tab.
116
- * Click on the three dots (...) of your latest deployment and select **Redeploy**. This is necessary for Vercel to pick up the new environment variable value in the client-side code.
117
-
118
- ## Troubleshooting
119
-
120
- - **Build Fails:** Check the "Logs". Common issues are missing dependencies in `requirements.txt` or file path errors.
121
- - **"403 Forbidden":** If your Space is **Private**, you cannot access it directly from a public frontend without passing an authorization header (Bearer token). For this hackathon/demo, switch the Space Visibility to **Public** in Settings > General.
122
- - **Model Load Error:** Ensure `Models/fraud_pipeline_final.pkl` was actually uploaded. Check the "Files" tab.
123
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
QUICK_START.md DELETED
@@ -1,92 +0,0 @@
1
- # ML API Quick Start
2
-
3
- Quick reference for deploying CloverShield ML Inference API.
4
-
5
- ## ✅ What's Included
6
-
7
- - ✅ FastAPI service with `/predict` endpoint
8
- - ✅ Docker containerization
9
- - ✅ Deployment configs for Render/Railway/Vercel
10
- - ✅ Complete documentation
11
-
12
- ## 🚀 Deploy in 5 Minutes
13
-
14
- ### Option 1: Render.com (Recommended)
15
-
16
- 1. **Sign up** at [render.com](https://render.com)
17
- 2. **Create Web Service** → Connect GitHub repo
18
- 3. **Configure**:
19
- - Environment: Docker
20
- - Build Command: (auto-detected)
21
- - Start Command: (auto-detected)
22
- 4. **Set Environment Variables**:
23
- ```
24
- MODEL_PATH=Models/fraud_pipeline_final.pkl
25
- GROQ_API_KEY=your-key (optional)
26
- ```
27
- 5. **Deploy** → Get URL: `https://your-app.onrender.com`
28
-
29
- ### Option 2: Railway.app
30
-
31
- 1. **Sign up** at [railway.app](https://railway.app)
32
- 2. **New Project** → Connect GitHub
33
- 3. **Railway auto-detects** `railway.json`
34
- 4. **Set environment variables** in dashboard
35
- 5. **Deploy** → Get URL automatically
36
-
37
- ### Option 3: Local Docker
38
-
39
- ```bash
40
- # Build
41
- docker build -t clovershield-ml-api .
42
-
43
- # Run
44
- docker run -p 8000:8000 \
45
- -v $(pwd)/Models:/app/Models \
46
- clovershield-ml-api
47
-
48
- # Test
49
- curl http://localhost:8000/health
50
- ```
51
-
52
- ## 📝 API Usage
53
-
54
- ### Health Check
55
- ```bash
56
- curl https://your-api-url.com/health
57
- ```
58
-
59
- ### Predict Fraud
60
- ```bash
61
- curl -X POST https://your-api-url.com/predict \
62
- -H "Content-Type: application/json" \
63
- -d '{
64
- "transaction": {
65
- "step": 1,
66
- "type": "TRANSFER",
67
- "amount": 5000.00,
68
- "nameOrig": "C123456789",
69
- "oldBalanceOrig": 50000.00,
70
- "newBalanceOrig": 45000.00,
71
- "nameDest": "C234567890",
72
- "oldBalanceDest": 25000.00,
73
- "newBalanceDest": 30000.00
74
- }
75
- }'
76
- ```
77
-
78
- ## 📚 Full Documentation
79
-
80
- - [README.md](./README.md) - Complete API documentation
81
- - [DEPLOYMENT_GUIDE.md](./DEPLOYMENT_GUIDE.md) - Detailed deployment guide
82
-
83
- ## ✅ Checklist
84
-
85
- - [ ] Model file available (`Models/fraud_pipeline_final.pkl`)
86
- - [ ] Environment variables configured
87
- - [ ] API deployed and accessible
88
- - [ ] Health check passing
89
- - [ ] Test prediction working
90
-
91
- **Ready to deploy!** 🚀
92
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
README.md DELETED
@@ -1,310 +0,0 @@
1
- ---
2
- title: CloverShield ML API
3
- emoji: 🛡️
4
- colorFrom: green
5
- colorTo: blue
6
- sdk: docker
7
- app_port: 7860
8
- ---
9
-
10
- # CloverShield ML Inference API
11
-
12
- FastAPI microservice for fraud detection predictions. This service replaces the local model loading mechanism in the Streamlit app.
13
-
14
- ## 🚀 Quick Start
15
-
16
- ### Local Development
17
-
18
- 1. **Install dependencies:**
19
- ```bash
20
- pip install -r requirements.txt
21
- ```
22
-
23
- 2. **Copy model file:**
24
- ```bash
25
- cp ../Models/fraud_pipeline_final.pkl Models/
26
- ```
27
-
28
- 3. **Set environment variables:**
29
- ```bash
30
- cp env.template .env
31
- # Edit .env with your values
32
- ```
33
-
34
- 4. **Run the API:**
35
- ```bash
36
- python main.py
37
- # Or
38
- uvicorn main:app --reload
39
- ```
40
-
41
- 5. **Test the API:**
42
- ```bash
43
- curl http://localhost:8000/health
44
- ```
45
-
46
- ### Docker
47
-
48
- 1. **Build the image:**
49
- ```bash
50
- docker build -t clovershield-ml-api .
51
- ```
52
-
53
- 2. **Run the container:**
54
- ```bash
55
- docker run -p 8000:8000 \
56
- -v $(pwd)/Models:/app/Models \
57
- -e GROQ_API_KEY=your-key \
58
- clovershield-ml-api
59
- ```
60
-
61
- ## 📚 API Endpoints
62
-
63
- ### `GET /`
64
- Root endpoint - service information
65
-
66
- ### `GET /health`
67
- Health check endpoint
68
-
69
- **Response:**
70
- ```json
71
- {
72
- "status": "healthy",
73
- "model_loaded": true,
74
- "model_version": "1.0.0",
75
- "shap_available": true,
76
- "llm_available": true
77
- }
78
- ```
79
-
80
- ### `GET /model/info`
81
- Get model metadata
82
-
83
- ### `POST /predict`
84
- Predict fraud probability for a single transaction
85
-
86
- **Request:**
87
- ```json
88
- {
89
- "transaction": {
90
- "step": 1,
91
- "type": "CASH_OUT",
92
- "amount": 5000.00,
93
- "nameOrig": "C123456789",
94
- "oldBalanceOrig": 50000.00,
95
- "newBalanceOrig": 45000.00,
96
- "nameDest": "C234567890",
97
- "oldBalanceDest": 25000.00,
98
- "newBalanceDest": 30000.00
99
- },
100
- "options": {
101
- "include_shap": true,
102
- "include_llm_explanation": false,
103
- "language": "en",
104
- "topk": 10
105
- }
106
- }
107
- ```
108
-
109
- **Response:**
110
- ```json
111
- {
112
- "transaction_id": "550e8400-e29b-41d4-a716-446655440000",
113
- "prediction": {
114
- "fraud_probability": 0.2345,
115
- "decision": "pass",
116
- "risk_level": "low",
117
- "confidence": 0.85
118
- },
119
- "shap_explanations": [
120
- {
121
- "feature": "amount_over_oldBalanceOrig",
122
- "value": 0.1,
123
- "shap": 0.0234,
124
- "shap_abs": 0.0234,
125
- "rank": 1
126
- }
127
- ],
128
- "llm_explanation": null,
129
- "processing_time_ms": 145,
130
- "model_version": "1.0.0",
131
- "timestamp": "2024-01-15T10:30:00Z"
132
- }
133
- ```
134
-
135
- ### `POST /predict/batch`
136
- Predict fraud probability for multiple transactions
137
-
138
- ## 🐳 Docker Deployment
139
-
140
- ### Build and Push to Registry
141
-
142
- ```bash
143
- # Build
144
- docker build -t clovershield-ml-api:latest .
145
-
146
- # Tag for registry
147
- docker tag clovershield-ml-api:latest your-registry/clovershield-ml-api:latest
148
-
149
- # Push
150
- docker push your-registry/clovershield-ml-api:latest
151
- ```
152
-
153
- ## ☁️ Platform Deployment
154
-
155
- ### Hugging Face Spaces (Recommended)
156
-
157
- 1. Create a new Space on Hugging Face.
158
- 2. Select "Docker" as the SDK.
159
- 3. Upload the contents of this directory to the Space.
160
- 4. Set the `GROQ_API_KEY` secret in the Space settings.
161
-
162
-
163
- ### Vercel
164
-
165
- 1. Install Vercel CLI: `npm i -g vercel`
166
- 2. Deploy: `vercel`
167
- 3. Set environment variables in Vercel dashboard
168
-
169
- **Note:** Vercel has limitations for ML models. Consider Render or Railway for better performance.
170
-
171
- ### Render
172
-
173
- 1. Connect your GitHub repository
174
- 2. Create new Web Service
175
- 3. Use `render.yaml` configuration
176
- 4. Set environment variables in dashboard
177
- 5. Deploy
178
-
179
- ### Railway
180
-
181
- 1. Connect your GitHub repository
182
- 2. Create new project
183
- 3. Railway will auto-detect `railway.json`
184
- 4. Set environment variables
185
- 5. Deploy
186
-
187
- ## 🔧 Configuration
188
-
189
- ### Environment Variables
190
-
191
- - `MODEL_PATH` - Path to model file (default: `Models/fraud_pipeline_final.pkl`)
192
- - `TEST_DATASET_PATH` - Path to test dataset CSV for feature engineering (optional, auto-detected if not provided)
193
- - `MODEL_THRESHOLD` - Decision threshold (default: `0.00754482` - optimized for 99% recall)
194
- - `PORT` - Server port (default: `8000`)
195
- - `HOST` - Server host (default: `0.0.0.0`)
196
- - `GROQ_API_KEY` - Optional Groq API key for LLM explanations
197
- - `PAGERANK_LIMIT` - Optional limit on nodes for PageRank computation (for memory optimization)
198
- - `MAX_FIT_ROWS` - Maximum rows to use for feature engineering fitting (default: `50000`)
199
-
200
- ### Model File
201
-
202
- The model file (`fraud_pipeline_final.pkl`) must be available at runtime. Options:
203
-
204
- 1. **Include in Docker image** (not recommended for large files)
205
- 2. **Mount as volume** (Docker)
206
- 3. **Download from cloud storage** (S3, GCS, etc.)
207
- 4. **Use environment variable** to specify path
208
-
209
- ## 📊 API Documentation
210
-
211
- Interactive API documentation available at:
212
- - Swagger UI: `http://localhost:8000/docs`
213
- - ReDoc: `http://localhost:8000/redoc`
214
-
215
- ## 🧪 Testing
216
-
217
- ### Test with curl
218
-
219
- ```bash
220
- # Health check
221
- curl http://localhost:8000/health
222
-
223
- # Prediction
224
- curl -X POST http://localhost:8000/predict \
225
- -H "Content-Type: application/json" \
226
- -d '{
227
- "transaction": {
228
- "step": 1,
229
- "type": "TRANSFER",
230
- "amount": 5000.00,
231
- "nameOrig": "C123456789",
232
- "oldBalanceOrig": 50000.00,
233
- "newBalanceOrig": 45000.00,
234
- "nameDest": "C234567890",
235
- "oldBalanceDest": 25000.00,
236
- "newBalanceDest": 30000.00
237
- }
238
- }'
239
- ```
240
-
241
- ### Test with Python
242
-
243
- ```python
244
- import requests
245
-
246
- response = requests.post(
247
- "http://localhost:8000/predict",
248
- json={
249
- "transaction": {
250
- "step": 1,
251
- "type": "TRANSFER",
252
- "amount": 5000.00,
253
- "nameOrig": "C123456789",
254
- "oldBalanceOrig": 50000.00,
255
- "newBalanceOrig": 45000.00,
256
- "nameDest": "C234567890",
257
- "oldBalanceDest": 25000.00,
258
- "newBalanceDest": 30000.00
259
- }
260
- }
261
- )
262
-
263
- print(response.json())
264
- ```
265
-
266
- ## 🔒 Security
267
-
268
- **Production Recommendations:**
269
- - Add API key authentication for production
270
- - Use HTTPS in production
271
- - Implement rate limiting
272
- - Validate all inputs
273
- - Monitor for abuse
274
- - Restrict CORS origins (currently set to `["*"` for development)
275
- - Use environment variables for sensitive data
276
-
277
- ## 📈 Performance
278
-
279
- **Optimization Tips:**
280
- - Model loads once on startup (lazy loading supported for serverless)
281
- - Single worker recommended (ML models are not thread-safe)
282
- - Consider caching for frequently accessed data
283
- - Monitor memory usage (XGBoost models can be large)
284
- - Feature engineering uses sampled dataset (configurable via `MAX_FIT_ROWS`)
285
- - SHAP background uses minimal sample size (100 rows) for efficiency
286
-
287
- ## 🐛 Troubleshooting
288
-
289
- ### Model not loading
290
-
291
- - Check `MODEL_PATH` environment variable
292
- - Verify model file exists
293
- - Check file permissions
294
- - Review logs for error messages
295
-
296
- ### SHAP not working
297
-
298
- - Ensure XGBoost is installed
299
- - Check model type compatibility
300
- - Review SHAP initialization logs
301
-
302
- ### LLM explanations failing
303
-
304
- - Verify `GROQ_API_KEY` is set
305
- - Check API key validity
306
- - Review Groq API status
307
-
308
- ## 📝 License
309
-
310
- See main project LICENSE file.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
railway.json DELETED
@@ -1,13 +0,0 @@
1
- {
2
- "$schema": "https://railway.app/railway.schema.json",
3
- "build": {
4
- "builder": "DOCKERFILE",
5
- "dockerfilePath": "Dockerfile"
6
- },
7
- "deploy": {
8
- "startCommand": "uvicorn main:app --host 0.0.0.0 --port $PORT",
9
- "restartPolicyType": "ON_FAILURE",
10
- "restartPolicyMaxRetries": 10
11
- }
12
- }
13
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
render.yaml DELETED
@@ -1,26 +0,0 @@
1
- # Render.com deployment configuration for CloverShield ML API
2
-
3
- services:
4
- - type: web
5
- name: clovershield-ml-api
6
- runtime: docker
7
- dockerfilePath: ./Dockerfile
8
- dockerContext: .
9
- envVars:
10
- - key: MODEL_PATH
11
- value: Models/fraud_pipeline_final.pkl
12
- - key: MODEL_THRESHOLD
13
- value: 0.0793
14
- - key: PORT
15
- value: 8000
16
- - key: HOST
17
- value: 0.0.0.0
18
- - key: MAX_FIT_ROWS
19
- value: "30000" # Reduced for Render free tier (512MB RAM)
20
- - key: PAGERANK_LIMIT
21
- value: "5000" # Reduced for memory efficiency
22
- - key: GROQ_API_KEY
23
- sync: false # Set in Render dashboard
24
- healthCheckPath: /health
25
- plan: starter # Upgrade for production
26
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
vercel.json DELETED
@@ -1,20 +0,0 @@
1
- {
2
- "version": 2,
3
- "builds": [
4
- {
5
- "src": "main.py",
6
- "use": "@vercel/python"
7
- }
8
- ],
9
- "routes": [
10
- {
11
- "src": "/(.*)",
12
- "dest": "main.py"
13
- }
14
- ],
15
- "env": {
16
- "MODEL_PATH": "Models/fraud_pipeline_final.pkl",
17
- "MODEL_THRESHOLD": "0.0793"
18
- }
19
- }
20
-