Spaces:
Sleeping
Sleeping
File size: 5,903 Bytes
94c7d2e 8a08300 94c7d2e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 |
---
title: PayShield-ML
emoji: ⚡
colorFrom: yellow
colorTo: red
sdk: docker
pinned: true
app_port: 7860
license: mit
thumbnail: >-
https://cdn-uploads.huggingface.co/production/uploads/664345bd8b3a005a73e3e430/cRvhV8zNVwjQuHvp42boL.jpeg
---
# 🛡️ PayShield-ML:Real-Time Fraud Engine
**A production-grade MLOps system providing low-latency fraud detection with stateful feature hydration and human-in-the-loop explainability.**
---
## 🏗️ System Architecture
The system follows a **Lambda Architecture** pattern, decoupling the high-latency model training from the low-latency inference path.
```mermaid
graph TD
User[Payment Gateway] -->|POST /predict| API[FastAPI Inference Service]
subgraph "Online Serving Layer (<50ms)"
API -->|1. Hydrate Velocity| Redis[(Redis Feature Store)]
API -->|2. Preprocess| Pipeline[Scikit-Learn Pipeline]
Pipeline -->|3. Predict| Model[XGBoost Classifier]
Model -->|4. Explain| SHAP[SHAP Engine]
end
subgraph "Offline Training Layer"
Data[(Transactional Data)] -->|Batch Load| Trainer[Training Pipeline]
Trainer -->|CV & Tuning| MLflow[MLflow Tracking]
Trainer -->|Export Artifact| Registry[Model Registry]
end
API -->|Async Log| ShadowLogs[Shadow Mode Logs]
API -->|Visualize| Dashboard[Analyst Dashboard]
```
### 🖥️ Analyst Workbench
*Real-time interface for fraud analysts to review blocked transactions and SHAP explanations.*
---
## 🚀 Key Features
### 1. Stateful Feature Store (Redis)
Traditional stateless APIs struggle with "Velocity Features" (e.g., *how many times did this user swipe in 24 hours?*). Our engine utilizes **Redis Sorted Sets (ZSET)** to maintain rolling windows, allowing feature hydration in **<2ms** with $O(\log N)$ complexity.
### 2. Shadow Mode (Dark Launch)
To mitigate the risk of model drift or false positives, the system supports a **Shadow Mode** configuration. The model runs in production, receives real traffic, and logs decisions, but never blocks a transaction. This allows for risk-free A/B testing against legacy rule engines.
### 3. Business-Aligned Optimization
Standard accuracy is misleading in fraud detection due to class imbalance. We implement a **Recall-Constraint Strategy** (Target: 80% Recall) ensuring the model captures the vast majority of fraud while maintaining a strict upper bound on False Positive Rates, as required by financial compliance.
### 4. Cold-Start Handling
Engineered logic to handle new users with zero history by defaulting to global medians and "warm-up" priors, preventing the system from unfairly blocking legitimate first-time customers.
---
## 📊 Performance & Metrics
The following metrics were achieved on a hold-out temporal test set (out-of-time validation):
| Metric | Result | Target / Bench |
| :--- | :--- | :--- |
| **PR-AUC** | 0.9245 | Excellent |
| **Precision** | 93.18% | Low False Positives |
| **Recall** | 80.06% | Target: 80% |
| **Inference Latency (p95)** | ~30ms | < 50ms |
---
## 🛠️ Quick Start
### Prerequisites
- Docker & Docker Compose
- (Optional) Python 3.12+ (managed via `uv`)
### Installation & Deployment
1. **Clone the Repository**
```bash
git clone https://github.com/Sibikrish3000/realtime-fraud-engine.git
cd realtime-fraud-engine
```
2. **Launch the Stack**
```bash
docker-compose up --build
```
*This starts the FastAPI Backend, Redis Feature Store, and Streamlit Dashboard.*
3. **Access the Services**
- **Analyst Dashboard:** [http://localhost:8501](http://localhost:8501)
- **API Documentation:** [http://localhost:8000/docs](http://localhost:8000/docs)
- **Redis Instance:** `localhost:6379`
---
## 📂 Project Structure
```text
src/
├── api/ # FastAPI service, schemas (Pydantic), and config
├── features/ # Redis feature store logic and sliding window constants
├── models/ # Training pipelines, metrics calculation, and XGBoost wrappers
├── frontend/ # Streamlit-based analyst workbench
├── data/ # Data ingestion and cleaning utilities
└── explainability.py # SHAP-based waterfall plots and global importance
```
See the [Development & MLOps Guide](docs/DEVELOPMENT.md) for detailed instructions on training and local development.
---
## 📡 API Reference
### Predict Transaction
`POST /v1/predict`
**Request:**
```bash
curl -X 'POST' \
'http://localhost:8000/v1/predict' \
-H 'Content-Type: application/json' \
-d '{
"user_id": "u12345",
"trans_date_trans_time": "2024-01-20 14:30:00",
"amt": 150.75,
"lat": 40.7128,
"long": -74.0060,
"merch_lat": 40.7306,
"merch_long": -73.9352,
"category": "grocery_pos"
}'
```
**Response:**
```json
{
"decision": "APPROVE",
"probability": 0.12,
"risk_score": 12.0,
"latency_ms": 28.5,
"shadow_mode": false,
"shap_values": {
"amt": 0.05,
"dist": 0.02
}
}
```
---
## 🗺️ Future Roadmap
- [ ] **Kafka Integration:** Transition to an asynchronous event-driven architecture for high-throughput stream processing.
- [ ] **KServe Deployment:** Migrate from standalone FastAPI to KServe for automated scaling and model versioning.
- [ ] **Graph Features:** Incorporate Neo4j-based features to detect fraud rings and synthetic identities.
---
**Author:** [Sibi Krishnamoorthy](https://github.com/Sibikrish3000)
*Machine Learning Engineer | Fintech & Risk Analytics* |