---
title: MuleGuard
emoji: 🛡️
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: true
license: mit
short_description: Real-time AI/ML detection of suspicious mule accounts
---

# MuleGuard — AI/ML Detection of Suspicious Mule Accounts

> **BOI Hackathon · Problem Statement 2** — *AI/ML-Based Classification of Suspicious Mule Accounts*
>
> An end-to-end system that ingests account/transaction features, scores every account for mule-risk in real time, explains every alert, and streams alerts to a fraud-analyst console — directly targeting the circulation of fraudulent proceeds through mule accounts.

---

## 1. Problem

Banks face growing cyber-enabled fraud where **mule accounts** receive, transfer, and conceal fraudulent funds across channels. Rule-based monitoring misses evolving patterns. We must **learn behavioral & transactional patterns** from the provided data to **classify suspicious/mule accounts**, with anomaly detection, predictive risk scoring, and intelligent alerts.

**Dataset:** `data/DataSet.csv` — **9,082 accounts × 3,924 anonymized features** (`F1`–`F3924`). **Target = `F3924`**. Severe imbalance: **9,001 legit (0) vs 81 mule (1) → 0.89% positive**. 18 features flagged by the bank as high-signal.

## 2. Our Solution — "MuleGuard"

A supervised **gradient-boosted classifier fused with an unsupervised anomaly score**, wrapped in a real-time scoring service, a live feed simulator, and an analyst dashboard:

```
DataSet.csv ─▶ Feature Pipeline ─▶ Model + Calibrator + SHAP  ─┐
                                                                ├─▶ artifacts/
Feed Simulator ─(streams accounts/tickets)─▶ FastAPI /score ───┘
                                                  │ risk 0-100, tier, reason codes, alert
                                                  ▼
                                       Streamlit Analyst Console
```

**Why this wins:** proven imbalanced-tabular method + novelty detection (catches *evolving* fraud) + full explainability (auditable alerts) + a working real-time demo that maps to the regulatory-feed vision.

## 3. Architecture & Engineering Approach

- **Anti-leakage discipline** — all transforms fit inside CV folds / train only.
- **Honest metrics** — PR-AUC, ROC-AUC, recall@precision, F2 (never accuracy).
- **Repeated stratified CV** — robust estimates despite only 81 positives.
- **Threshold tuning + probability calibration** — meaningful 0–100 risk scores.
- **SHAP reason codes** — every alert is explainable.

Built using a custom `.claude/` toolkit (agents + skills) authored **before** implementation — see §6.

## 4. Repository Layout

```
.claude/         custom agents + skills (incl. vendored superpowers & get-shit-done)
data/            DataSet.csv (gitignored)
src/data/        load, profile, clean
src/features/    selection + engineering pipeline
src/models/      train, evaluate, calibrate, explain
src/api/         FastAPI scoring service
src/simulator/   real-time feed simulator
dashboard/       Streamlit analyst console
artifacts/       trained model, calibrator, threshold, feature list, SHAP explainer
reports/         data profile, feature selection, model card, figures
docs/            architecture diagrams + spec
deck/            pitch deck (.pptx)
```

## 5. How to Run

```bash
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/python -m src.data.profile          # 1. profile data       -> reports/data_profile.md
.venv/bin/python -m src.features.build         # 2. build features     -> artifacts/
.venv/bin/python -m src.models.train           # 3. train + evaluate   -> artifacts/, reports/model_report.md
.venv/bin/python -m src.models.explain         # 4. SHAP explainability -> artifacts/, reports/figures/
.venv/bin/uvicorn src.api.main:app --port 8000 # 5. scoring API
.venv/bin/python -m src.simulator.feed          # 6. stream live alerts
.venv/bin/streamlit run dashboard/app.py        # 7. analyst console
```

## 6. Custom `.claude/` Toolkit (built first, then used)

**Agents** (`.claude/agents/`): `data-profiler`, `feature-engineer`, `ml-trainer`, `explainability-analyst`, `fraud-domain-expert`, `demo-builder`.

**Skills** (`.claude/skills/`): `imbalanced-fraud-classification`, `fraud-feature-engineering` (authored), plus **`superpowers`** and **`get-shit-done`** vendored from GitHub as-is.

---

## 7. Progress Tracker

| # | Phase | Status |
|---|-------|--------|
| 0 | Scaffold: structure, venv, deps, `.claude/` agents & skills | ✅ Done |
| 1 | Data profiling (`reports/data_profile.md`) | ✅ Done |
| 2 | Feature selection + engineering pipeline (103 features) | ✅ Done |
| 3 | Model training, CV, calibration, threshold | ✅ Done |
| 4 | SHAP explainability | ✅ Done |
| 5 | FastAPI scoring service | ✅ Done |
| 6 | Real-time feed simulator | ✅ Done |
| 7 | Streamlit analyst dashboard | ✅ Done |
| 8 | Architecture diagrams (`docs/`) | ✅ Done |
| 9 | Pitch deck (`deck/MuleGuard_Pitch.pptx`) | ✅ Done |

## 8. Results

**Model:** Calibrated LightGBM fused with an Isolation Forest anomaly score · **103 features** selected from 3,924.

### Leakage defense (the rigor that makes the numbers trustworthy)
Two label-leaking features were detected and **removed** — without this, the model "achieved" a meaningless 100%:
- **`F3912`** — a binary fraud-flag aligned ~perfectly with the label (single-feature AUC **0.987**).
- **`F2230`** — observation month; all legit accounts in Oct-25, all mules in Sep/Nov/Dec-25 (temporal artifact, AUC **1.000**).

### Headline metrics

| Metric | Value |
|---|---|
| **Cross-validated PR-AUC** (repeated stratified) | **0.882 ± 0.071** |
| Cross-validated ROC-AUC | 0.986 ± 0.012 |
| Held-out test PR-AUC | 0.956 |
| Held-out test **recall** | **0.81** (caught 13/16 mules) |
| Held-out test **precision** | **0.87** (only 2 false positives in 1,801 legit) |
| Test F2 | 0.82 |
| Alert rate | **0.8%** of accounts |
| Logistic baseline PR-AUC (sanity) | 0.389 → the large gap proves genuine non-linear signal, **not** leakage |

**Why this is a strong result:** 81% of mules caught at an **0.8% alert rate** is a queue a real fraud team can work. The fused anomaly score was selected in **every** CV fold, addressing the "evolving fraud patterns" requirement. Every alert ships with SHAP reason codes.

See `reports/model_report.md` (model card), `reports/feature_selection.md`, and `reports/data_profile.md` for full detail.

---

*Generated for the BOI Hackathon. Model decisions are explainable and auditable; capabilities demonstrated on the provided dataset, with a clear productionization roadmap to live regulatory/cross-channel feeds.*