Spaces:
Paused
Paused
Commit ·
ffea7f4
1
Parent(s): aa55f7e
Iteration_4: expand task bank 20→30, add variants for all tasks, fix baseline policy for chain-gated CRIT tasks
Browse files- README.md +20 -10
- openenv.yaml +1 -1
- run_tests.sh +79 -29
- scripts_util.py +9 -0
- tasks.py +855 -11
- validate.py +1 -1
README.md
CHANGED
|
@@ -35,7 +35,7 @@ Authorised Push Payment (APP) scams.
|
|
| 35 |
|
| 36 |
## Environment Description
|
| 37 |
|
| 38 |
-
Each **episode** steps through all **
|
| 39 |
For each transaction the agent observes a rich set of signals and chooses one
|
| 40 |
of **10 possible actions** — 5 terminal decisions and 5 investigation sub-actions.
|
| 41 |
A reward is returned immediately, and the next transaction is presented until
|
|
@@ -121,8 +121,10 @@ Investigation sub-actions (with budget cost) reveal more information and let the
|
|
| 121 |
| EASY-002 | Textbook fraud: unknown sender, offshore, sanctioned country, risk=0.97 | `reject` |
|
| 122 |
| EASY-003 | Standard refund to verified customer; tiny amount, no flags | `approve` |
|
| 123 |
| EASY-004 | ATM withdrawal burst — 15 withdrawals in 58 minutes | `flag` |
|
|
|
|
|
|
|
| 124 |
|
| 125 |
-
### Medium (
|
| 126 |
|
| 127 |
| ID | Description | Correct Action |
|
| 128 |
|---------|-------------|----------------|
|
|
@@ -132,8 +134,10 @@ Investigation sub-actions (with budget cost) reveal more information and let the
|
|
| 132 |
| MED-004 | Payment to licensed crypto exchange from verified personal account | `flag` |
|
| 133 |
| MED-005 | Expired KYC on high-frequency corporate payroll account; KYC lapsed 12 days ago | `hold` |
|
| 134 |
| MED-006 | Real estate advance payment; large first-time transfer to new receiver but signed contract exists | `escalate` |
|
|
|
|
|
|
|
| 135 |
|
| 136 |
-
### Hard (
|
| 137 |
|
| 138 |
| ID | Description | Correct Action |
|
| 139 |
|----------|-------------|----------------|
|
|
@@ -142,16 +146,22 @@ Investigation sub-actions (with budget cost) reveal more information and let the
|
|
| 142 |
| HARD-003 | Structuring / smurfing: just-below-CTR-threshold payments, same UBO | `reject` |
|
| 143 |
| HARD-004 | Legitimate FX correspondent banking settlement — looks alarming, is not | `approve` |
|
| 144 |
| HARD-005 | Insider threat: employee initiating transfers to personal family accounts | `escalate` |
|
| 145 |
-
| HARD-006 | Ghost account: dormant 5 years, suddenly received 20 inbound transfers this week
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
|
| 147 |
-
### Critical (
|
| 148 |
|
| 149 |
| ID | Description | Correct Action |
|
| 150 |
|----------|-------------|----------------|
|
| 151 |
-
| CRIT-001 | Multi-step chain: large wire to new counterparty;
|
| 152 |
| CRIT-002 | Fraud ring: coordinated small payments from 3 related accounts aggregating above reporting threshold; SAR required | `reject` |
|
| 153 |
-
| CRIT-003 | Trade-based money laundering: over-invoiced international trade payment (4× market price)
|
| 154 |
| CRIT-004 | Compromised corporate account: geo-impossible login (NY → Lagos in 8 min); confirmed account takeover | `reject` |
|
|
|
|
|
|
|
| 155 |
|
| 156 |
---
|
| 157 |
|
|
@@ -265,7 +275,7 @@ print(f"Episode score: {score['normalised_score']:.4f}")
|
|
| 265 |
|
| 266 |
The rule-based baseline uses a deterministic priority-ordered policy in `scripts_util.py`.
|
| 267 |
|
| 268 |
-
| Metric | Rule-based baseline (v2,
|
| 269 |
|--------|------------------------------------|
|
| 270 |
| Normalised score | 0.68–0.76 |
|
| 271 |
| Passed (≥ 0.5) | Yes |
|
|
@@ -410,7 +420,7 @@ PYTHONPATH=$(pwd) python payops_env/inference.py
|
|
| 410 |
payops_env/
|
| 411 |
├── models.py # PayOpsAction, PayOpsObservation, PayOpsState (Pydantic)
|
| 412 |
├── environment.py # PayOpsEnvironment — reset_async / step_async / state
|
| 413 |
-
├── tasks.py #
|
| 414 |
├── grader.py # Partial-credit reward function + episode grader
|
| 415 |
├── scripts_util.py # Baseline runner helper (used by /baseline endpoint)
|
| 416 |
├── server/
|
|
@@ -430,7 +440,7 @@ payops_env/
|
|
| 430 |
| Criterion | Implementation |
|
| 431 |
|-----------|---------------|
|
| 432 |
| Real-world utility | Payment fraud and compliance triage — deployed daily by fintech ops teams worldwide |
|
| 433 |
-
| Task & grader quality |
|
| 434 |
| Environment design | 30-field observation space; 10-action space (5 terminal + 5 investigation); budget mechanic; episode state tracking |
|
| 435 |
| Code quality & spec compliance | Pydantic v2 models; async API; all 11 required endpoints; openenv.yaml v2; Dockerfile; validate.py |
|
| 436 |
| Creativity & novelty | Adversarial model-poisoning task; APP scam; AML structuring with SAR requirement; PEP detection |
|
|
|
|
| 35 |
|
| 36 |
## Environment Description
|
| 37 |
|
| 38 |
+
Each **episode** steps through all **30 transactions** (6 easy, 8 medium, 10 hard, 6 critical).
|
| 39 |
For each transaction the agent observes a rich set of signals and chooses one
|
| 40 |
of **10 possible actions** — 5 terminal decisions and 5 investigation sub-actions.
|
| 41 |
A reward is returned immediately, and the next transaction is presented until
|
|
|
|
| 121 |
| EASY-002 | Textbook fraud: unknown sender, offshore, sanctioned country, risk=0.97 | `reject` |
|
| 122 |
| EASY-003 | Standard refund to verified customer; tiny amount, no flags | `approve` |
|
| 123 |
| EASY-004 | ATM withdrawal burst — 15 withdrawals in 58 minutes | `flag` |
|
| 124 |
+
| EASY-005 | Scheduled monthly mortgage repayment; regular amount, verified borrower | `approve` |
|
| 125 |
+
| EASY-006 | Suspected duplicate payment: same sender/receiver/amount submitted twice in 4 minutes | `flag` |
|
| 126 |
|
| 127 |
+
### Medium (8 tasks — ambiguous, multi-signal reasoning required)
|
| 128 |
|
| 129 |
| ID | Description | Correct Action |
|
| 130 |
|---------|-------------|----------------|
|
|
|
|
| 134 |
| MED-004 | Payment to licensed crypto exchange from verified personal account | `flag` |
|
| 135 |
| MED-005 | Expired KYC on high-frequency corporate payroll account; KYC lapsed 12 days ago | `hold` |
|
| 136 |
| MED-006 | Real estate advance payment; large first-time transfer to new receiver but signed contract exists | `escalate` |
|
| 137 |
+
| MED-007 | Supplier emails to say bank details have changed; first payment to new account matches large invoice (BEC indicator) | `hold` |
|
| 138 |
+
| MED-008 | Buy Now Pay Later high-value purchase; new account, thin credit file, elevated risk signals | `flag` |
|
| 139 |
|
| 140 |
+
### Hard (10 tasks — adversarial / edge-case)
|
| 141 |
|
| 142 |
| ID | Description | Correct Action |
|
| 143 |
|----------|-------------|----------------|
|
|
|
|
| 146 |
| HARD-003 | Structuring / smurfing: just-below-CTR-threshold payments, same UBO | `reject` |
|
| 147 |
| HARD-004 | Legitimate FX correspondent banking settlement — looks alarming, is not | `approve` |
|
| 148 |
| HARD-005 | Insider threat: employee initiating transfers to personal family accounts | `escalate` |
|
| 149 |
+
| HARD-006 | Ghost account: dormant 5 years, suddenly received 20 inbound transfers this week | `flag` |
|
| 150 |
+
| HARD-007 | SIM-swap attack: phone ported 6 hours ago; account now requesting large crypto withdrawal to new address | `reject` |
|
| 151 |
+
| HARD-008 | Romance scam / pig butchering: 4th escalating transfer to overseas 'romantic partner' met online | `reject` |
|
| 152 |
+
| HARD-009 | Synthetic identity fraud: new business account with AI-generated-looking perfect profile | `escalate` |
|
| 153 |
+
| HARD-010 | Payroll diversion: HR system breach rerouted employee salary to newly added account | `reject` |
|
| 154 |
|
| 155 |
+
### Critical (6 tasks — regulatory + multi-step investigation chains)
|
| 156 |
|
| 157 |
| ID | Description | Correct Action |
|
| 158 |
|----------|-------------|----------------|
|
| 159 |
+
| CRIT-001 | Multi-step chain: large PE wire to new counterparty; inspect then request docs before deciding (chain of 3) | `approve` |
|
| 160 |
| CRIT-002 | Fraud ring: coordinated small payments from 3 related accounts aggregating above reporting threshold; SAR required | `reject` |
|
| 161 |
+
| CRIT-003 | Trade-based money laundering: over-invoiced international trade payment (4× market price) | `escalate` |
|
| 162 |
| CRIT-004 | Compromised corporate account: geo-impossible login (NY → Lagos in 8 min); confirmed account takeover | `reject` |
|
| 163 |
+
| CRIT-005 | OFAC sanctions evasion: large USD payment routed through UAE shell chain; UBO is on SDN list (chain of 3) | `reject` |
|
| 164 |
+
| CRIT-006 | Correspondent banking: partner bank added to FinCEN 311 Special Measures list; in-flight payments must be escalated | `escalate` |
|
| 165 |
|
| 166 |
---
|
| 167 |
|
|
|
|
| 275 |
|
| 276 |
The rule-based baseline uses a deterministic priority-ordered policy in `scripts_util.py`.
|
| 277 |
|
| 278 |
+
| Metric | Rule-based baseline (v2, 30 tasks) |
|
| 279 |
|--------|------------------------------------|
|
| 280 |
| Normalised score | 0.68–0.76 |
|
| 281 |
| Passed (≥ 0.5) | Yes |
|
|
|
|
| 420 |
payops_env/
|
| 421 |
├── models.py # PayOpsAction, PayOpsObservation, PayOpsState (Pydantic)
|
| 422 |
├── environment.py # PayOpsEnvironment — reset_async / step_async / state
|
| 423 |
+
├── tasks.py # 30 tasks (EASY×6, MED×8, HARD×10, CRIT×6) with ground-truth labels
|
| 424 |
├── grader.py # Partial-credit reward function + episode grader
|
| 425 |
├── scripts_util.py # Baseline runner helper (used by /baseline endpoint)
|
| 426 |
├── server/
|
|
|
|
| 440 |
| Criterion | Implementation |
|
| 441 |
|-----------|---------------|
|
| 442 |
| Real-world utility | Payment fraud and compliance triage — deployed daily by fintech ops teams worldwide |
|
| 443 |
+
| Task & grader quality | 30 tasks across 4 difficulty tiers (easy→critical); partial-credit grader; clear pass/fail |
|
| 444 |
| Environment design | 30-field observation space; 10-action space (5 terminal + 5 investigation); budget mechanic; episode state tracking |
|
| 445 |
| Code quality & spec compliance | Pydantic v2 models; async API; all 11 required endpoints; openenv.yaml v2; Dockerfile; validate.py |
|
| 446 |
| Creativity & novelty | Adversarial model-poisoning task; APP scam; AML structuring with SAR requirement; PEP detection |
|
openenv.yaml
CHANGED
|
@@ -114,7 +114,7 @@ observation_space:
|
|
| 114 |
|
| 115 |
# ── Tasks ─────────────────────────────────────────────────────────────────────
|
| 116 |
tasks:
|
| 117 |
-
count:
|
| 118 |
difficulties:
|
| 119 |
- easy
|
| 120 |
- medium
|
|
|
|
| 114 |
|
| 115 |
# ── Tasks ─────────────────────────────────────────────────────────────────────
|
| 116 |
tasks:
|
| 117 |
+
count: 30
|
| 118 |
difficulties:
|
| 119 |
- easy
|
| 120 |
- medium
|
run_tests.sh
CHANGED
|
@@ -97,9 +97,9 @@ section "GROUP B — Tasks endpoint"
|
|
| 97 |
|
| 98 |
TASKS_RESP="$(curl -s $BASE/tasks)"
|
| 99 |
|
| 100 |
-
# B-01
|
| 101 |
-
check "B-01 GET /tasks → count=
|
| 102 |
-
"$TASKS_RESP" '"count":
|
| 103 |
|
| 104 |
# B-02 All 4 difficulty tiers present
|
| 105 |
check "B-02 Tasks include 'easy' tier" "$TASKS_RESP" '"difficulty":"easy"'
|
|
@@ -159,6 +159,16 @@ check "D-04 EASY-004 flag → reward=1.0" \
|
|
| 159 |
"$(step '{"action_type":"flag","transaction_id":"TXN-E004"}')" \
|
| 160 |
'"reward":1.0'
|
| 161 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
# D-05 escalate correct → reward=1.0
|
| 163 |
check "D-05 MED-001 escalate → reward=1.0" \
|
| 164 |
"$(step '{"action_type":"escalate","transaction_id":"TXN-M001"}')" \
|
|
@@ -202,6 +212,8 @@ step '{"action_type":"approve","transaction_id":"TXN-E001"}' > /dev/null
|
|
| 202 |
step '{"action_type":"reject","transaction_id":"TXN-E002"}' > /dev/null
|
| 203 |
step '{"action_type":"approve","transaction_id":"TXN-E003"}' > /dev/null
|
| 204 |
step '{"action_type":"flag","transaction_id":"TXN-E004"}' > /dev/null
|
|
|
|
|
|
|
| 205 |
R=$(step '{"action_type":"flag","transaction_id":"TXN-M001"}')
|
| 206 |
check "E-03 Partial credit: flag on MED-001 (correct=escalate) → reward > 0" \
|
| 207 |
"$R" '"reward":0\.[0-9]'
|
|
@@ -295,11 +307,18 @@ check "H-07 GET /grader → passed field present" "$GRADER" '"passed":'
|
|
| 295 |
# ═══════════════════════════════════════════════════════════
|
| 296 |
section "GROUP I — /replay endpoint (offline eval)"
|
| 297 |
|
| 298 |
-
# I-01 Replay with all correct actions
|
| 299 |
-
|
| 300 |
-
|
| 301 |
-
|
| 302 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 303 |
R=$(curl -s -X POST $BASE/replay \
|
| 304 |
-H "Content-Type: application/json" \
|
| 305 |
-d "$REPLAY_PERFECT")
|
|
@@ -307,11 +326,11 @@ check "I-01 Replay correct terminals (no investigation) → score<1.0 (hard/cri
|
|
| 307 |
check "I-02 Replay → passed=true (score>0.5 despite penalty)" "$R" '"passed":true'
|
| 308 |
check "I-03 Replay → budget_spent=0.0 (no inv actions)" "$R" '"budget_spent":0.0'
|
| 309 |
|
| 310 |
-
# I-04 Replay with investigation actions included
|
| 311 |
-
REPLAY_WITH_INV='{"actions":["inspect","approve","reject","approve","flag",
|
| 312 |
-
"escalate","hold","flag","flag","hold","escalate",
|
| 313 |
-
"escalate","reject","reject","approve","escalate","flag",
|
| 314 |
-
"approve","reject","escalate","reject"]}'
|
| 315 |
R=$(curl -s -X POST $BASE/replay \
|
| 316 |
-H "Content-Type: application/json" \
|
| 317 |
-d "$REPLAY_WITH_INV")
|
|
@@ -370,16 +389,33 @@ def get(path):
|
|
| 370 |
with urllib.request.urlopen(f"{BASE}{path}", context=_ssl) as r: return json.loads(r.read())
|
| 371 |
|
| 372 |
post("/reset", {"seed": 0})
|
| 373 |
-
|
|
|
|
| 374 |
("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
|
|
|
|
| 375 |
("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
|
| 376 |
-
("hold","TXN-M005"),("escalate","TXN-M006"),
|
| 377 |
("escalate","TXN-H001"),("reject","TXN-H002"),("reject","TXN-H003"),("approve","TXN-H004"),
|
| 378 |
-
("escalate","TXN-H005"),("flag","TXN-H006"),
|
| 379 |
-
("
|
| 380 |
-
]
|
| 381 |
-
for action, txn in actions:
|
| 382 |
post("/step", {"action_type": action, "transaction_id": txn})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 383 |
PYEOF
|
| 384 |
|
| 385 |
ANA="$(curl -s $BASE/analytics)"
|
|
@@ -408,7 +444,7 @@ check "L-06 GET /leaderboard → budget_spent in entry" "$LB" '"budget_spent":'
|
|
| 408 |
# ═══════════════════════════════════════════════════════════
|
| 409 |
# GROUP M — Perfect episode score
|
| 410 |
# ═══════════════════════════════════════════════════════════
|
| 411 |
-
section "GROUP M — Perfect episode (all
|
| 412 |
|
| 413 |
reset
|
| 414 |
BASE=$BASE python3 - <<'PYEOF'
|
|
@@ -424,14 +460,15 @@ def post(path, body=None):
|
|
| 424 |
with urllib.request.urlopen(req, context=_ssl) as r: return json.loads(r.read())
|
| 425 |
|
| 426 |
post("/reset", {"seed": 0})
|
| 427 |
-
# Easy +
|
| 428 |
for action, txn in [
|
| 429 |
("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
|
|
|
|
| 430 |
("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
|
| 431 |
-
("hold","TXN-M005"),("escalate","TXN-M006"),
|
| 432 |
]:
|
| 433 |
post("/step", {"action_type": action, "transaction_id": txn})
|
| 434 |
-
# Hard
|
| 435 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-H001"})
|
| 436 |
post("/step", {"action_type": "escalate", "transaction_id": "TXN-H001"})
|
| 437 |
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H002"})
|
|
@@ -445,7 +482,15 @@ post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H005"})
|
|
| 445 |
post("/step", {"action_type": "escalate", "transaction_id": "TXN-H005"})
|
| 446 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-H006"})
|
| 447 |
post("/step", {"action_type": "flag", "transaction_id": "TXN-H006"})
|
| 448 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 449 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C001"})
|
| 450 |
post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C001"})
|
| 451 |
post("/step", {"action_type": "approve", "transaction_id": "TXN-C001"})
|
|
@@ -457,6 +502,11 @@ post("/step", {"action_type": "escalate", "transaction_id": "TXN-C003"})
|
|
| 457 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C004"})
|
| 458 |
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-C004"})
|
| 459 |
post("/step", {"action_type": "reject", "transaction_id": "TXN-C004"})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 460 |
PYEOF
|
| 461 |
|
| 462 |
GRADER="$(curl -s $BASE/grader)"
|
|
@@ -470,16 +520,16 @@ check "M-03 Perfect episode → budget_penalty=0.0" "$GRADER" '"budget_penalty"
|
|
| 470 |
section "GROUP N — Difficulty weighting in grader"
|
| 471 |
|
| 472 |
# A critical task correct should contribute more weight than easy
|
| 473 |
-
REPLAY_CRIT='{"actions":["approve","reject","approve","flag",
|
| 474 |
-
"escalate","hold","flag","flag","hold","escalate",
|
| 475 |
-
"escalate","reject","reject","approve","escalate","flag",
|
| 476 |
-
"approve","reject","escalate","reject"]}'
|
| 477 |
|
| 478 |
FULL=$(curl -s -X POST $BASE/replay \
|
| 479 |
-H "Content-Type: application/json" \
|
| 480 |
-d "$REPLAY_CRIT")
|
| 481 |
check "N-01 Full correct replay → max_possible_reward includes weights" \
|
| 482 |
-
"$FULL" '"max_possible_reward":
|
| 483 |
check "N-02 Correct-but-no-investigation → total_reward below max (penalty applied)" \
|
| 484 |
"$(echo "$FULL" | python3 -c "
|
| 485 |
import sys,json,re
|
|
|
|
| 97 |
|
| 98 |
TASKS_RESP="$(curl -s $BASE/tasks)"
|
| 99 |
|
| 100 |
+
# B-01 30 tasks
|
| 101 |
+
check "B-01 GET /tasks → count=30" \
|
| 102 |
+
"$TASKS_RESP" '"count":30'
|
| 103 |
|
| 104 |
# B-02 All 4 difficulty tiers present
|
| 105 |
check "B-02 Tasks include 'easy' tier" "$TASKS_RESP" '"difficulty":"easy"'
|
|
|
|
| 159 |
"$(step '{"action_type":"flag","transaction_id":"TXN-E004"}')" \
|
| 160 |
'"reward":1.0'
|
| 161 |
|
| 162 |
+
# D-04b approve correct (mortgage repayment) → reward=1.0
|
| 163 |
+
check "D-04b EASY-005 approve → reward=1.0" \
|
| 164 |
+
"$(step '{"action_type":"approve","transaction_id":"TXN-E005"}')" \
|
| 165 |
+
'"reward":1.0'
|
| 166 |
+
|
| 167 |
+
# D-04c flag correct (duplicate payment) → reward=1.0
|
| 168 |
+
check "D-04c EASY-006 flag → reward=1.0" \
|
| 169 |
+
"$(step '{"action_type":"flag","transaction_id":"TXN-E006"}')" \
|
| 170 |
+
'"reward":1.0'
|
| 171 |
+
|
| 172 |
# D-05 escalate correct → reward=1.0
|
| 173 |
check "D-05 MED-001 escalate → reward=1.0" \
|
| 174 |
"$(step '{"action_type":"escalate","transaction_id":"TXN-M001"}')" \
|
|
|
|
| 212 |
step '{"action_type":"reject","transaction_id":"TXN-E002"}' > /dev/null
|
| 213 |
step '{"action_type":"approve","transaction_id":"TXN-E003"}' > /dev/null
|
| 214 |
step '{"action_type":"flag","transaction_id":"TXN-E004"}' > /dev/null
|
| 215 |
+
step '{"action_type":"approve","transaction_id":"TXN-E005"}' > /dev/null
|
| 216 |
+
step '{"action_type":"flag","transaction_id":"TXN-E006"}' > /dev/null
|
| 217 |
R=$(step '{"action_type":"flag","transaction_id":"TXN-M001"}')
|
| 218 |
check "E-03 Partial credit: flag on MED-001 (correct=escalate) → reward > 0" \
|
| 219 |
"$R" '"reward":0\.[0-9]'
|
|
|
|
| 307 |
# ═══════════════════════════════════════════════════════════
|
| 308 |
section "GROUP I — /replay endpoint (offline eval)"
|
| 309 |
|
| 310 |
+
# I-01 Replay with all 30 correct terminal actions (no investigation)
|
| 311 |
+
# Hard/critical tasks should be penalised (×0.80) → 0.7–0.9 range
|
| 312 |
+
# Task order: EASY×6, MED×8, HARD×10, CRIT×6
|
| 313 |
+
# Correct actions:
|
| 314 |
+
# EASY: approve reject approve flag approve flag
|
| 315 |
+
# MED: escalate hold flag flag hold escalate hold flag
|
| 316 |
+
# HARD: escalate reject reject approve escalate flag reject reject escalate reject
|
| 317 |
+
# CRIT: approve reject escalate reject reject escalate
|
| 318 |
+
REPLAY_PERFECT='{"actions":["approve","reject","approve","flag","approve","flag",
|
| 319 |
+
"escalate","hold","flag","flag","hold","escalate","hold","flag",
|
| 320 |
+
"escalate","reject","reject","approve","escalate","flag","reject","reject","escalate","reject",
|
| 321 |
+
"approve","reject","escalate","reject","reject","escalate"]}'
|
| 322 |
R=$(curl -s -X POST $BASE/replay \
|
| 323 |
-H "Content-Type: application/json" \
|
| 324 |
-d "$REPLAY_PERFECT")
|
|
|
|
| 326 |
check "I-02 Replay → passed=true (score>0.5 despite penalty)" "$R" '"passed":true'
|
| 327 |
check "I-03 Replay → budget_spent=0.0 (no inv actions)" "$R" '"budget_spent":0.0'
|
| 328 |
|
| 329 |
+
# I-04 Replay with investigation actions included (inspect before first task)
|
| 330 |
+
REPLAY_WITH_INV='{"actions":["inspect","approve","reject","approve","flag","approve","flag",
|
| 331 |
+
"escalate","hold","flag","flag","hold","escalate","hold","flag",
|
| 332 |
+
"escalate","reject","reject","approve","escalate","flag","reject","reject","escalate","reject",
|
| 333 |
+
"approve","reject","escalate","reject","reject","escalate"]}'
|
| 334 |
R=$(curl -s -X POST $BASE/replay \
|
| 335 |
-H "Content-Type: application/json" \
|
| 336 |
-d "$REPLAY_WITH_INV")
|
|
|
|
| 389 |
with urllib.request.urlopen(f"{BASE}{path}", context=_ssl) as r: return json.loads(r.read())
|
| 390 |
|
| 391 |
post("/reset", {"seed": 0})
|
| 392 |
+
# EASY x6 + MED x8 + HARD x10: no chain gate, terminal only
|
| 393 |
+
for action, txn in [
|
| 394 |
("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
|
| 395 |
+
("approve","TXN-E005"),("flag","TXN-E006"),
|
| 396 |
("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
|
| 397 |
+
("hold","TXN-M005"),("escalate","TXN-M006"),("hold","TXN-M007"),("flag","TXN-M008"),
|
| 398 |
("escalate","TXN-H001"),("reject","TXN-H002"),("reject","TXN-H003"),("approve","TXN-H004"),
|
| 399 |
+
("escalate","TXN-H005"),("flag","TXN-H006"),("reject","TXN-H007"),("reject","TXN-H008"),
|
| 400 |
+
("escalate","TXN-H009"),("reject","TXN-H010"),
|
| 401 |
+
]:
|
|
|
|
| 402 |
post("/step", {"action_type": action, "transaction_id": txn})
|
| 403 |
+
# CRIT x6: chain-gated — must provide chain_min investigation steps first
|
| 404 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C001"})
|
| 405 |
+
post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C001"})
|
| 406 |
+
post("/step", {"action_type": "approve", "transaction_id": "TXN-C001"})
|
| 407 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C002"})
|
| 408 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-C002"})
|
| 409 |
+
post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C003"})
|
| 410 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C003"})
|
| 411 |
+
post("/step", {"action_type": "escalate", "transaction_id": "TXN-C003"})
|
| 412 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C004"})
|
| 413 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-C004"})
|
| 414 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C005"})
|
| 415 |
+
post("/step", {"action_type": "verify_kyc", "transaction_id": "TXN-C005"})
|
| 416 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-C005"})
|
| 417 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C006"})
|
| 418 |
+
post("/step", {"action_type": "escalate", "transaction_id": "TXN-C006"})
|
| 419 |
PYEOF
|
| 420 |
|
| 421 |
ANA="$(curl -s $BASE/analytics)"
|
|
|
|
| 444 |
# ═══════════════════════════════════════════════════════════
|
| 445 |
# GROUP M — Perfect episode score
|
| 446 |
# ═══════════════════════════════════════════════════════════
|
| 447 |
+
section "GROUP M — Perfect episode (all 30 correct + investigation on hard/critical)"
|
| 448 |
|
| 449 |
reset
|
| 450 |
BASE=$BASE python3 - <<'PYEOF'
|
|
|
|
| 460 |
with urllib.request.urlopen(req, context=_ssl) as r: return json.loads(r.read())
|
| 461 |
|
| 462 |
post("/reset", {"seed": 0})
|
| 463 |
+
# Easy x6 + Medium x8: terminal only
|
| 464 |
for action, txn in [
|
| 465 |
("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
|
| 466 |
+
("approve","TXN-E005"),("flag","TXN-E006"),
|
| 467 |
("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
|
| 468 |
+
("hold","TXN-M005"),("escalate","TXN-M006"),("hold","TXN-M007"),("flag","TXN-M008"),
|
| 469 |
]:
|
| 470 |
post("/step", {"action_type": action, "transaction_id": txn})
|
| 471 |
+
# Hard x10: one required investigation sub-action before each terminal
|
| 472 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-H001"})
|
| 473 |
post("/step", {"action_type": "escalate", "transaction_id": "TXN-H001"})
|
| 474 |
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H002"})
|
|
|
|
| 482 |
post("/step", {"action_type": "escalate", "transaction_id": "TXN-H005"})
|
| 483 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-H006"})
|
| 484 |
post("/step", {"action_type": "flag", "transaction_id": "TXN-H006"})
|
| 485 |
+
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H007"})
|
| 486 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-H007"})
|
| 487 |
+
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H008"})
|
| 488 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-H008"})
|
| 489 |
+
post("/step", {"action_type": "verify_kyc", "transaction_id": "TXN-H009"})
|
| 490 |
+
post("/step", {"action_type": "escalate", "transaction_id": "TXN-H009"})
|
| 491 |
+
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H010"})
|
| 492 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-H010"})
|
| 493 |
+
# Critical x6: required investigation sub-actions before each terminal
|
| 494 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C001"})
|
| 495 |
post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C001"})
|
| 496 |
post("/step", {"action_type": "approve", "transaction_id": "TXN-C001"})
|
|
|
|
| 502 |
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C004"})
|
| 503 |
post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-C004"})
|
| 504 |
post("/step", {"action_type": "reject", "transaction_id": "TXN-C004"})
|
| 505 |
+
post("/step", {"action_type": "inspect", "transaction_id": "TXN-C005"})
|
| 506 |
+
post("/step", {"action_type": "verify_kyc", "transaction_id": "TXN-C005"})
|
| 507 |
+
post("/step", {"action_type": "reject", "transaction_id": "TXN-C005"})
|
| 508 |
+
post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C006"})
|
| 509 |
+
post("/step", {"action_type": "escalate", "transaction_id": "TXN-C006"})
|
| 510 |
PYEOF
|
| 511 |
|
| 512 |
GRADER="$(curl -s $BASE/grader)"
|
|
|
|
| 520 |
section "GROUP N — Difficulty weighting in grader"
|
| 521 |
|
| 522 |
# A critical task correct should contribute more weight than easy
|
| 523 |
+
REPLAY_CRIT='{"actions":["approve","reject","approve","flag","approve","flag",
|
| 524 |
+
"escalate","hold","flag","flag","hold","escalate","hold","flag",
|
| 525 |
+
"escalate","reject","reject","approve","escalate","flag","reject","reject","escalate","reject",
|
| 526 |
+
"approve","reject","escalate","reject","reject","escalate"]}'
|
| 527 |
|
| 528 |
FULL=$(curl -s -X POST $BASE/replay \
|
| 529 |
-H "Content-Type: application/json" \
|
| 530 |
-d "$REPLAY_CRIT")
|
| 531 |
check "N-01 Full correct replay → max_possible_reward includes weights" \
|
| 532 |
+
"$FULL" '"max_possible_reward":4[0-9]\.'
|
| 533 |
check "N-02 Correct-but-no-investigation → total_reward below max (penalty applied)" \
|
| 534 |
"$(echo "$FULL" | python3 -c "
|
| 535 |
import sys,json,re
|
scripts_util.py
CHANGED
|
@@ -77,6 +77,15 @@ def _should_investigate(obs) -> Optional[str]:
|
|
| 77 |
if (ml_conf < 0.60 or watchlist_hit) and "inspect" not in already:
|
| 78 |
return "inspect"
|
| 79 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
return None
|
| 81 |
|
| 82 |
|
|
|
|
| 77 |
if (ml_conf < 0.60 or watchlist_hit) and "inspect" not in already:
|
| 78 |
return "inspect"
|
| 79 |
|
| 80 |
+
# Fallback for chain-gated tasks (chain_total > 1): if not enough investigation
|
| 81 |
+
# steps have been done yet, issue generic actions in priority order so the
|
| 82 |
+
# baseline agent never gets stuck in an infinite chain-gate loop.
|
| 83 |
+
chain_min = max(0, getattr(obs, "chain_total", 1) - 1)
|
| 84 |
+
if chain_min > 0 and len(already) < chain_min:
|
| 85 |
+
for inv_action in ("inspect", "verify_kyc", "request_docs", "contact_sender", "file_sar"):
|
| 86 |
+
if inv_action not in already:
|
| 87 |
+
return inv_action
|
| 88 |
+
|
| 89 |
return None
|
| 90 |
|
| 91 |
|
tasks.py
CHANGED
|
@@ -206,11 +206,57 @@ EASY_TASKS: List[PayOpsTask] = [
|
|
| 206 |
inspect_reveal="15 ATM withdrawals in 58 min across 4 ATMs. Pattern consistent with card clone.",
|
| 207 |
key_flags=["velocity_breach"],
|
| 208 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
]
|
| 210 |
|
| 211 |
|
| 212 |
# =============================================================================
|
| 213 |
-
# MEDIUM (
|
| 214 |
# =============================================================================
|
| 215 |
|
| 216 |
MEDIUM_TASKS: List[PayOpsTask] = [
|
|
@@ -356,11 +402,66 @@ MEDIUM_TASKS: List[PayOpsTask] = [
|
|
| 356 |
docs_reveal="Signed purchase agreement found. Escrow agent licensed and registered. Standard conveyancing practice.",
|
| 357 |
key_flags=["first_time_payee", "large_first_transfer"],
|
| 358 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 359 |
]
|
| 360 |
|
| 361 |
|
| 362 |
# =============================================================================
|
| 363 |
-
# HARD (
|
| 364 |
# =============================================================================
|
| 365 |
|
| 366 |
HARD_TASKS: List[PayOpsTask] = [
|
|
@@ -540,11 +641,161 @@ HARD_TASKS: List[PayOpsTask] = [
|
|
| 540 |
),
|
| 541 |
key_flags=["dormant_receiver", "sudden_activity"],
|
| 542 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 543 |
]
|
| 544 |
|
| 545 |
|
| 546 |
# =============================================================================
|
| 547 |
-
# CRITICAL (
|
| 548 |
# =============================================================================
|
| 549 |
|
| 550 |
CRITICAL_TASKS: List[PayOpsTask] = [
|
|
@@ -673,6 +924,103 @@ CRITICAL_TASKS: List[PayOpsTask] = [
|
|
| 673 |
contact_reveal="CEO confirms they did NOT initiate this transfer. Account takeover confirmed.",
|
| 674 |
key_flags=["geo_impossible_login", "account_takeover_indicator"],
|
| 675 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 676 |
]
|
| 677 |
|
| 678 |
|
|
@@ -715,10 +1063,233 @@ ACTION_COSTS: Dict[str, float] = {
|
|
| 715 |
# ---------------------------------------------------------------------------
|
| 716 |
TASK_VARIANTS: Dict[str, List[Dict]] = {
|
| 717 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 718 |
# ── MEDIUM ────────────────────────────────────────────────────────────
|
| 719 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 720 |
"MED-003": [
|
| 721 |
-
# v1: unauthorised recurring billing → hold
|
| 722 |
{
|
| 723 |
"correct_action": "hold",
|
| 724 |
"partial_credit_actions": {"flag": 0.5, "escalate": 0.3},
|
|
@@ -748,8 +1319,8 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 748 |
"partial_credit_actions": {"flag": 0.5, "hold": 0.3},
|
| 749 |
"inspect_reveal": (
|
| 750 |
"Sender has made 7 crypto exchange payments in 5 days totalling $28k "
|
| 751 |
-
"across 3 different platforms. Amounts stay just below exchange
|
| 752 |
-
"limits. Pattern matches structuring. Escalate for AML review."
|
| 753 |
),
|
| 754 |
},
|
| 755 |
# v2: clean recurring investment → approve
|
|
@@ -815,6 +1386,69 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 815 |
},
|
| 816 |
],
|
| 817 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 818 |
# ── HARD ──────────────────────────────────────────────────────────────
|
| 819 |
|
| 820 |
"HARD-001": [
|
|
@@ -955,7 +1589,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 955 |
"Reject and file SAR immediately."
|
| 956 |
),
|
| 957 |
},
|
| 958 |
-
# v2: estate probate settlement, legitimate
|
| 959 |
{
|
| 960 |
"correct_action": "escalate",
|
| 961 |
"partial_credit_actions": {"flag": 0.6, "hold": 0.4},
|
|
@@ -968,6 +1602,135 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 968 |
},
|
| 969 |
],
|
| 970 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 971 |
# ── CRITICAL ──────────────────────────────────────────────────────────
|
| 972 |
|
| 973 |
"CRIT-001": [
|
|
@@ -1007,7 +1770,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 1007 |
],
|
| 1008 |
|
| 1009 |
"CRIT-002": [
|
| 1010 |
-
# v1: same SME
|
| 1011 |
{
|
| 1012 |
"correct_action": "hold",
|
| 1013 |
"partial_credit_actions": {"reject": 0.5, "escalate": 0.4},
|
|
@@ -1049,7 +1812,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 1049 |
"Hong Kong and the destination country. No discrepancies."
|
| 1050 |
),
|
| 1051 |
},
|
| 1052 |
-
# v2: criminal-scale TBML
|
| 1053 |
{
|
| 1054 |
"correct_action": "reject",
|
| 1055 |
"partial_credit_actions": {"escalate": 0.5, "flag": 0.3},
|
|
@@ -1068,7 +1831,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 1068 |
],
|
| 1069 |
|
| 1070 |
"CRIT-004": [
|
| 1071 |
-
# v1: CEO delegated to CFO who was in Lagos
|
| 1072 |
{
|
| 1073 |
"correct_action": "hold",
|
| 1074 |
"partial_credit_actions": {"escalate": 0.5, "reject": 0.4},
|
|
@@ -1084,7 +1847,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 1084 |
"Hold briefly to verify the papertrail, then release."
|
| 1085 |
),
|
| 1086 |
},
|
| 1087 |
-
# v2: CEO unreachable,
|
| 1088 |
{
|
| 1089 |
"correct_action": "escalate",
|
| 1090 |
"partial_credit_actions": {"reject": 0.5, "hold": 0.4},
|
|
@@ -1102,6 +1865,87 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
|
|
| 1102 |
),
|
| 1103 |
},
|
| 1104 |
],
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1105 |
}
|
| 1106 |
|
| 1107 |
|
|
|
|
| 206 |
inspect_reveal="15 ATM withdrawals in 58 min across 4 ATMs. Pattern consistent with card clone.",
|
| 207 |
key_flags=["velocity_breach"],
|
| 208 |
),
|
| 209 |
+
PayOpsTask(
|
| 210 |
+
task_id="EASY-005",
|
| 211 |
+
difficulty="easy",
|
| 212 |
+
description="Scheduled monthly mortgage repayment. Regular amount, verified on-time borrower, no flags.",
|
| 213 |
+
transaction_id="TXN-E005",
|
| 214 |
+
amount=1_247.00,
|
| 215 |
+
currency="USD",
|
| 216 |
+
sender="john.doe@homeowner.com",
|
| 217 |
+
receiver="mortgages@national-bank.com",
|
| 218 |
+
transaction_type="loan_repayment",
|
| 219 |
+
risk_score=0.03,
|
| 220 |
+
ml_confidence=0.99,
|
| 221 |
+
flags=[],
|
| 222 |
+
velocity_1h=1, velocity_24h=1,
|
| 223 |
+
avg_transaction_amount=1_247.00,
|
| 224 |
+
account_age_days=1_825,
|
| 225 |
+
country_risk="low", kyc_status="verified", kyc_expiry_days=280,
|
| 226 |
+
previous_violations=0, previous_sars=0, counterparty_risk="clean",
|
| 227 |
+
correct_action="approve",
|
| 228 |
+
partial_credit_actions={},
|
| 229 |
+
inspect_reveal="Loan account in good standing. Payment 22 of 360. Amount matches amortization schedule exactly.",
|
| 230 |
+
key_flags=[],
|
| 231 |
+
),
|
| 232 |
+
PayOpsTask(
|
| 233 |
+
task_id="EASY-006",
|
| 234 |
+
difficulty="easy",
|
| 235 |
+
description="Suspected duplicate payment: same sender, receiver, and amount submitted twice within 4 minutes.",
|
| 236 |
+
transaction_id="TXN-E006",
|
| 237 |
+
amount=523.50,
|
| 238 |
+
currency="USD",
|
| 239 |
+
sender="vendor@supplies-ltd.com",
|
| 240 |
+
receiver="buyer@procurement.com",
|
| 241 |
+
transaction_type="payment",
|
| 242 |
+
risk_score=0.55,
|
| 243 |
+
ml_confidence=0.91,
|
| 244 |
+
flags=["duplicate_transaction", "velocity_breach"],
|
| 245 |
+
velocity_1h=2, velocity_24h=3,
|
| 246 |
+
avg_transaction_amount=520.0,
|
| 247 |
+
account_age_days=980,
|
| 248 |
+
country_risk="low", kyc_status="verified", kyc_expiry_days=150,
|
| 249 |
+
previous_violations=0, previous_sars=0, counterparty_risk="clean",
|
| 250 |
+
correct_action="flag",
|
| 251 |
+
partial_credit_actions={"hold": 0.6},
|
| 252 |
+
inspect_reveal="TXN-E006-prev processed 4 min ago for identical amount and beneficiary. Duplicate submission detected — flag for double-charge review before releasing.",
|
| 253 |
+
key_flags=["duplicate_transaction"],
|
| 254 |
+
),
|
| 255 |
]
|
| 256 |
|
| 257 |
|
| 258 |
# =============================================================================
|
| 259 |
+
# MEDIUM (8 tasks — ambiguous, multi-signal)
|
| 260 |
# =============================================================================
|
| 261 |
|
| 262 |
MEDIUM_TASKS: List[PayOpsTask] = [
|
|
|
|
| 402 |
docs_reveal="Signed purchase agreement found. Escrow agent licensed and registered. Standard conveyancing practice.",
|
| 403 |
key_flags=["first_time_payee", "large_first_transfer"],
|
| 404 |
),
|
| 405 |
+
PayOpsTask(
|
| 406 |
+
task_id="MED-007",
|
| 407 |
+
difficulty="medium",
|
| 408 |
+
description=(
|
| 409 |
+
"Supplier emails to say bank details have changed; first payment to the "
|
| 410 |
+
"new account matches a large outstanding invoice. Classic BEC indicator."
|
| 411 |
+
),
|
| 412 |
+
transaction_id="TXN-M007",
|
| 413 |
+
amount=43_800.00,
|
| 414 |
+
currency="EUR",
|
| 415 |
+
sender="accounts@client-corp.de",
|
| 416 |
+
receiver="supplier-newacct@euro-clearing.eu",
|
| 417 |
+
transaction_type="payment",
|
| 418 |
+
risk_score=0.63,
|
| 419 |
+
ml_confidence=0.67,
|
| 420 |
+
flags=["bank_detail_change", "first_time_payee", "high_value"],
|
| 421 |
+
velocity_1h=1, velocity_24h=1,
|
| 422 |
+
avg_transaction_amount=44_000.0,
|
| 423 |
+
account_age_days=1_200,
|
| 424 |
+
country_risk="medium", kyc_status="verified", kyc_expiry_days=130,
|
| 425 |
+
previous_violations=0, previous_sars=0, counterparty_risk="unknown",
|
| 426 |
+
correct_action="hold",
|
| 427 |
+
partial_credit_actions={"escalate": 0.6, "flag": 0.4},
|
| 428 |
+
requires_investigation={"request_docs", "contact_sender"},
|
| 429 |
+
docs_reveal="Email requesting bank change was sent from supplier-noreply@euro-cIearing.eu (capital I, not l). No signed bank-change authorisation on file. Possible BEC.",
|
| 430 |
+
contact_reveal="Called original supplier contact number on record. They confirmed they did NOT send a bank-change notice. BEC fraud confirmed — hold and alert client.",
|
| 431 |
+
key_flags=["bank_detail_change", "first_time_payee"],
|
| 432 |
+
),
|
| 433 |
+
PayOpsTask(
|
| 434 |
+
task_id="MED-008",
|
| 435 |
+
difficulty="medium",
|
| 436 |
+
description=(
|
| 437 |
+
"Buy Now Pay Later purchase: high-value electronics order, account opened "
|
| 438 |
+
"3 weeks ago, thin credit file, elevated risk signals."
|
| 439 |
+
),
|
| 440 |
+
transaction_id="TXN-M008",
|
| 441 |
+
amount=2_899.00,
|
| 442 |
+
currency="USD",
|
| 443 |
+
sender="bnpl-platform@payin4.io",
|
| 444 |
+
receiver="electronics@megastore.com",
|
| 445 |
+
transaction_type="payment",
|
| 446 |
+
risk_score=0.60,
|
| 447 |
+
ml_confidence=0.71,
|
| 448 |
+
flags=["thin_credit_file", "high_value_bnpl", "new_account"],
|
| 449 |
+
velocity_1h=1, velocity_24h=2,
|
| 450 |
+
avg_transaction_amount=None,
|
| 451 |
+
account_age_days=21,
|
| 452 |
+
country_risk="low", kyc_status="pending", kyc_expiry_days=None,
|
| 453 |
+
previous_violations=0, previous_sars=0, counterparty_risk="clean",
|
| 454 |
+
correct_action="flag",
|
| 455 |
+
partial_credit_actions={"hold": 0.6, "escalate": 0.4},
|
| 456 |
+
requires_investigation={"verify_kyc"},
|
| 457 |
+
kyc_reveal="KYC check: identity document submitted but address verification failed — billing address does not match ID. BNPL fraud profile. Flag for manual identity review before releasing funds.",
|
| 458 |
+
key_flags=["thin_credit_file", "new_account"],
|
| 459 |
+
),
|
| 460 |
]
|
| 461 |
|
| 462 |
|
| 463 |
# =============================================================================
|
| 464 |
+
# HARD (10 tasks — adversarial, conflicting, edge-cases)
|
| 465 |
# =============================================================================
|
| 466 |
|
| 467 |
HARD_TASKS: List[PayOpsTask] = [
|
|
|
|
| 641 |
),
|
| 642 |
key_flags=["dormant_receiver", "sudden_activity"],
|
| 643 |
),
|
| 644 |
+
PayOpsTask(
|
| 645 |
+
task_id="HARD-007",
|
| 646 |
+
difficulty="hard",
|
| 647 |
+
description=(
|
| 648 |
+
"SIM-swap attack: account holder's phone number was ported to a new SIM "
|
| 649 |
+
"6 hours ago; account now requesting a large crypto withdrawal to a new address."
|
| 650 |
+
),
|
| 651 |
+
transaction_id="TXN-H007",
|
| 652 |
+
amount=14_200.00,
|
| 653 |
+
currency="USD",
|
| 654 |
+
sender="crypto-user@exchange.io",
|
| 655 |
+
receiver="external-wallet-9f3a@coldwallet.net",
|
| 656 |
+
transaction_type="withdrawal",
|
| 657 |
+
risk_score=0.74,
|
| 658 |
+
ml_confidence=0.78,
|
| 659 |
+
flags=["sim_swap_detected", "new_withdrawal_address", "high_value"],
|
| 660 |
+
velocity_1h=1, velocity_24h=1,
|
| 661 |
+
avg_transaction_amount=500.0,
|
| 662 |
+
account_age_days=620,
|
| 663 |
+
country_risk="low", kyc_status="verified", kyc_expiry_days=195,
|
| 664 |
+
previous_violations=0, previous_sars=0, counterparty_risk="unknown",
|
| 665 |
+
correct_action="reject",
|
| 666 |
+
partial_credit_actions={"hold": 0.5, "escalate": 0.4, "flag": 0.3},
|
| 667 |
+
requires_investigation={"inspect", "contact_sender"},
|
| 668 |
+
inspect_reveal=(
|
| 669 |
+
"Telecom provider confirmed: SIM swap on this account's registered phone "
|
| 670 |
+
"completed 6 h 12 min ago. External wallet address first appeared in this "
|
| 671 |
+
"session only. IP geolocation mismatch vs prior 90-day baseline."
|
| 672 |
+
),
|
| 673 |
+
contact_reveal=(
|
| 674 |
+
"Attempted call to registered phone: intercepted by new SIM holder, "
|
| 675 |
+
"who provided all correct security answers (likely obtained via social "
|
| 676 |
+
"engineering). Account holder unreachable on backup email. SIM-swap "
|
| 677 |
+
"takeover confirmed — reject and lock account."
|
| 678 |
+
),
|
| 679 |
+
key_flags=["sim_swap_detected", "new_withdrawal_address"],
|
| 680 |
+
),
|
| 681 |
+
PayOpsTask(
|
| 682 |
+
task_id="HARD-008",
|
| 683 |
+
difficulty="hard",
|
| 684 |
+
description=(
|
| 685 |
+
"Romance scam / pig butchering: customer making 4th escalating transfer "
|
| 686 |
+
"to an overseas 'romantic partner' met on a dating app 7 weeks ago. "
|
| 687 |
+
"Amounts have doubled each time."
|
| 688 |
+
),
|
| 689 |
+
transaction_id="TXN-H008",
|
| 690 |
+
amount=8_000.00,
|
| 691 |
+
currency="USD",
|
| 692 |
+
sender="victim@personal-bank.com",
|
| 693 |
+
receiver="crypto-invest@hongkong-fund.net",
|
| 694 |
+
transaction_type="transfer",
|
| 695 |
+
risk_score=0.66,
|
| 696 |
+
ml_confidence=0.73,
|
| 697 |
+
flags=["romance_scam_indicator", "escalating_transfers", "offshore_receiver"],
|
| 698 |
+
velocity_1h=1, velocity_24h=1,
|
| 699 |
+
avg_transaction_amount=1_200.0,
|
| 700 |
+
account_age_days=4_015,
|
| 701 |
+
country_risk="high", kyc_status="verified", kyc_expiry_days=220,
|
| 702 |
+
previous_violations=0, previous_sars=0, counterparty_risk="blacklist",
|
| 703 |
+
correct_action="reject",
|
| 704 |
+
partial_credit_actions={"hold": 0.5, "escalate": 0.4, "flag": 0.3},
|
| 705 |
+
requires_investigation={"contact_sender", "inspect"},
|
| 706 |
+
contact_reveal=(
|
| 707 |
+
"Spoke with account holder. They believe they are investing with a "
|
| 708 |
+
"partner they met on a dating app. Prior transfers: $500, $1,000, $4,000. "
|
| 709 |
+
"Receiver domain registered 3 months ago. Pig-butchering romance scam confirmed."
|
| 710 |
+
),
|
| 711 |
+
inspect_reveal=(
|
| 712 |
+
"Transfer history: $500 → $1,000 → $4,000 → $8,000 over 7 weeks. "
|
| 713 |
+
"Receiver account linked to 23 other inbound romance-scam reports in the "
|
| 714 |
+
"financial intelligence database. All prior transfers drained offshore within hours."
|
| 715 |
+
),
|
| 716 |
+
key_flags=["romance_scam_indicator", "escalating_transfers"],
|
| 717 |
+
network_graph={"victim": "victim@personal-bank.com", "scammer_fund": "crypto-invest@hongkong-fund.net"},
|
| 718 |
+
),
|
| 719 |
+
PayOpsTask(
|
| 720 |
+
task_id="HARD-009",
|
| 721 |
+
difficulty="hard",
|
| 722 |
+
description=(
|
| 723 |
+
"Synthetic identity fraud: new business account with a flawless-looking "
|
| 724 |
+
"profile (perfect score, clean history) requests a large first outbound "
|
| 725 |
+
"wire within 10 days of opening."
|
| 726 |
+
),
|
| 727 |
+
transaction_id="TXN-H009",
|
| 728 |
+
amount=19_500.00,
|
| 729 |
+
currency="USD",
|
| 730 |
+
sender="newco-llc@startup-bank.com",
|
| 731 |
+
receiver="vendor@offshore-services.biz",
|
| 732 |
+
transaction_type="transfer",
|
| 733 |
+
risk_score=0.35,
|
| 734 |
+
ml_confidence=0.40, # low confidence: model flagging identity inconsistency
|
| 735 |
+
flags=["new_account_10d", "perfect_score_anomaly", "large_first_wire"],
|
| 736 |
+
velocity_1h=1, velocity_24h=1,
|
| 737 |
+
avg_transaction_amount=None,
|
| 738 |
+
account_age_days=10,
|
| 739 |
+
country_risk="medium", kyc_status="verified", kyc_expiry_days=360,
|
| 740 |
+
previous_violations=0, previous_sars=0, counterparty_risk="unknown",
|
| 741 |
+
correct_action="escalate",
|
| 742 |
+
partial_credit_actions={"hold": 0.5, "flag": 0.4, "reject": 0.3},
|
| 743 |
+
requires_investigation={"verify_kyc", "inspect"},
|
| 744 |
+
kyc_reveal=(
|
| 745 |
+
"KYC documents re-examined: EIN was issued 11 days ago but the 'years "
|
| 746 |
+
"in business' field on the application states 4 years. Registered address "
|
| 747 |
+
"is a UPS store mailbox. Director SSN belongs to a real person with no "
|
| 748 |
+
"knowledge of this LLC. Synthetic identity confirmed."
|
| 749 |
+
),
|
| 750 |
+
inspect_reveal=(
|
| 751 |
+
"Account ML confidence flagged as low: identity features are statistically "
|
| 752 |
+
"perfect (no noise in credit history, address, DOB fields) — consistent "
|
| 753 |
+
"with synthetically generated identity. Escalate for manual identity verification."
|
| 754 |
+
),
|
| 755 |
+
key_flags=["new_account_10d", "perfect_score_anomaly"],
|
| 756 |
+
),
|
| 757 |
+
PayOpsTask(
|
| 758 |
+
task_id="HARD-010",
|
| 759 |
+
difficulty="hard",
|
| 760 |
+
description=(
|
| 761 |
+
"Payroll diversion: HR system breach rerouted an employee's salary to "
|
| 762 |
+
"a newly added bank account 2 days ago. Employee has not been notified."
|
| 763 |
+
),
|
| 764 |
+
transaction_id="TXN-H010",
|
| 765 |
+
amount=5_640.00,
|
| 766 |
+
currency="USD",
|
| 767 |
+
sender="payroll@employer-corp.com",
|
| 768 |
+
receiver="new-acct-redirect@routing-bank.com",
|
| 769 |
+
transaction_type="payroll",
|
| 770 |
+
risk_score=0.58,
|
| 771 |
+
ml_confidence=0.63,
|
| 772 |
+
flags=["direct_deposit_changed", "recent_account_change", "payroll_diversion_indicator"],
|
| 773 |
+
velocity_1h=1, velocity_24h=1,
|
| 774 |
+
avg_transaction_amount=5_640.0,
|
| 775 |
+
account_age_days=2_920,
|
| 776 |
+
country_risk="low", kyc_status="verified", kyc_expiry_days=180,
|
| 777 |
+
previous_violations=0, previous_sars=0, counterparty_risk="unknown",
|
| 778 |
+
correct_action="reject",
|
| 779 |
+
partial_credit_actions={"hold": 0.5, "escalate": 0.4, "flag": 0.3},
|
| 780 |
+
requires_investigation={"inspect", "contact_sender"},
|
| 781 |
+
inspect_reveal=(
|
| 782 |
+
"HR system audit: direct deposit account changed 2 days ago via an "
|
| 783 |
+
"unrecognized IP at 2:14 AM. Employee's primary device was offline at the "
|
| 784 |
+
"time. No 2FA challenge was issued. Account change was unauthorized."
|
| 785 |
+
),
|
| 786 |
+
contact_reveal=(
|
| 787 |
+
"Reached employee by mobile: they confirm they did NOT change their "
|
| 788 |
+
"direct deposit account. Did not receive any email or SMS about the "
|
| 789 |
+
"change. HR system credentials were likely phished. Reject this payroll "
|
| 790 |
+
"disbursement and revert to prior account."
|
| 791 |
+
),
|
| 792 |
+
key_flags=["direct_deposit_changed", "payroll_diversion_indicator"],
|
| 793 |
+
),
|
| 794 |
]
|
| 795 |
|
| 796 |
|
| 797 |
# =============================================================================
|
| 798 |
+
# CRITICAL (6 tasks — multi-step chains, regulatory stakes)
|
| 799 |
# =============================================================================
|
| 800 |
|
| 801 |
CRITICAL_TASKS: List[PayOpsTask] = [
|
|
|
|
| 924 |
contact_reveal="CEO confirms they did NOT initiate this transfer. Account takeover confirmed.",
|
| 925 |
key_flags=["geo_impossible_login", "account_takeover_indicator"],
|
| 926 |
),
|
| 927 |
+
PayOpsTask(
|
| 928 |
+
task_id="CRIT-005",
|
| 929 |
+
difficulty="critical",
|
| 930 |
+
description=(
|
| 931 |
+
"OFAC sanctions evasion: large USD payment routed through a UAE shell "
|
| 932 |
+
"company chain. Investigation reveals the ultimate beneficial owner is "
|
| 933 |
+
"on the OFAC Specially Designated Nationals (SDN) list."
|
| 934 |
+
),
|
| 935 |
+
transaction_id="TXN-C005",
|
| 936 |
+
amount=490_000.00,
|
| 937 |
+
currency="USD",
|
| 938 |
+
sender="uae-trading@al-rashid-fze.ae",
|
| 939 |
+
receiver="intermediary@gulf-clearing-bv.nl",
|
| 940 |
+
transaction_type="transfer",
|
| 941 |
+
risk_score=0.69,
|
| 942 |
+
ml_confidence=0.60,
|
| 943 |
+
flags=["high_value", "cross_border", "shell_company_indicator", "sanctions_adjacent"],
|
| 944 |
+
velocity_1h=1, velocity_24h=2,
|
| 945 |
+
avg_transaction_amount=380_000.0,
|
| 946 |
+
account_age_days=180,
|
| 947 |
+
country_risk="high", kyc_status="verified", kyc_expiry_days=45,
|
| 948 |
+
previous_violations=0, previous_sars=0, counterparty_risk="watchlist",
|
| 949 |
+
correct_action="reject",
|
| 950 |
+
partial_credit_actions={"escalate": 0.5, "hold": 0.4},
|
| 951 |
+
requires_investigation={"inspect", "verify_kyc", "request_docs"},
|
| 952 |
+
regulatory_action=True,
|
| 953 |
+
chain_total=3,
|
| 954 |
+
inspect_reveal=(
|
| 955 |
+
"Corporate registry search: al-rashid-fze.ae incorporated 6 months ago. "
|
| 956 |
+
"Sole director: a nominee agent used by 47 other FZE shells. Beneficial "
|
| 957 |
+
"ownership records absent. Payment routed via 3 intermediate entities "
|
| 958 |
+
"before reaching the UAE sender — classic sanctions-evasion layering."
|
| 959 |
+
),
|
| 960 |
+
kyc_reveal=(
|
| 961 |
+
"UBO lookup via SWIFT KYC Registry and Refinitiv: ultimate beneficial "
|
| 962 |
+
"owner resolves to Tariq Al-Mansouri, listed on OFAC SDN list since 2021 "
|
| 963 |
+
"(Ref: SDN-2021-04-19-TM). Transacting with this UBO violates 31 C.F.R. "
|
| 964 |
+
"§ 594. Reject immediately and file SAR."
|
| 965 |
+
),
|
| 966 |
+
docs_reveal=(
|
| 967 |
+
"Provided trade invoice shows goods described as 'industrial equipment'. "
|
| 968 |
+
"HS code used (8479.89) does not match the attached packing list. "
|
| 969 |
+
"Likely misclassification to obscure dual-use goods export controls."
|
| 970 |
+
),
|
| 971 |
+
key_flags=["shell_company_indicator", "sanctions_adjacent"],
|
| 972 |
+
network_graph={
|
| 973 |
+
"sender_shell": "al-rashid-fze.ae",
|
| 974 |
+
"intermediary": "gulf-clearing-bv.nl",
|
| 975 |
+
"ubo_sdn": "Tariq Al-Mansouri",
|
| 976 |
+
},
|
| 977 |
+
),
|
| 978 |
+
PayOpsTask(
|
| 979 |
+
task_id="CRIT-006",
|
| 980 |
+
difficulty="critical",
|
| 981 |
+
description=(
|
| 982 |
+
"Correspondent banking alert: a downstream correspondent partner bank "
|
| 983 |
+
"was added to FinCEN's 311 Special Measures list yesterday. Payments "
|
| 984 |
+
"routed through them are now legally restricted."
|
| 985 |
+
),
|
| 986 |
+
transaction_id="TXN-C006",
|
| 987 |
+
amount=215_000.00,
|
| 988 |
+
currency="USD",
|
| 989 |
+
sender="trade-desk@our-bank.com",
|
| 990 |
+
receiver="correspondent@partner-bank-latam.bz",
|
| 991 |
+
transaction_type="transfer",
|
| 992 |
+
risk_score=0.71,
|
| 993 |
+
ml_confidence=0.65,
|
| 994 |
+
flags=["fincen_311_alert", "correspondent_risk", "high_value", "cross_border"],
|
| 995 |
+
velocity_1h=3, velocity_24h=12,
|
| 996 |
+
avg_transaction_amount=200_000.0,
|
| 997 |
+
account_age_days=2_100,
|
| 998 |
+
country_risk="high", kyc_status="verified", kyc_expiry_days=20,
|
| 999 |
+
previous_violations=0, previous_sars=0, counterparty_risk="watchlist",
|
| 1000 |
+
correct_action="escalate",
|
| 1001 |
+
partial_credit_actions={"hold": 0.5, "reject": 0.4},
|
| 1002 |
+
requires_investigation={"inspect", "request_docs"},
|
| 1003 |
+
regulatory_action=True,
|
| 1004 |
+
chain_total=2,
|
| 1005 |
+
inspect_reveal=(
|
| 1006 |
+
"FinCEN Section 311 Special Measure imposed 2024-04-01 on partner-bank-latam.bz "
|
| 1007 |
+
"for 'primary money laundering concern'. Special Measure 5 prohibits US "
|
| 1008 |
+
"correspondent accounts and payable-through accounts. All in-flight wires "
|
| 1009 |
+
"must be escalated to Compliance before release — unilateral rejection "
|
| 1010 |
+
"also requires Compliance sign-off (may breach existing agreements)."
|
| 1011 |
+
),
|
| 1012 |
+
docs_reveal=(
|
| 1013 |
+
"Correspondent agreement review: clause 12.3 requires 90-day wind-down "
|
| 1014 |
+
"notice before terminating the relationship. Legal and Compliance must "
|
| 1015 |
+
"jointly approve next steps. Escalate immediately — do not reject or "
|
| 1016 |
+
"approve without written senior Compliance authorisation."
|
| 1017 |
+
),
|
| 1018 |
+
key_flags=["fincen_311_alert", "correspondent_risk"],
|
| 1019 |
+
network_graph={
|
| 1020 |
+
"our_bank": "trade-desk@our-bank.com",
|
| 1021 |
+
"correspondent": "partner-bank-latam.bz",
|
| 1022 |
+
},
|
| 1023 |
+
),
|
| 1024 |
]
|
| 1025 |
|
| 1026 |
|
|
|
|
| 1063 |
# ---------------------------------------------------------------------------
|
| 1064 |
TASK_VARIANTS: Dict[str, List[Dict]] = {
|
| 1065 |
|
| 1066 |
+
# ── EASY ──────────────────────────────────────────────────────────────
|
| 1067 |
+
|
| 1068 |
+
"EASY-001": [
|
| 1069 |
+
# v1: account accessed from unknown device just before transfer → hold
|
| 1070 |
+
{
|
| 1071 |
+
"correct_action": "hold",
|
| 1072 |
+
"partial_credit_actions": {"flag": 0.6, "escalate": 0.3},
|
| 1073 |
+
"flags": ["unusual_login_location", "device_change"],
|
| 1074 |
+
"risk_score": 0.61,
|
| 1075 |
+
"inspect_reveal": (
|
| 1076 |
+
"Sender logged in from an unrecognized device in a different country "
|
| 1077 |
+
"18 minutes ago. Possible account takeover. All signals otherwise "
|
| 1078 |
+
"identical. Hold pending confirmation with account holder."
|
| 1079 |
+
),
|
| 1080 |
+
},
|
| 1081 |
+
# v2: amount exceeds sender's daily limit for new payee → flag
|
| 1082 |
+
{
|
| 1083 |
+
"correct_action": "flag",
|
| 1084 |
+
"partial_credit_actions": {"hold": 0.6, "approve": 0.2},
|
| 1085 |
+
"flags": ["first_time_payee", "amount_spike"],
|
| 1086 |
+
"risk_score": 0.44,
|
| 1087 |
+
"inspect_reveal": (
|
| 1088 |
+
"Bob is a new payee — first time ever on this account. System "
|
| 1089 |
+
"limit for first-time payees is $25. Transfer amount $47.50 exceeds "
|
| 1090 |
+
"new-payee daily cap. Flag for standard new-payee challenge-and-confirm."
|
| 1091 |
+
),
|
| 1092 |
+
},
|
| 1093 |
+
],
|
| 1094 |
+
|
| 1095 |
+
"EASY-002": [
|
| 1096 |
+
# v1: signals are real but entity is a licensed MSB with OFAC authorisation → escalate
|
| 1097 |
+
{
|
| 1098 |
+
"correct_action": "escalate",
|
| 1099 |
+
"partial_credit_actions": {"reject": 0.4, "hold": 0.3},
|
| 1100 |
+
"inspect_reveal": (
|
| 1101 |
+
"Sender is a licensed Money Service Business (MSB) with a FinCEN "
|
| 1102 |
+
"registration number and a current OFAC sanctions-compliance programme. "
|
| 1103 |
+
"The 'sanctioned country' flag was triggered by destination country "
|
| 1104 |
+
"of receiver's beneficial owner, but this MSB holds a specific OFAC "
|
| 1105 |
+
"licence for this corridor. Escalate for senior review rather than "
|
| 1106 |
+
"unilateral reject."
|
| 1107 |
+
),
|
| 1108 |
+
},
|
| 1109 |
+
# v2: additional money mule node detected → reject + SAR
|
| 1110 |
+
{
|
| 1111 |
+
"correct_action": "reject",
|
| 1112 |
+
"partial_credit_actions": {"escalate": 0.5},
|
| 1113 |
+
"regulatory_action": True,
|
| 1114 |
+
"inspect_reveal": (
|
| 1115 |
+
"Receiver account linked to 6 prior confirmed fraud rejections this "
|
| 1116 |
+
"quarter, totalling $58k. New network node in a known mule ring. "
|
| 1117 |
+
"All prior transfers drained offshore within 30 minutes. Reject "
|
| 1118 |
+
"immediately and file SAR."
|
| 1119 |
+
),
|
| 1120 |
+
},
|
| 1121 |
+
],
|
| 1122 |
+
|
| 1123 |
+
"EASY-003": [
|
| 1124 |
+
# v1: refund issued to a different card than the original purchase → flag
|
| 1125 |
+
{
|
| 1126 |
+
"correct_action": "flag",
|
| 1127 |
+
"partial_credit_actions": {"hold": 0.5, "escalate": 0.3},
|
| 1128 |
+
"flags": ["refund_mismatch", "different_card"],
|
| 1129 |
+
"risk_score": 0.52,
|
| 1130 |
+
"inspect_reveal": (
|
| 1131 |
+
"Original purchase was made on Visa ending 4412. Refund is being "
|
| 1132 |
+
"issued to Mastercard ending 7891 — a different card not on file for "
|
| 1133 |
+
"this order. Possible refund fraud. Flag for merchant verification "
|
| 1134 |
+
"before processing."
|
| 1135 |
+
),
|
| 1136 |
+
},
|
| 1137 |
+
# v2: refund claimed on a fraudulent chargeback dispute → reject
|
| 1138 |
+
{
|
| 1139 |
+
"correct_action": "reject",
|
| 1140 |
+
"partial_credit_actions": {"flag": 0.5, "escalate": 0.3},
|
| 1141 |
+
"flags": ["chargeback_fraud", "dispute_open"],
|
| 1142 |
+
"risk_score": 0.68,
|
| 1143 |
+
"inspect_reveal": (
|
| 1144 |
+
"Customer filed a chargeback claiming non-receipt. Courier tracking "
|
| 1145 |
+
"shows package delivered and signed for 3 days ago. Customer's "
|
| 1146 |
+
"claimed delivery address matches the signature. Friendly fraud — "
|
| 1147 |
+
"reject refund and respond to chargeback with delivery evidence."
|
| 1148 |
+
),
|
| 1149 |
+
},
|
| 1150 |
+
],
|
| 1151 |
+
|
| 1152 |
+
"EASY-004": [
|
| 1153 |
+
# v1: card-cloning confirmed by bank intelligence → reject
|
| 1154 |
+
{
|
| 1155 |
+
"correct_action": "reject",
|
| 1156 |
+
"partial_credit_actions": {"escalate": 0.5, "flag": 0.4},
|
| 1157 |
+
"inspect_reveal": (
|
| 1158 |
+
"Cross-check with card-fraud intelligence feed: the card number "
|
| 1159 |
+
"involved in this ATM burst matches a batch of 340 card numbers "
|
| 1160 |
+
"from a point-of-sale skimming incident reported yesterday. Confirmed "
|
| 1161 |
+
"card-clone attack. Reject all withdrawals and block card immediately."
|
| 1162 |
+
),
|
| 1163 |
+
},
|
| 1164 |
+
# v2: legitimate end-of-month cash distribution for a small cash-and-carry → approve
|
| 1165 |
+
{
|
| 1166 |
+
"correct_action": "approve",
|
| 1167 |
+
"partial_credit_actions": {"flag": 0.3},
|
| 1168 |
+
"inspect_reveal": (
|
| 1169 |
+
"Account holder is a licensed cash-and-carry wholesaler. Same "
|
| 1170 |
+
"end-of-month ATM burst pattern appears in 11 of the last 12 months — "
|
| 1171 |
+
"consistent with weekly cash float replenishment for their 4 shop "
|
| 1172 |
+
"locations. No fraud intelligence matches. Approve."
|
| 1173 |
+
),
|
| 1174 |
+
},
|
| 1175 |
+
],
|
| 1176 |
+
|
| 1177 |
+
"EASY-005": [
|
| 1178 |
+
# v1: loan account in arrears, payment covers only partial balance → flag
|
| 1179 |
+
{
|
| 1180 |
+
"correct_action": "flag",
|
| 1181 |
+
"partial_credit_actions": {"hold": 0.5, "approve": 0.3},
|
| 1182 |
+
"amount": 650.00,
|
| 1183 |
+
"flags": ["partial_payment", "loan_arrears"],
|
| 1184 |
+
"risk_score": 0.38,
|
| 1185 |
+
"inspect_reveal": (
|
| 1186 |
+
"Loan account 2 months in arrears. Required minimum payment is "
|
| 1187 |
+
"$1,247 but sender submitted $650. Partial payment does not clear "
|
| 1188 |
+
"the overdue balance. Flag for borrower contact and payment plan "
|
| 1189 |
+
"communication before crediting."
|
| 1190 |
+
),
|
| 1191 |
+
},
|
| 1192 |
+
# v2: overpayment submitted — process normally but note discrepancy → approve
|
| 1193 |
+
{
|
| 1194 |
+
"correct_action": "approve",
|
| 1195 |
+
"partial_credit_actions": {"flag": 0.3},
|
| 1196 |
+
"amount": 2_500.00,
|
| 1197 |
+
"flags": ["overpayment"],
|
| 1198 |
+
"risk_score": 0.08,
|
| 1199 |
+
"inspect_reveal": (
|
| 1200 |
+
"Borrower submitted $2,500 vs scheduled $1,247. Overpayments are "
|
| 1201 |
+
"permitted under the loan agreement (clause 8.2) and reduce principal "
|
| 1202 |
+
"balance. No fraud indicators. Approve and apply excess to principal."
|
| 1203 |
+
),
|
| 1204 |
+
},
|
| 1205 |
+
],
|
| 1206 |
+
|
| 1207 |
+
"EASY-006": [
|
| 1208 |
+
# v1: genuine re-submission after first instance failed → approve
|
| 1209 |
+
{
|
| 1210 |
+
"correct_action": "approve",
|
| 1211 |
+
"partial_credit_actions": {"flag": 0.4},
|
| 1212 |
+
"flags": ["resubmission"],
|
| 1213 |
+
"risk_score": 0.18,
|
| 1214 |
+
"inspect_reveal": (
|
| 1215 |
+
"First submission (4 min ago) failed with a bank routing error "
|
| 1216 |
+
"(error code: ROUTE-001 — invalid sort code format). This is a valid "
|
| 1217 |
+
"resubmission with corrected routing. No double-charge risk. Approve."
|
| 1218 |
+
),
|
| 1219 |
+
},
|
| 1220 |
+
# v2: vendor-side system error caused 8 duplicate submissions → hold
|
| 1221 |
+
{
|
| 1222 |
+
"correct_action": "hold",
|
| 1223 |
+
"partial_credit_actions": {"flag": 0.6},
|
| 1224 |
+
"velocity_1h": 8,
|
| 1225 |
+
"flags": ["duplicate_transaction", "velocity_breach", "api_error_burst"],
|
| 1226 |
+
"risk_score": 0.72,
|
| 1227 |
+
"inspect_reveal": (
|
| 1228 |
+
"Vendor payment API experienced a timeout loop at 14:02. 8 identical "
|
| 1229 |
+
"payment submissions in 5 minutes from the same API key. Only 1 "
|
| 1230 |
+
"should be processed. Hold all 8; contact vendor to identify the "
|
| 1231 |
+
"canonical intended payment before releasing any."
|
| 1232 |
+
),
|
| 1233 |
+
},
|
| 1234 |
+
],
|
| 1235 |
+
|
| 1236 |
# ── MEDIUM ────────────────────────────────────────────────────────────
|
| 1237 |
|
| 1238 |
+
"MED-001": [
|
| 1239 |
+
# v1: CEO impersonation (BEC) — fraudulent wire instruction → reject
|
| 1240 |
+
{
|
| 1241 |
+
"correct_action": "reject",
|
| 1242 |
+
"partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
|
| 1243 |
+
"flags": ["high_value", "cross_border", "bec_indicator"],
|
| 1244 |
+
"risk_score": 0.74,
|
| 1245 |
+
"inspect_reveal": (
|
| 1246 |
+
"The CFO's email domain in the wire instructions is globalcorp.co "
|
| 1247 |
+
"(note: .co not .com). CFO's actual domain is globalcorp.com. "
|
| 1248 |
+
"Phone verification: real CFO is on holiday and did NOT authorise "
|
| 1249 |
+
"this transfer. Business Email Compromise confirmed — reject."
|
| 1250 |
+
),
|
| 1251 |
+
},
|
| 1252 |
+
# v2: contract and counterparty fully verified → approve
|
| 1253 |
+
{
|
| 1254 |
+
"correct_action": "approve",
|
| 1255 |
+
"partial_credit_actions": {"escalate": 0.3, "flag": 0.2},
|
| 1256 |
+
"inspect_reveal": (
|
| 1257 |
+
"Vendor contract on file, signed 3 months ago (ref VC-EUR-0712). "
|
| 1258 |
+
"Receiver is an EU-licensed entity (UID DE315441892). Payment matches "
|
| 1259 |
+
"milestone 2 of 4. CFO confirmed by phone. Cross-border and amount "
|
| 1260 |
+
"are within normal operating parameters. Approve."
|
| 1261 |
+
),
|
| 1262 |
+
},
|
| 1263 |
+
],
|
| 1264 |
+
|
| 1265 |
+
"MED-002": [
|
| 1266 |
+
# v1: KYC re-check reveals UBO mismatch → escalate
|
| 1267 |
+
{
|
| 1268 |
+
"correct_action": "escalate",
|
| 1269 |
+
"partial_credit_actions": {"hold": 0.5, "flag": 0.3},
|
| 1270 |
+
"kyc_reveal": (
|
| 1271 |
+
"KYC renewal triggered a UBO re-check. Companies House filing shows "
|
| 1272 |
+
"the ultimate beneficial owner changed 3 months ago to a holding "
|
| 1273 |
+
"company domiciled in a secrecy jurisdiction, without bank "
|
| 1274 |
+
"notification. Material KYC change — escalate for enhanced due "
|
| 1275 |
+
"diligence before releasing any treasury transfers."
|
| 1276 |
+
),
|
| 1277 |
+
},
|
| 1278 |
+
# v2: board resolution authorises emergency liquidity transfer → approve
|
| 1279 |
+
{
|
| 1280 |
+
"correct_action": "approve",
|
| 1281 |
+
"partial_credit_actions": {"hold": 0.4},
|
| 1282 |
+
"kyc_reveal": (
|
| 1283 |
+
"KYC renewal: both entities share the same UBO (verified). Pending "
|
| 1284 |
+
"status is purely administrative — renewal submitted 6 days ago, "
|
| 1285 |
+
"no change of ownership. Board resolution ref BR-2024-Q1-08 "
|
| 1286 |
+
"authorises this emergency liquidity transfer. Approve."
|
| 1287 |
+
),
|
| 1288 |
+
},
|
| 1289 |
+
],
|
| 1290 |
+
|
| 1291 |
"MED-003": [
|
| 1292 |
+
# v1: unauthorised recurring billing → hold
|
| 1293 |
{
|
| 1294 |
"correct_action": "hold",
|
| 1295 |
"partial_credit_actions": {"flag": 0.5, "escalate": 0.3},
|
|
|
|
| 1319 |
"partial_credit_actions": {"flag": 0.5, "hold": 0.3},
|
| 1320 |
"inspect_reveal": (
|
| 1321 |
"Sender has made 7 crypto exchange payments in 5 days totalling $28k "
|
| 1322 |
+
"across 3 different platforms. Amounts stay just below exchange "
|
| 1323 |
+
"reporting limits. Pattern matches structuring. Escalate for AML review."
|
| 1324 |
),
|
| 1325 |
},
|
| 1326 |
# v2: clean recurring investment → approve
|
|
|
|
| 1386 |
},
|
| 1387 |
],
|
| 1388 |
|
| 1389 |
+
"MED-007": [
|
| 1390 |
+
# v1: genuine bank account change, properly authorised → approve
|
| 1391 |
+
{
|
| 1392 |
+
"correct_action": "approve",
|
| 1393 |
+
"partial_credit_actions": {"hold": 0.5, "flag": 0.3},
|
| 1394 |
+
"docs_reveal": (
|
| 1395 |
+
"Signed bank-change authorisation letter provided on supplier "
|
| 1396 |
+
"letterhead, with notarised director signature matching KYC records. "
|
| 1397 |
+
"Called supplier finance director on known contact number — confirmed "
|
| 1398 |
+
"the account change. New account is registered to the same company. "
|
| 1399 |
+
"Approve."
|
| 1400 |
+
),
|
| 1401 |
+
"contact_reveal": (
|
| 1402 |
+
"Original supplier contact verified the bank change was genuine — "
|
| 1403 |
+
"they switched banks last month due to a merger. Reference number "
|
| 1404 |
+
"SBC-2024-0312 matches their internal change management log."
|
| 1405 |
+
),
|
| 1406 |
+
},
|
| 1407 |
+
# v2: multiple lookalike invoice redirect emails sent to same client → escalate
|
| 1408 |
+
{
|
| 1409 |
+
"correct_action": "escalate",
|
| 1410 |
+
"partial_credit_actions": {"hold": 0.5, "reject": 0.4},
|
| 1411 |
+
"docs_reveal": (
|
| 1412 |
+
"Client reports receiving 3 lookalike invoice redirect emails in 2 "
|
| 1413 |
+
"weeks from slightly different supplier domains. Two payments to "
|
| 1414 |
+
"prior redirect accounts were already processed (total €87k). This "
|
| 1415 |
+
"is a systematic BEC campaign targeting this client relationship. "
|
| 1416 |
+
"Escalate to fraud and financial crime team immediately."
|
| 1417 |
+
),
|
| 1418 |
+
"contact_reveal": (
|
| 1419 |
+
"Genuine supplier finance director: they have sent zero bank-change "
|
| 1420 |
+
"notices this year. The lookalike domain was registered 3 weeks ago. "
|
| 1421 |
+
"Client's email system has been compromised. Escalate."
|
| 1422 |
+
),
|
| 1423 |
+
},
|
| 1424 |
+
],
|
| 1425 |
+
|
| 1426 |
+
"MED-008": [
|
| 1427 |
+
# v1: KYC passes, address verified — legitimate BNPL purchase → approve
|
| 1428 |
+
{
|
| 1429 |
+
"correct_action": "approve",
|
| 1430 |
+
"partial_credit_actions": {"flag": 0.4},
|
| 1431 |
+
"kyc_reveal": (
|
| 1432 |
+
"KYC recheck: address verification passed on second attempt — "
|
| 1433 |
+
"customer had entered a unit number incorrectly. Identity document "
|
| 1434 |
+
"authentic. Credit bureau confirms thin file is consistent with "
|
| 1435 |
+
"a first-time credit applicant (age 22). No fraud indicators. "
|
| 1436 |
+
"Approve BNPL transaction."
|
| 1437 |
+
),
|
| 1438 |
+
},
|
| 1439 |
+
# v2: device fingerprint linked to BNPL fraud network → reject
|
| 1440 |
+
{
|
| 1441 |
+
"correct_action": "reject",
|
| 1442 |
+
"partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
|
| 1443 |
+
"kyc_reveal": (
|
| 1444 |
+
"Device fingerprint matches 14 prior BNPL fraud applications across "
|
| 1445 |
+
"3 platforms in the last 6 weeks. All 14 prior accounts defaulted on "
|
| 1446 |
+
"first instalment. Identity documents were synthetically generated. "
|
| 1447 |
+
"Reject and blacklist device fingerprint."
|
| 1448 |
+
),
|
| 1449 |
+
},
|
| 1450 |
+
],
|
| 1451 |
+
|
| 1452 |
# ── HARD ──────────────────────────────────────────────────────────────
|
| 1453 |
|
| 1454 |
"HARD-001": [
|
|
|
|
| 1589 |
"Reject and file SAR immediately."
|
| 1590 |
),
|
| 1591 |
},
|
| 1592 |
+
# v2: estate probate settlement, legitimate reactivation → escalate
|
| 1593 |
{
|
| 1594 |
"correct_action": "escalate",
|
| 1595 |
"partial_credit_actions": {"flag": 0.6, "hold": 0.4},
|
|
|
|
| 1602 |
},
|
| 1603 |
],
|
| 1604 |
|
| 1605 |
+
"HARD-007": [
|
| 1606 |
+
# v1: SIM was ported by legitimate account holder (travelling) → hold
|
| 1607 |
+
{
|
| 1608 |
+
"correct_action": "hold",
|
| 1609 |
+
"partial_credit_actions": {"escalate": 0.5, "reject": 0.4},
|
| 1610 |
+
"inspect_reveal": (
|
| 1611 |
+
"Telecom records: SIM swap was initiated via the carrier's verified "
|
| 1612 |
+
"app using biometric authentication. Account holder is overseas and "
|
| 1613 |
+
"replaced a broken SIM abroad. No third-party involvement detected. "
|
| 1614 |
+
"New withdrawal address is a cold storage wallet previously linked "
|
| 1615 |
+
"to the account. Hold briefly for confirmation call."
|
| 1616 |
+
),
|
| 1617 |
+
"contact_reveal": (
|
| 1618 |
+
"Reached account holder on a backup email. They confirm the SIM swap "
|
| 1619 |
+
"and the withdrawal are legitimate. They are in Japan and broke their "
|
| 1620 |
+
"phone. Hold approved — confirm by secure callback before releasing."
|
| 1621 |
+
),
|
| 1622 |
+
},
|
| 1623 |
+
# v2: no response from account holder, third-party SIM swap confirmed → reject
|
| 1624 |
+
{
|
| 1625 |
+
"correct_action": "reject",
|
| 1626 |
+
"partial_credit_actions": {"hold": 0.4, "escalate": 0.5},
|
| 1627 |
+
"contact_reveal": (
|
| 1628 |
+
"Account holder unreachable on all registered contact methods. "
|
| 1629 |
+
"Carrier fraud team confirmed: SIM swap was initiated via a social "
|
| 1630 |
+
"engineering call to a customer service agent — not by the account "
|
| 1631 |
+
"holder. No biometric authentication used. Third-party SIM swap "
|
| 1632 |
+
"confirmed. Reject and lock account."
|
| 1633 |
+
),
|
| 1634 |
+
},
|
| 1635 |
+
],
|
| 1636 |
+
|
| 1637 |
+
"HARD-008": [
|
| 1638 |
+
# v1: customer is aware it's high-risk investment but not a scam → escalate
|
| 1639 |
+
{
|
| 1640 |
+
"correct_action": "escalate",
|
| 1641 |
+
"partial_credit_actions": {"reject": 0.4, "hold": 0.4},
|
| 1642 |
+
"contact_reveal": (
|
| 1643 |
+
"Customer is fully aware of the risk and has a history of crypto "
|
| 1644 |
+
"investing. The 'romantic partner' is actually a friend from a "
|
| 1645 |
+
"verified online investment forum. Prior transfers were legitimate "
|
| 1646 |
+
"investments that returned profit. However, the escalating pattern "
|
| 1647 |
+
"and offshore destination warrant senior review. Escalate."
|
| 1648 |
+
),
|
| 1649 |
+
"inspect_reveal": (
|
| 1650 |
+
"Transaction history: prior 3 payments were invested and partially "
|
| 1651 |
+
"returned. Receiver fund has a verifiable online presence (2 years). "
|
| 1652 |
+
"However, the most recent dormancy of returns and amount escalation "
|
| 1653 |
+
"match late-stage pig-butchering conversion. Escalate for human review."
|
| 1654 |
+
),
|
| 1655 |
+
},
|
| 1656 |
+
# v2: scam confirmed, customer is distressed victim → reject
|
| 1657 |
+
{
|
| 1658 |
+
"correct_action": "reject",
|
| 1659 |
+
"partial_credit_actions": {"hold": 0.4, "escalate": 0.3},
|
| 1660 |
+
"contact_reveal": (
|
| 1661 |
+
"Customer broke down on the call. They have now been blocked by the "
|
| 1662 |
+
"'romantic partner' and cannot reach them. They realise they have been "
|
| 1663 |
+
"scammed. All prior funds ($5,500) are lost. Reject this transfer, "
|
| 1664 |
+
"file SAR, and refer customer to scam recovery support."
|
| 1665 |
+
),
|
| 1666 |
+
},
|
| 1667 |
+
],
|
| 1668 |
+
|
| 1669 |
+
"HARD-009": [
|
| 1670 |
+
# v1: new tech entrepreneur, thin file is legitimate → approve
|
| 1671 |
+
{
|
| 1672 |
+
"correct_action": "approve",
|
| 1673 |
+
"partial_credit_actions": {"hold": 0.5, "escalate": 0.4},
|
| 1674 |
+
"kyc_reveal": (
|
| 1675 |
+
"KYC deep-dive: EIN issued 11 days ago but LLC was pre-registered "
|
| 1676 |
+
"90 days ago (formation date vs EIN issuance differ — normal for new "
|
| 1677 |
+
"companies). Registered address is a co-working space (common for "
|
| 1678 |
+
"startups). Director SSN cross-check passsed. Wire is to a verified "
|
| 1679 |
+
"software vendor (AWS partner). Thin file is genuine, not synthetic. "
|
| 1680 |
+
"Approve."
|
| 1681 |
+
),
|
| 1682 |
+
"inspect_reveal": (
|
| 1683 |
+
"ML confidence flag was fired due to thin data, not anomalous data. "
|
| 1684 |
+
"No features match synthetic identity profile (all values have "
|
| 1685 |
+
"natural variance). The 'perfect score anomaly' flag was a false "
|
| 1686 |
+
"positive from the rule engine. Safe to approve."
|
| 1687 |
+
),
|
| 1688 |
+
},
|
| 1689 |
+
# v2: identity confirmed as synthetic via government database check → reject
|
| 1690 |
+
{
|
| 1691 |
+
"correct_action": "reject",
|
| 1692 |
+
"partial_credit_actions": {"escalate": 0.4, "hold": 0.3},
|
| 1693 |
+
"kyc_reveal": (
|
| 1694 |
+
"IRS TIN matching service: EIN does not match the business name in "
|
| 1695 |
+
"the IRS database. SSN belongs to a deceased individual. Address is "
|
| 1696 |
+
"a virtual office with no physical occupancy. Synthetic identity "
|
| 1697 |
+
"confirmed. Reject all transactions and report to FinCEN."
|
| 1698 |
+
),
|
| 1699 |
+
},
|
| 1700 |
+
],
|
| 1701 |
+
|
| 1702 |
+
"HARD-010": [
|
| 1703 |
+
# v1: employee did change their account (legitimate, notified HR verbally) → approve
|
| 1704 |
+
{
|
| 1705 |
+
"correct_action": "approve",
|
| 1706 |
+
"partial_credit_actions": {"hold": 0.5, "flag": 0.4},
|
| 1707 |
+
"inspect_reveal": (
|
| 1708 |
+
"HR audit: account change was requested by the employee at HR's "
|
| 1709 |
+
"'walk-in' desk 3 days ago. The change was logged manually by an "
|
| 1710 |
+
"HR admin and entered into the system at 2:10 AM during the overnight "
|
| 1711 |
+
"batch. Unusual timestamp is from the batch processor, not an "
|
| 1712 |
+
"attacker. Legitimate change confirmed."
|
| 1713 |
+
),
|
| 1714 |
+
"contact_reveal": (
|
| 1715 |
+
"Employee confirms: they switched to a new bank and submitted the "
|
| 1716 |
+
"form in person. Was not aware any notification would be sent. "
|
| 1717 |
+
"Batch processing time was confusing but legitimate. Approve."
|
| 1718 |
+
),
|
| 1719 |
+
},
|
| 1720 |
+
# v2: HR credentials were phished, second employee already victimised → reject
|
| 1721 |
+
{
|
| 1722 |
+
"correct_action": "reject",
|
| 1723 |
+
"partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
|
| 1724 |
+
"inspect_reveal": (
|
| 1725 |
+
"HR system audit: same rogue IP (185.220.101.x — known Tor exit node) "
|
| 1726 |
+
"changed direct deposit accounts for 3 employees overnight. Two "
|
| 1727 |
+
"employees have already confirmed they did not make changes. Payroll "
|
| 1728 |
+
"diversion campaign in progress. Reject all affected disbursements "
|
| 1729 |
+
"and freeze HR system access."
|
| 1730 |
+
),
|
| 1731 |
+
},
|
| 1732 |
+
],
|
| 1733 |
+
|
| 1734 |
# ── CRITICAL ──────────────────────────────────────────────────────────
|
| 1735 |
|
| 1736 |
"CRIT-001": [
|
|
|
|
| 1770 |
],
|
| 1771 |
|
| 1772 |
"CRIT-002": [
|
| 1773 |
+
# v1: same SME multiple business accounts — coincidental threshold → hold
|
| 1774 |
{
|
| 1775 |
"correct_action": "hold",
|
| 1776 |
"partial_credit_actions": {"reject": 0.5, "escalate": 0.4},
|
|
|
|
| 1812 |
"Hong Kong and the destination country. No discrepancies."
|
| 1813 |
),
|
| 1814 |
},
|
| 1815 |
+
# v2: criminal-scale TBML → reject
|
| 1816 |
{
|
| 1817 |
"correct_action": "reject",
|
| 1818 |
"partial_credit_actions": {"escalate": 0.5, "flag": 0.3},
|
|
|
|
| 1831 |
],
|
| 1832 |
|
| 1833 |
"CRIT-004": [
|
| 1834 |
+
# v1: CEO delegated to CFO who was in Lagos → hold for confirmation
|
| 1835 |
{
|
| 1836 |
"correct_action": "hold",
|
| 1837 |
"partial_credit_actions": {"escalate": 0.5, "reject": 0.4},
|
|
|
|
| 1847 |
"Hold briefly to verify the papertrail, then release."
|
| 1848 |
),
|
| 1849 |
},
|
| 1850 |
+
# v2: CEO unreachable, audit trail ambiguous → escalate
|
| 1851 |
{
|
| 1852 |
"correct_action": "escalate",
|
| 1853 |
"partial_credit_actions": {"reject": 0.5, "hold": 0.4},
|
|
|
|
| 1865 |
),
|
| 1866 |
},
|
| 1867 |
],
|
| 1868 |
+
|
| 1869 |
+
"CRIT-005": [
|
| 1870 |
+
# v1: shell layers cleared, legitimate commodity trade → approve
|
| 1871 |
+
{
|
| 1872 |
+
"correct_action": "approve",
|
| 1873 |
+
"partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
|
| 1874 |
+
"inspect_reveal": (
|
| 1875 |
+
"UAE FZE registration traced through 3 layers: ultimate beneficial "
|
| 1876 |
+
"owner is a listed European industrial group (Frankfurt Stock Exchange: "
|
| 1877 |
+
"DXI.DE). No SDN or PEP matches. Shell structure is standard for "
|
| 1878 |
+
"legitimate UAE free-zone trading entities. All FATF risk factors "
|
| 1879 |
+
"cleared by AML team. Approve."
|
| 1880 |
+
),
|
| 1881 |
+
"kyc_reveal": (
|
| 1882 |
+
"UBO lookup: al-Rashid FZE is a subsidiary of Deutsche Xport Industries "
|
| 1883 |
+
"AG, a publicly listed entity. No sanctions exposure. KYC pack complete "
|
| 1884 |
+
"and verified. Trade invoice amount matches published commodity prices."
|
| 1885 |
+
),
|
| 1886 |
+
"docs_reveal": (
|
| 1887 |
+
"Trade invoice and shipping manifest verified. HS code 8479.89 matches "
|
| 1888 |
+
"industrial mixing equipment — no dual-use export restrictions apply. "
|
| 1889 |
+
"All documentation clean. Approve."
|
| 1890 |
+
),
|
| 1891 |
+
},
|
| 1892 |
+
# v2: SDN match confirmed at deeper UBO level → reject
|
| 1893 |
+
{
|
| 1894 |
+
"correct_action": "reject",
|
| 1895 |
+
"partial_credit_actions": {"escalate": 0.4, "hold": 0.3},
|
| 1896 |
+
"inspect_reveal": (
|
| 1897 |
+
"Shell chain traced: al-rashid-fze.ae → Gibraltar holding → BVI "
|
| 1898 |
+
"nominee → Tariq Al-Mansouri (OFAC SDN since 2021, Ref SDN-2021-04-19). "
|
| 1899 |
+
"3-layer obfuscation structure is classic sanctions evasion. Reject "
|
| 1900 |
+
"and report to OFAC within 10 business days."
|
| 1901 |
+
),
|
| 1902 |
+
"kyc_reveal": (
|
| 1903 |
+
"UBO confirmed as Tariq Al-Mansouri via FinScan premium SDN database "
|
| 1904 |
+
"cross-reference. Transacting with this UBO violates 31 C.F.R. § 594. "
|
| 1905 |
+
"Reject immediately and file SAR with FinCEN."
|
| 1906 |
+
),
|
| 1907 |
+
},
|
| 1908 |
+
],
|
| 1909 |
+
|
| 1910 |
+
"CRIT-006": [
|
| 1911 |
+
# v1: FinCEN notice is a false positive — different bank with same name → approve
|
| 1912 |
+
{
|
| 1913 |
+
"correct_action": "approve",
|
| 1914 |
+
"partial_credit_actions": {"hold": 0.5, "escalate": 0.4},
|
| 1915 |
+
"inspect_reveal": (
|
| 1916 |
+
"FinCEN 311 action reviewed: the targeted bank is 'Global Partner Bank "
|
| 1917 |
+
"of Belize', ABA 082901987. Our correspondent is 'Partner Bank of Latin "
|
| 1918 |
+
"America, Belize', ABA 082901462 — a distinct entity. The alert was "
|
| 1919 |
+
"triggered by a partial name match. Our correspondent is not subject "
|
| 1920 |
+
"to the 311 measure. Approve with a compliance memo on file."
|
| 1921 |
+
),
|
| 1922 |
+
"docs_reveal": (
|
| 1923 |
+
"SWIFT BIC PTBZBZBZ confirmed as Partner Bank of Latin America — "
|
| 1924 |
+
"no sanctions linkage. Legal confirmed no restriction applies to "
|
| 1925 |
+
"this specific correspondent relationship. Approve this payment and "
|
| 1926 |
+
"update the name-match rule to prevent future false positives."
|
| 1927 |
+
),
|
| 1928 |
+
},
|
| 1929 |
+
# v2: FinCEN 311 applies, must cease all correspondent transactions → escalate
|
| 1930 |
+
{
|
| 1931 |
+
"correct_action": "escalate",
|
| 1932 |
+
"partial_credit_actions": {"hold": 0.5, "reject": 0.4},
|
| 1933 |
+
"inspect_reveal": (
|
| 1934 |
+
"FinCEN 311 Special Measure 5 confirmed as applicable to our "
|
| 1935 |
+
"correspondent (partner-bank-latam.bz, SWIFT PTBZBZBZ). All US "
|
| 1936 |
+
"dollar correspondent and payable-through accounts must be closed "
|
| 1937 |
+
"within 30 days. In-flight transactions require Compliance authorisation. "
|
| 1938 |
+
"Escalate immediately — unilateral rejection may breach our "
|
| 1939 |
+
"contractual wind-down obligations."
|
| 1940 |
+
),
|
| 1941 |
+
"docs_reveal": (
|
| 1942 |
+
"Correspondent agreement clause 12.3 requires 90-day written wind-down "
|
| 1943 |
+
"notice. FinCEN has set a 30-day compliance deadline creating a "
|
| 1944 |
+
"legal conflict. Legal and Compliance must jointly manage the "
|
| 1945 |
+
"resolution. Escalate with full documentation."
|
| 1946 |
+
),
|
| 1947 |
+
},
|
| 1948 |
+
],
|
| 1949 |
}
|
| 1950 |
|
| 1951 |
|
validate.py
CHANGED
|
@@ -150,7 +150,7 @@ def check_openenv_yaml():
|
|
| 150 |
("API_BASE_URL", "API_BASE_URL env var declared"),
|
| 151 |
("MODEL_NAME", "MODEL_NAME env var declared"),
|
| 152 |
("HF_TOKEN", "HF_TOKEN env var declared"),
|
| 153 |
-
("count:
|
| 154 |
("inference.py", "inference script reference"),
|
| 155 |
("approve", "approve action"),
|
| 156 |
("reject", "reject action"),
|
|
|
|
| 150 |
("API_BASE_URL", "API_BASE_URL env var declared"),
|
| 151 |
("MODEL_NAME", "MODEL_NAME env var declared"),
|
| 152 |
("HF_TOKEN", "HF_TOKEN env var declared"),
|
| 153 |
+
("count: 30", "tasks count=30"),
|
| 154 |
("inference.py", "inference script reference"),
|
| 155 |
("approve", "approve action"),
|
| 156 |
("reject", "reject action"),
|