padmapriyagosakan commited on
Commit
ffea7f4
·
1 Parent(s): aa55f7e

Iteration_4: expand task bank 20→30, add variants for all tasks, fix baseline policy for chain-gated CRIT tasks

Browse files
Files changed (6) hide show
  1. README.md +20 -10
  2. openenv.yaml +1 -1
  3. run_tests.sh +79 -29
  4. scripts_util.py +9 -0
  5. tasks.py +855 -11
  6. validate.py +1 -1
README.md CHANGED
@@ -35,7 +35,7 @@ Authorised Push Payment (APP) scams.
35
 
36
  ## Environment Description
37
 
38
- Each **episode** steps through all **20 transactions** (4 easy, 6 medium, 6 hard, 4 critical).
39
  For each transaction the agent observes a rich set of signals and chooses one
40
  of **10 possible actions** — 5 terminal decisions and 5 investigation sub-actions.
41
  A reward is returned immediately, and the next transaction is presented until
@@ -121,8 +121,10 @@ Investigation sub-actions (with budget cost) reveal more information and let the
121
  | EASY-002 | Textbook fraud: unknown sender, offshore, sanctioned country, risk=0.97 | `reject` |
122
  | EASY-003 | Standard refund to verified customer; tiny amount, no flags | `approve` |
123
  | EASY-004 | ATM withdrawal burst — 15 withdrawals in 58 minutes | `flag` |
 
 
124
 
125
- ### Medium (6 tasks — ambiguous, multi-signal reasoning required)
126
 
127
  | ID | Description | Correct Action |
128
  |---------|-------------|----------------|
@@ -132,8 +134,10 @@ Investigation sub-actions (with budget cost) reveal more information and let the
132
  | MED-004 | Payment to licensed crypto exchange from verified personal account | `flag` |
133
  | MED-005 | Expired KYC on high-frequency corporate payroll account; KYC lapsed 12 days ago | `hold` |
134
  | MED-006 | Real estate advance payment; large first-time transfer to new receiver but signed contract exists | `escalate` |
 
 
135
 
136
- ### Hard (6 tasks — adversarial / edge-case)
137
 
138
  | ID | Description | Correct Action |
139
  |----------|-------------|----------------|
@@ -142,16 +146,22 @@ Investigation sub-actions (with budget cost) reveal more information and let the
142
  | HARD-003 | Structuring / smurfing: just-below-CTR-threshold payments, same UBO | `reject` |
143
  | HARD-004 | Legitimate FX correspondent banking settlement — looks alarming, is not | `approve` |
144
  | HARD-005 | Insider threat: employee initiating transfers to personal family accounts | `escalate` |
145
- | HARD-006 | Ghost account: dormant 5 years, suddenly received 20 inbound transfers this week — possible account takeover | `flag` |
 
 
 
 
146
 
147
- ### Critical (4 tasks — regulatory + multi-step investigation chains)
148
 
149
  | ID | Description | Correct Action |
150
  |----------|-------------|----------------|
151
- | CRIT-001 | Multi-step chain: large wire to new counterparty; agent must inspect then request docs before deciding (chain of 3) | `approve` |
152
  | CRIT-002 | Fraud ring: coordinated small payments from 3 related accounts aggregating above reporting threshold; SAR required | `reject` |
153
- | CRIT-003 | Trade-based money laundering: over-invoiced international trade payment (4× market price); regulatory escalation required | `escalate` |
154
  | CRIT-004 | Compromised corporate account: geo-impossible login (NY → Lagos in 8 min); confirmed account takeover | `reject` |
 
 
155
 
156
  ---
157
 
@@ -265,7 +275,7 @@ print(f"Episode score: {score['normalised_score']:.4f}")
265
 
266
  The rule-based baseline uses a deterministic priority-ordered policy in `scripts_util.py`.
267
 
268
- | Metric | Rule-based baseline (v2, 20 tasks) |
269
  |--------|------------------------------------|
270
  | Normalised score | 0.68–0.76 |
271
  | Passed (≥ 0.5) | Yes |
@@ -410,7 +420,7 @@ PYTHONPATH=$(pwd) python payops_env/inference.py
410
  payops_env/
411
  ├── models.py # PayOpsAction, PayOpsObservation, PayOpsState (Pydantic)
412
  ├── environment.py # PayOpsEnvironment — reset_async / step_async / state
413
- ├── tasks.py # 20 tasks (EASY×4, MED×6, HARD×6, CRIT×4) with ground-truth labels
414
  ├── grader.py # Partial-credit reward function + episode grader
415
  ├── scripts_util.py # Baseline runner helper (used by /baseline endpoint)
416
  ├── server/
@@ -430,7 +440,7 @@ payops_env/
430
  | Criterion | Implementation |
431
  |-----------|---------------|
432
  | Real-world utility | Payment fraud and compliance triage — deployed daily by fintech ops teams worldwide |
433
- | Task & grader quality | 20 tasks across 4 difficulty tiers (easy→critical); partial-credit grader; clear pass/fail |
434
  | Environment design | 30-field observation space; 10-action space (5 terminal + 5 investigation); budget mechanic; episode state tracking |
435
  | Code quality & spec compliance | Pydantic v2 models; async API; all 11 required endpoints; openenv.yaml v2; Dockerfile; validate.py |
436
  | Creativity & novelty | Adversarial model-poisoning task; APP scam; AML structuring with SAR requirement; PEP detection |
 
35
 
36
  ## Environment Description
37
 
38
+ Each **episode** steps through all **30 transactions** (6 easy, 8 medium, 10 hard, 6 critical).
39
  For each transaction the agent observes a rich set of signals and chooses one
40
  of **10 possible actions** — 5 terminal decisions and 5 investigation sub-actions.
41
  A reward is returned immediately, and the next transaction is presented until
 
121
  | EASY-002 | Textbook fraud: unknown sender, offshore, sanctioned country, risk=0.97 | `reject` |
122
  | EASY-003 | Standard refund to verified customer; tiny amount, no flags | `approve` |
123
  | EASY-004 | ATM withdrawal burst — 15 withdrawals in 58 minutes | `flag` |
124
+ | EASY-005 | Scheduled monthly mortgage repayment; regular amount, verified borrower | `approve` |
125
+ | EASY-006 | Suspected duplicate payment: same sender/receiver/amount submitted twice in 4 minutes | `flag` |
126
 
127
+ ### Medium (8 tasks — ambiguous, multi-signal reasoning required)
128
 
129
  | ID | Description | Correct Action |
130
  |---------|-------------|----------------|
 
134
  | MED-004 | Payment to licensed crypto exchange from verified personal account | `flag` |
135
  | MED-005 | Expired KYC on high-frequency corporate payroll account; KYC lapsed 12 days ago | `hold` |
136
  | MED-006 | Real estate advance payment; large first-time transfer to new receiver but signed contract exists | `escalate` |
137
+ | MED-007 | Supplier emails to say bank details have changed; first payment to new account matches large invoice (BEC indicator) | `hold` |
138
+ | MED-008 | Buy Now Pay Later high-value purchase; new account, thin credit file, elevated risk signals | `flag` |
139
 
140
+ ### Hard (10 tasks — adversarial / edge-case)
141
 
142
  | ID | Description | Correct Action |
143
  |----------|-------------|----------------|
 
146
  | HARD-003 | Structuring / smurfing: just-below-CTR-threshold payments, same UBO | `reject` |
147
  | HARD-004 | Legitimate FX correspondent banking settlement — looks alarming, is not | `approve` |
148
  | HARD-005 | Insider threat: employee initiating transfers to personal family accounts | `escalate` |
149
+ | HARD-006 | Ghost account: dormant 5 years, suddenly received 20 inbound transfers this week | `flag` |
150
+ | HARD-007 | SIM-swap attack: phone ported 6 hours ago; account now requesting large crypto withdrawal to new address | `reject` |
151
+ | HARD-008 | Romance scam / pig butchering: 4th escalating transfer to overseas 'romantic partner' met online | `reject` |
152
+ | HARD-009 | Synthetic identity fraud: new business account with AI-generated-looking perfect profile | `escalate` |
153
+ | HARD-010 | Payroll diversion: HR system breach rerouted employee salary to newly added account | `reject` |
154
 
155
+ ### Critical (6 tasks — regulatory + multi-step investigation chains)
156
 
157
  | ID | Description | Correct Action |
158
  |----------|-------------|----------------|
159
+ | CRIT-001 | Multi-step chain: large PE wire to new counterparty; inspect then request docs before deciding (chain of 3) | `approve` |
160
  | CRIT-002 | Fraud ring: coordinated small payments from 3 related accounts aggregating above reporting threshold; SAR required | `reject` |
161
+ | CRIT-003 | Trade-based money laundering: over-invoiced international trade payment (4× market price) | `escalate` |
162
  | CRIT-004 | Compromised corporate account: geo-impossible login (NY → Lagos in 8 min); confirmed account takeover | `reject` |
163
+ | CRIT-005 | OFAC sanctions evasion: large USD payment routed through UAE shell chain; UBO is on SDN list (chain of 3) | `reject` |
164
+ | CRIT-006 | Correspondent banking: partner bank added to FinCEN 311 Special Measures list; in-flight payments must be escalated | `escalate` |
165
 
166
  ---
167
 
 
275
 
276
  The rule-based baseline uses a deterministic priority-ordered policy in `scripts_util.py`.
277
 
278
+ | Metric | Rule-based baseline (v2, 30 tasks) |
279
  |--------|------------------------------------|
280
  | Normalised score | 0.68–0.76 |
281
  | Passed (≥ 0.5) | Yes |
 
420
  payops_env/
421
  ├── models.py # PayOpsAction, PayOpsObservation, PayOpsState (Pydantic)
422
  ├── environment.py # PayOpsEnvironment — reset_async / step_async / state
423
+ ├── tasks.py # 30 tasks (EASY×6, MED×8, HARD×10, CRIT×6) with ground-truth labels
424
  ├── grader.py # Partial-credit reward function + episode grader
425
  ├── scripts_util.py # Baseline runner helper (used by /baseline endpoint)
426
  ├── server/
 
440
  | Criterion | Implementation |
441
  |-----------|---------------|
442
  | Real-world utility | Payment fraud and compliance triage — deployed daily by fintech ops teams worldwide |
443
+ | Task & grader quality | 30 tasks across 4 difficulty tiers (easy→critical); partial-credit grader; clear pass/fail |
444
  | Environment design | 30-field observation space; 10-action space (5 terminal + 5 investigation); budget mechanic; episode state tracking |
445
  | Code quality & spec compliance | Pydantic v2 models; async API; all 11 required endpoints; openenv.yaml v2; Dockerfile; validate.py |
446
  | Creativity & novelty | Adversarial model-poisoning task; APP scam; AML structuring with SAR requirement; PEP detection |
openenv.yaml CHANGED
@@ -114,7 +114,7 @@ observation_space:
114
 
115
  # ── Tasks ─────────────────────────────────────────────────────────────────────
116
  tasks:
117
- count: 20
118
  difficulties:
119
  - easy
120
  - medium
 
114
 
115
  # ── Tasks ─────────────────────────────────────────────────────────────────────
116
  tasks:
117
+ count: 30
118
  difficulties:
119
  - easy
120
  - medium
run_tests.sh CHANGED
@@ -97,9 +97,9 @@ section "GROUP B — Tasks endpoint"
97
 
98
  TASKS_RESP="$(curl -s $BASE/tasks)"
99
 
100
- # B-01 20 tasks
101
- check "B-01 GET /tasks → count=20" \
102
- "$TASKS_RESP" '"count":20'
103
 
104
  # B-02 All 4 difficulty tiers present
105
  check "B-02 Tasks include 'easy' tier" "$TASKS_RESP" '"difficulty":"easy"'
@@ -159,6 +159,16 @@ check "D-04 EASY-004 flag → reward=1.0" \
159
  "$(step '{"action_type":"flag","transaction_id":"TXN-E004"}')" \
160
  '"reward":1.0'
161
 
 
 
 
 
 
 
 
 
 
 
162
  # D-05 escalate correct → reward=1.0
163
  check "D-05 MED-001 escalate → reward=1.0" \
164
  "$(step '{"action_type":"escalate","transaction_id":"TXN-M001"}')" \
@@ -202,6 +212,8 @@ step '{"action_type":"approve","transaction_id":"TXN-E001"}' > /dev/null
202
  step '{"action_type":"reject","transaction_id":"TXN-E002"}' > /dev/null
203
  step '{"action_type":"approve","transaction_id":"TXN-E003"}' > /dev/null
204
  step '{"action_type":"flag","transaction_id":"TXN-E004"}' > /dev/null
 
 
205
  R=$(step '{"action_type":"flag","transaction_id":"TXN-M001"}')
206
  check "E-03 Partial credit: flag on MED-001 (correct=escalate) → reward > 0" \
207
  "$R" '"reward":0\.[0-9]'
@@ -295,11 +307,18 @@ check "H-07 GET /grader → passed field present" "$GRADER" '"passed":'
295
  # ═══════════════════════════════════════════════════════════
296
  section "GROUP I — /replay endpoint (offline eval)"
297
 
298
- # I-01 Replay with all correct actions score=1.0
299
- REPLAY_PERFECT='{"actions":["approve","reject","approve","flag",
300
- "escalate","hold","flag","flag","hold","escalate",
301
- "escalate","reject","reject","approve","escalate","flag",
302
- "approve","reject","escalate","reject"]}'
 
 
 
 
 
 
 
303
  R=$(curl -s -X POST $BASE/replay \
304
  -H "Content-Type: application/json" \
305
  -d "$REPLAY_PERFECT")
@@ -307,11 +326,11 @@ check "I-01 Replay correct terminals (no investigation) → score<1.0 (hard/cri
307
  check "I-02 Replay → passed=true (score>0.5 despite penalty)" "$R" '"passed":true'
308
  check "I-03 Replay → budget_spent=0.0 (no inv actions)" "$R" '"budget_spent":0.0'
309
 
310
- # I-04 Replay with investigation actions included
311
- REPLAY_WITH_INV='{"actions":["inspect","approve","reject","approve","flag",
312
- "escalate","hold","flag","flag","hold","escalate",
313
- "escalate","reject","reject","approve","escalate","flag",
314
- "approve","reject","escalate","reject"]}'
315
  R=$(curl -s -X POST $BASE/replay \
316
  -H "Content-Type: application/json" \
317
  -d "$REPLAY_WITH_INV")
@@ -370,16 +389,33 @@ def get(path):
370
  with urllib.request.urlopen(f"{BASE}{path}", context=_ssl) as r: return json.loads(r.read())
371
 
372
  post("/reset", {"seed": 0})
373
- actions = [
 
374
  ("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
 
375
  ("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
376
- ("hold","TXN-M005"),("escalate","TXN-M006"),
377
  ("escalate","TXN-H001"),("reject","TXN-H002"),("reject","TXN-H003"),("approve","TXN-H004"),
378
- ("escalate","TXN-H005"),("flag","TXN-H006"),
379
- ("approve","TXN-C001"),("reject","TXN-C002"),("escalate","TXN-C003"),("reject","TXN-C004"),
380
- ]
381
- for action, txn in actions:
382
  post("/step", {"action_type": action, "transaction_id": txn})
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
383
  PYEOF
384
 
385
  ANA="$(curl -s $BASE/analytics)"
@@ -408,7 +444,7 @@ check "L-06 GET /leaderboard → budget_spent in entry" "$LB" '"budget_spent":'
408
  # ═══════════════════════════════════════════════════════════
409
  # GROUP M — Perfect episode score
410
  # ═══════════════════════════════════════════════════════════
411
- section "GROUP M — Perfect episode (all 20 correct + investigation on hard/critical)"
412
 
413
  reset
414
  BASE=$BASE python3 - <<'PYEOF'
@@ -424,14 +460,15 @@ def post(path, body=None):
424
  with urllib.request.urlopen(req, context=_ssl) as r: return json.loads(r.read())
425
 
426
  post("/reset", {"seed": 0})
427
- # Easy + medium: terminal only
428
  for action, txn in [
429
  ("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
 
430
  ("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
431
- ("hold","TXN-M005"),("escalate","TXN-M006"),
432
  ]:
433
  post("/step", {"action_type": action, "transaction_id": txn})
434
- # Hard tasks: one required investigation sub-action before each terminal
435
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-H001"})
436
  post("/step", {"action_type": "escalate", "transaction_id": "TXN-H001"})
437
  post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H002"})
@@ -445,7 +482,15 @@ post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H005"})
445
  post("/step", {"action_type": "escalate", "transaction_id": "TXN-H005"})
446
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-H006"})
447
  post("/step", {"action_type": "flag", "transaction_id": "TXN-H006"})
448
- # Critical tasks: required investigation sub-actions before each terminal
 
 
 
 
 
 
 
 
449
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-C001"})
450
  post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C001"})
451
  post("/step", {"action_type": "approve", "transaction_id": "TXN-C001"})
@@ -457,6 +502,11 @@ post("/step", {"action_type": "escalate", "transaction_id": "TXN-C003"})
457
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-C004"})
458
  post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-C004"})
459
  post("/step", {"action_type": "reject", "transaction_id": "TXN-C004"})
 
 
 
 
 
460
  PYEOF
461
 
462
  GRADER="$(curl -s $BASE/grader)"
@@ -470,16 +520,16 @@ check "M-03 Perfect episode → budget_penalty=0.0" "$GRADER" '"budget_penalty"
470
  section "GROUP N — Difficulty weighting in grader"
471
 
472
  # A critical task correct should contribute more weight than easy
473
- REPLAY_CRIT='{"actions":["approve","reject","approve","flag",
474
- "escalate","hold","flag","flag","hold","escalate",
475
- "escalate","reject","reject","approve","escalate","flag",
476
- "approve","reject","escalate","reject"]}'
477
 
478
  FULL=$(curl -s -X POST $BASE/replay \
479
  -H "Content-Type: application/json" \
480
  -d "$REPLAY_CRIT")
481
  check "N-01 Full correct replay → max_possible_reward includes weights" \
482
- "$FULL" '"max_possible_reward":2[0-9]\.'
483
  check "N-02 Correct-but-no-investigation → total_reward below max (penalty applied)" \
484
  "$(echo "$FULL" | python3 -c "
485
  import sys,json,re
 
97
 
98
  TASKS_RESP="$(curl -s $BASE/tasks)"
99
 
100
+ # B-01 30 tasks
101
+ check "B-01 GET /tasks → count=30" \
102
+ "$TASKS_RESP" '"count":30'
103
 
104
  # B-02 All 4 difficulty tiers present
105
  check "B-02 Tasks include 'easy' tier" "$TASKS_RESP" '"difficulty":"easy"'
 
159
  "$(step '{"action_type":"flag","transaction_id":"TXN-E004"}')" \
160
  '"reward":1.0'
161
 
162
+ # D-04b approve correct (mortgage repayment) → reward=1.0
163
+ check "D-04b EASY-005 approve → reward=1.0" \
164
+ "$(step '{"action_type":"approve","transaction_id":"TXN-E005"}')" \
165
+ '"reward":1.0'
166
+
167
+ # D-04c flag correct (duplicate payment) → reward=1.0
168
+ check "D-04c EASY-006 flag → reward=1.0" \
169
+ "$(step '{"action_type":"flag","transaction_id":"TXN-E006"}')" \
170
+ '"reward":1.0'
171
+
172
  # D-05 escalate correct → reward=1.0
173
  check "D-05 MED-001 escalate → reward=1.0" \
174
  "$(step '{"action_type":"escalate","transaction_id":"TXN-M001"}')" \
 
212
  step '{"action_type":"reject","transaction_id":"TXN-E002"}' > /dev/null
213
  step '{"action_type":"approve","transaction_id":"TXN-E003"}' > /dev/null
214
  step '{"action_type":"flag","transaction_id":"TXN-E004"}' > /dev/null
215
+ step '{"action_type":"approve","transaction_id":"TXN-E005"}' > /dev/null
216
+ step '{"action_type":"flag","transaction_id":"TXN-E006"}' > /dev/null
217
  R=$(step '{"action_type":"flag","transaction_id":"TXN-M001"}')
218
  check "E-03 Partial credit: flag on MED-001 (correct=escalate) → reward > 0" \
219
  "$R" '"reward":0\.[0-9]'
 
307
  # ═══════════════════════════════════════════════════════════
308
  section "GROUP I — /replay endpoint (offline eval)"
309
 
310
+ # I-01 Replay with all 30 correct terminal actions (no investigation)
311
+ # Hard/critical tasks should be penalised (×0.80) → 0.7–0.9 range
312
+ # Task order: EASY×6, MED×8, HARD×10, CRIT×6
313
+ # Correct actions:
314
+ # EASY: approve reject approve flag approve flag
315
+ # MED: escalate hold flag flag hold escalate hold flag
316
+ # HARD: escalate reject reject approve escalate flag reject reject escalate reject
317
+ # CRIT: approve reject escalate reject reject escalate
318
+ REPLAY_PERFECT='{"actions":["approve","reject","approve","flag","approve","flag",
319
+ "escalate","hold","flag","flag","hold","escalate","hold","flag",
320
+ "escalate","reject","reject","approve","escalate","flag","reject","reject","escalate","reject",
321
+ "approve","reject","escalate","reject","reject","escalate"]}'
322
  R=$(curl -s -X POST $BASE/replay \
323
  -H "Content-Type: application/json" \
324
  -d "$REPLAY_PERFECT")
 
326
  check "I-02 Replay → passed=true (score>0.5 despite penalty)" "$R" '"passed":true'
327
  check "I-03 Replay → budget_spent=0.0 (no inv actions)" "$R" '"budget_spent":0.0'
328
 
329
+ # I-04 Replay with investigation actions included (inspect before first task)
330
+ REPLAY_WITH_INV='{"actions":["inspect","approve","reject","approve","flag","approve","flag",
331
+ "escalate","hold","flag","flag","hold","escalate","hold","flag",
332
+ "escalate","reject","reject","approve","escalate","flag","reject","reject","escalate","reject",
333
+ "approve","reject","escalate","reject","reject","escalate"]}'
334
  R=$(curl -s -X POST $BASE/replay \
335
  -H "Content-Type: application/json" \
336
  -d "$REPLAY_WITH_INV")
 
389
  with urllib.request.urlopen(f"{BASE}{path}", context=_ssl) as r: return json.loads(r.read())
390
 
391
  post("/reset", {"seed": 0})
392
+ # EASY x6 + MED x8 + HARD x10: no chain gate, terminal only
393
+ for action, txn in [
394
  ("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
395
+ ("approve","TXN-E005"),("flag","TXN-E006"),
396
  ("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
397
+ ("hold","TXN-M005"),("escalate","TXN-M006"),("hold","TXN-M007"),("flag","TXN-M008"),
398
  ("escalate","TXN-H001"),("reject","TXN-H002"),("reject","TXN-H003"),("approve","TXN-H004"),
399
+ ("escalate","TXN-H005"),("flag","TXN-H006"),("reject","TXN-H007"),("reject","TXN-H008"),
400
+ ("escalate","TXN-H009"),("reject","TXN-H010"),
401
+ ]:
 
402
  post("/step", {"action_type": action, "transaction_id": txn})
403
+ # CRIT x6: chain-gated — must provide chain_min investigation steps first
404
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C001"})
405
+ post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C001"})
406
+ post("/step", {"action_type": "approve", "transaction_id": "TXN-C001"})
407
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C002"})
408
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-C002"})
409
+ post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C003"})
410
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C003"})
411
+ post("/step", {"action_type": "escalate", "transaction_id": "TXN-C003"})
412
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C004"})
413
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-C004"})
414
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C005"})
415
+ post("/step", {"action_type": "verify_kyc", "transaction_id": "TXN-C005"})
416
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-C005"})
417
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C006"})
418
+ post("/step", {"action_type": "escalate", "transaction_id": "TXN-C006"})
419
  PYEOF
420
 
421
  ANA="$(curl -s $BASE/analytics)"
 
444
  # ═══════════════════════════════════════════════════════════
445
  # GROUP M — Perfect episode score
446
  # ═══════════════════════════════════════════════════════════
447
+ section "GROUP M — Perfect episode (all 30 correct + investigation on hard/critical)"
448
 
449
  reset
450
  BASE=$BASE python3 - <<'PYEOF'
 
460
  with urllib.request.urlopen(req, context=_ssl) as r: return json.loads(r.read())
461
 
462
  post("/reset", {"seed": 0})
463
+ # Easy x6 + Medium x8: terminal only
464
  for action, txn in [
465
  ("approve","TXN-E001"),("reject","TXN-E002"),("approve","TXN-E003"),("flag","TXN-E004"),
466
+ ("approve","TXN-E005"),("flag","TXN-E006"),
467
  ("escalate","TXN-M001"),("hold","TXN-M002"),("flag","TXN-M003"),("flag","TXN-M004"),
468
+ ("hold","TXN-M005"),("escalate","TXN-M006"),("hold","TXN-M007"),("flag","TXN-M008"),
469
  ]:
470
  post("/step", {"action_type": action, "transaction_id": txn})
471
+ # Hard x10: one required investigation sub-action before each terminal
472
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-H001"})
473
  post("/step", {"action_type": "escalate", "transaction_id": "TXN-H001"})
474
  post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H002"})
 
482
  post("/step", {"action_type": "escalate", "transaction_id": "TXN-H005"})
483
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-H006"})
484
  post("/step", {"action_type": "flag", "transaction_id": "TXN-H006"})
485
+ post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H007"})
486
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-H007"})
487
+ post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H008"})
488
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-H008"})
489
+ post("/step", {"action_type": "verify_kyc", "transaction_id": "TXN-H009"})
490
+ post("/step", {"action_type": "escalate", "transaction_id": "TXN-H009"})
491
+ post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-H010"})
492
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-H010"})
493
+ # Critical x6: required investigation sub-actions before each terminal
494
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-C001"})
495
  post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C001"})
496
  post("/step", {"action_type": "approve", "transaction_id": "TXN-C001"})
 
502
  post("/step", {"action_type": "inspect", "transaction_id": "TXN-C004"})
503
  post("/step", {"action_type": "contact_sender", "transaction_id": "TXN-C004"})
504
  post("/step", {"action_type": "reject", "transaction_id": "TXN-C004"})
505
+ post("/step", {"action_type": "inspect", "transaction_id": "TXN-C005"})
506
+ post("/step", {"action_type": "verify_kyc", "transaction_id": "TXN-C005"})
507
+ post("/step", {"action_type": "reject", "transaction_id": "TXN-C005"})
508
+ post("/step", {"action_type": "request_docs", "transaction_id": "TXN-C006"})
509
+ post("/step", {"action_type": "escalate", "transaction_id": "TXN-C006"})
510
  PYEOF
511
 
512
  GRADER="$(curl -s $BASE/grader)"
 
520
  section "GROUP N — Difficulty weighting in grader"
521
 
522
  # A critical task correct should contribute more weight than easy
523
+ REPLAY_CRIT='{"actions":["approve","reject","approve","flag","approve","flag",
524
+ "escalate","hold","flag","flag","hold","escalate","hold","flag",
525
+ "escalate","reject","reject","approve","escalate","flag","reject","reject","escalate","reject",
526
+ "approve","reject","escalate","reject","reject","escalate"]}'
527
 
528
  FULL=$(curl -s -X POST $BASE/replay \
529
  -H "Content-Type: application/json" \
530
  -d "$REPLAY_CRIT")
531
  check "N-01 Full correct replay → max_possible_reward includes weights" \
532
+ "$FULL" '"max_possible_reward":4[0-9]\.'
533
  check "N-02 Correct-but-no-investigation → total_reward below max (penalty applied)" \
534
  "$(echo "$FULL" | python3 -c "
535
  import sys,json,re
scripts_util.py CHANGED
@@ -77,6 +77,15 @@ def _should_investigate(obs) -> Optional[str]:
77
  if (ml_conf < 0.60 or watchlist_hit) and "inspect" not in already:
78
  return "inspect"
79
 
 
 
 
 
 
 
 
 
 
80
  return None
81
 
82
 
 
77
  if (ml_conf < 0.60 or watchlist_hit) and "inspect" not in already:
78
  return "inspect"
79
 
80
+ # Fallback for chain-gated tasks (chain_total > 1): if not enough investigation
81
+ # steps have been done yet, issue generic actions in priority order so the
82
+ # baseline agent never gets stuck in an infinite chain-gate loop.
83
+ chain_min = max(0, getattr(obs, "chain_total", 1) - 1)
84
+ if chain_min > 0 and len(already) < chain_min:
85
+ for inv_action in ("inspect", "verify_kyc", "request_docs", "contact_sender", "file_sar"):
86
+ if inv_action not in already:
87
+ return inv_action
88
+
89
  return None
90
 
91
 
tasks.py CHANGED
@@ -206,11 +206,57 @@ EASY_TASKS: List[PayOpsTask] = [
206
  inspect_reveal="15 ATM withdrawals in 58 min across 4 ATMs. Pattern consistent with card clone.",
207
  key_flags=["velocity_breach"],
208
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
209
  ]
210
 
211
 
212
  # =============================================================================
213
- # MEDIUM (6 tasks — ambiguous, multi-signal)
214
  # =============================================================================
215
 
216
  MEDIUM_TASKS: List[PayOpsTask] = [
@@ -356,11 +402,66 @@ MEDIUM_TASKS: List[PayOpsTask] = [
356
  docs_reveal="Signed purchase agreement found. Escrow agent licensed and registered. Standard conveyancing practice.",
357
  key_flags=["first_time_payee", "large_first_transfer"],
358
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
359
  ]
360
 
361
 
362
  # =============================================================================
363
- # HARD (6 tasks — adversarial, conflicting, edge-cases)
364
  # =============================================================================
365
 
366
  HARD_TASKS: List[PayOpsTask] = [
@@ -540,11 +641,161 @@ HARD_TASKS: List[PayOpsTask] = [
540
  ),
541
  key_flags=["dormant_receiver", "sudden_activity"],
542
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
543
  ]
544
 
545
 
546
  # =============================================================================
547
- # CRITICAL (4 tasks — multi-step chains, regulatory stakes)
548
  # =============================================================================
549
 
550
  CRITICAL_TASKS: List[PayOpsTask] = [
@@ -673,6 +924,103 @@ CRITICAL_TASKS: List[PayOpsTask] = [
673
  contact_reveal="CEO confirms they did NOT initiate this transfer. Account takeover confirmed.",
674
  key_flags=["geo_impossible_login", "account_takeover_indicator"],
675
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
676
  ]
677
 
678
 
@@ -715,10 +1063,233 @@ ACTION_COSTS: Dict[str, float] = {
715
  # ---------------------------------------------------------------------------
716
  TASK_VARIANTS: Dict[str, List[Dict]] = {
717
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
718
  # ── MEDIUM ────────────────────────────────────────────────────────────
719
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
720
  "MED-003": [
721
- # v1: unauthorised recurring billing → hold to freeze
722
  {
723
  "correct_action": "hold",
724
  "partial_credit_actions": {"flag": 0.5, "escalate": 0.3},
@@ -748,8 +1319,8 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
748
  "partial_credit_actions": {"flag": 0.5, "hold": 0.3},
749
  "inspect_reveal": (
750
  "Sender has made 7 crypto exchange payments in 5 days totalling $28k "
751
- "across 3 different platforms. Amounts stay just below exchange reporting "
752
- "limits. Pattern matches structuring. Escalate for AML review."
753
  ),
754
  },
755
  # v2: clean recurring investment → approve
@@ -815,6 +1386,69 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
815
  },
816
  ],
817
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
818
  # ── HARD ──────────────────────────────────────────────────────────────
819
 
820
  "HARD-001": [
@@ -955,7 +1589,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
955
  "Reject and file SAR immediately."
956
  ),
957
  },
958
- # v2: estate probate settlement, legitimate size → escalate
959
  {
960
  "correct_action": "escalate",
961
  "partial_credit_actions": {"flag": 0.6, "hold": 0.4},
@@ -968,6 +1602,135 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
968
  },
969
  ],
970
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
971
  # ── CRITICAL ──────────────────────────────────────────────────────────
972
 
973
  "CRIT-001": [
@@ -1007,7 +1770,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
1007
  ],
1008
 
1009
  "CRIT-002": [
1010
- # v1: same SME, multiple business accounts — coincidental threshold → hold
1011
  {
1012
  "correct_action": "hold",
1013
  "partial_credit_actions": {"reject": 0.5, "escalate": 0.4},
@@ -1049,7 +1812,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
1049
  "Hong Kong and the destination country. No discrepancies."
1050
  ),
1051
  },
1052
- # v2: criminal-scale TBML, reject immediately
1053
  {
1054
  "correct_action": "reject",
1055
  "partial_credit_actions": {"escalate": 0.5, "flag": 0.3},
@@ -1068,7 +1831,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
1068
  ],
1069
 
1070
  "CRIT-004": [
1071
- # v1: CEO delegated to CFO who was in Lagos hold for confirmation
1072
  {
1073
  "correct_action": "hold",
1074
  "partial_credit_actions": {"escalate": 0.5, "reject": 0.4},
@@ -1084,7 +1847,7 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
1084
  "Hold briefly to verify the papertrail, then release."
1085
  ),
1086
  },
1087
- # v2: CEO unreachable, dual-approval controls met, escalate
1088
  {
1089
  "correct_action": "escalate",
1090
  "partial_credit_actions": {"reject": 0.5, "hold": 0.4},
@@ -1102,6 +1865,87 @@ TASK_VARIANTS: Dict[str, List[Dict]] = {
1102
  ),
1103
  },
1104
  ],
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1105
  }
1106
 
1107
 
 
206
  inspect_reveal="15 ATM withdrawals in 58 min across 4 ATMs. Pattern consistent with card clone.",
207
  key_flags=["velocity_breach"],
208
  ),
209
+ PayOpsTask(
210
+ task_id="EASY-005",
211
+ difficulty="easy",
212
+ description="Scheduled monthly mortgage repayment. Regular amount, verified on-time borrower, no flags.",
213
+ transaction_id="TXN-E005",
214
+ amount=1_247.00,
215
+ currency="USD",
216
+ sender="john.doe@homeowner.com",
217
+ receiver="mortgages@national-bank.com",
218
+ transaction_type="loan_repayment",
219
+ risk_score=0.03,
220
+ ml_confidence=0.99,
221
+ flags=[],
222
+ velocity_1h=1, velocity_24h=1,
223
+ avg_transaction_amount=1_247.00,
224
+ account_age_days=1_825,
225
+ country_risk="low", kyc_status="verified", kyc_expiry_days=280,
226
+ previous_violations=0, previous_sars=0, counterparty_risk="clean",
227
+ correct_action="approve",
228
+ partial_credit_actions={},
229
+ inspect_reveal="Loan account in good standing. Payment 22 of 360. Amount matches amortization schedule exactly.",
230
+ key_flags=[],
231
+ ),
232
+ PayOpsTask(
233
+ task_id="EASY-006",
234
+ difficulty="easy",
235
+ description="Suspected duplicate payment: same sender, receiver, and amount submitted twice within 4 minutes.",
236
+ transaction_id="TXN-E006",
237
+ amount=523.50,
238
+ currency="USD",
239
+ sender="vendor@supplies-ltd.com",
240
+ receiver="buyer@procurement.com",
241
+ transaction_type="payment",
242
+ risk_score=0.55,
243
+ ml_confidence=0.91,
244
+ flags=["duplicate_transaction", "velocity_breach"],
245
+ velocity_1h=2, velocity_24h=3,
246
+ avg_transaction_amount=520.0,
247
+ account_age_days=980,
248
+ country_risk="low", kyc_status="verified", kyc_expiry_days=150,
249
+ previous_violations=0, previous_sars=0, counterparty_risk="clean",
250
+ correct_action="flag",
251
+ partial_credit_actions={"hold": 0.6},
252
+ inspect_reveal="TXN-E006-prev processed 4 min ago for identical amount and beneficiary. Duplicate submission detected — flag for double-charge review before releasing.",
253
+ key_flags=["duplicate_transaction"],
254
+ ),
255
  ]
256
 
257
 
258
  # =============================================================================
259
+ # MEDIUM (8 tasks — ambiguous, multi-signal)
260
  # =============================================================================
261
 
262
  MEDIUM_TASKS: List[PayOpsTask] = [
 
402
  docs_reveal="Signed purchase agreement found. Escrow agent licensed and registered. Standard conveyancing practice.",
403
  key_flags=["first_time_payee", "large_first_transfer"],
404
  ),
405
+ PayOpsTask(
406
+ task_id="MED-007",
407
+ difficulty="medium",
408
+ description=(
409
+ "Supplier emails to say bank details have changed; first payment to the "
410
+ "new account matches a large outstanding invoice. Classic BEC indicator."
411
+ ),
412
+ transaction_id="TXN-M007",
413
+ amount=43_800.00,
414
+ currency="EUR",
415
+ sender="accounts@client-corp.de",
416
+ receiver="supplier-newacct@euro-clearing.eu",
417
+ transaction_type="payment",
418
+ risk_score=0.63,
419
+ ml_confidence=0.67,
420
+ flags=["bank_detail_change", "first_time_payee", "high_value"],
421
+ velocity_1h=1, velocity_24h=1,
422
+ avg_transaction_amount=44_000.0,
423
+ account_age_days=1_200,
424
+ country_risk="medium", kyc_status="verified", kyc_expiry_days=130,
425
+ previous_violations=0, previous_sars=0, counterparty_risk="unknown",
426
+ correct_action="hold",
427
+ partial_credit_actions={"escalate": 0.6, "flag": 0.4},
428
+ requires_investigation={"request_docs", "contact_sender"},
429
+ docs_reveal="Email requesting bank change was sent from supplier-noreply@euro-cIearing.eu (capital I, not l). No signed bank-change authorisation on file. Possible BEC.",
430
+ contact_reveal="Called original supplier contact number on record. They confirmed they did NOT send a bank-change notice. BEC fraud confirmed — hold and alert client.",
431
+ key_flags=["bank_detail_change", "first_time_payee"],
432
+ ),
433
+ PayOpsTask(
434
+ task_id="MED-008",
435
+ difficulty="medium",
436
+ description=(
437
+ "Buy Now Pay Later purchase: high-value electronics order, account opened "
438
+ "3 weeks ago, thin credit file, elevated risk signals."
439
+ ),
440
+ transaction_id="TXN-M008",
441
+ amount=2_899.00,
442
+ currency="USD",
443
+ sender="bnpl-platform@payin4.io",
444
+ receiver="electronics@megastore.com",
445
+ transaction_type="payment",
446
+ risk_score=0.60,
447
+ ml_confidence=0.71,
448
+ flags=["thin_credit_file", "high_value_bnpl", "new_account"],
449
+ velocity_1h=1, velocity_24h=2,
450
+ avg_transaction_amount=None,
451
+ account_age_days=21,
452
+ country_risk="low", kyc_status="pending", kyc_expiry_days=None,
453
+ previous_violations=0, previous_sars=0, counterparty_risk="clean",
454
+ correct_action="flag",
455
+ partial_credit_actions={"hold": 0.6, "escalate": 0.4},
456
+ requires_investigation={"verify_kyc"},
457
+ kyc_reveal="KYC check: identity document submitted but address verification failed — billing address does not match ID. BNPL fraud profile. Flag for manual identity review before releasing funds.",
458
+ key_flags=["thin_credit_file", "new_account"],
459
+ ),
460
  ]
461
 
462
 
463
  # =============================================================================
464
+ # HARD (10 tasks — adversarial, conflicting, edge-cases)
465
  # =============================================================================
466
 
467
  HARD_TASKS: List[PayOpsTask] = [
 
641
  ),
642
  key_flags=["dormant_receiver", "sudden_activity"],
643
  ),
644
+ PayOpsTask(
645
+ task_id="HARD-007",
646
+ difficulty="hard",
647
+ description=(
648
+ "SIM-swap attack: account holder's phone number was ported to a new SIM "
649
+ "6 hours ago; account now requesting a large crypto withdrawal to a new address."
650
+ ),
651
+ transaction_id="TXN-H007",
652
+ amount=14_200.00,
653
+ currency="USD",
654
+ sender="crypto-user@exchange.io",
655
+ receiver="external-wallet-9f3a@coldwallet.net",
656
+ transaction_type="withdrawal",
657
+ risk_score=0.74,
658
+ ml_confidence=0.78,
659
+ flags=["sim_swap_detected", "new_withdrawal_address", "high_value"],
660
+ velocity_1h=1, velocity_24h=1,
661
+ avg_transaction_amount=500.0,
662
+ account_age_days=620,
663
+ country_risk="low", kyc_status="verified", kyc_expiry_days=195,
664
+ previous_violations=0, previous_sars=0, counterparty_risk="unknown",
665
+ correct_action="reject",
666
+ partial_credit_actions={"hold": 0.5, "escalate": 0.4, "flag": 0.3},
667
+ requires_investigation={"inspect", "contact_sender"},
668
+ inspect_reveal=(
669
+ "Telecom provider confirmed: SIM swap on this account's registered phone "
670
+ "completed 6 h 12 min ago. External wallet address first appeared in this "
671
+ "session only. IP geolocation mismatch vs prior 90-day baseline."
672
+ ),
673
+ contact_reveal=(
674
+ "Attempted call to registered phone: intercepted by new SIM holder, "
675
+ "who provided all correct security answers (likely obtained via social "
676
+ "engineering). Account holder unreachable on backup email. SIM-swap "
677
+ "takeover confirmed — reject and lock account."
678
+ ),
679
+ key_flags=["sim_swap_detected", "new_withdrawal_address"],
680
+ ),
681
+ PayOpsTask(
682
+ task_id="HARD-008",
683
+ difficulty="hard",
684
+ description=(
685
+ "Romance scam / pig butchering: customer making 4th escalating transfer "
686
+ "to an overseas 'romantic partner' met on a dating app 7 weeks ago. "
687
+ "Amounts have doubled each time."
688
+ ),
689
+ transaction_id="TXN-H008",
690
+ amount=8_000.00,
691
+ currency="USD",
692
+ sender="victim@personal-bank.com",
693
+ receiver="crypto-invest@hongkong-fund.net",
694
+ transaction_type="transfer",
695
+ risk_score=0.66,
696
+ ml_confidence=0.73,
697
+ flags=["romance_scam_indicator", "escalating_transfers", "offshore_receiver"],
698
+ velocity_1h=1, velocity_24h=1,
699
+ avg_transaction_amount=1_200.0,
700
+ account_age_days=4_015,
701
+ country_risk="high", kyc_status="verified", kyc_expiry_days=220,
702
+ previous_violations=0, previous_sars=0, counterparty_risk="blacklist",
703
+ correct_action="reject",
704
+ partial_credit_actions={"hold": 0.5, "escalate": 0.4, "flag": 0.3},
705
+ requires_investigation={"contact_sender", "inspect"},
706
+ contact_reveal=(
707
+ "Spoke with account holder. They believe they are investing with a "
708
+ "partner they met on a dating app. Prior transfers: $500, $1,000, $4,000. "
709
+ "Receiver domain registered 3 months ago. Pig-butchering romance scam confirmed."
710
+ ),
711
+ inspect_reveal=(
712
+ "Transfer history: $500 → $1,000 → $4,000 → $8,000 over 7 weeks. "
713
+ "Receiver account linked to 23 other inbound romance-scam reports in the "
714
+ "financial intelligence database. All prior transfers drained offshore within hours."
715
+ ),
716
+ key_flags=["romance_scam_indicator", "escalating_transfers"],
717
+ network_graph={"victim": "victim@personal-bank.com", "scammer_fund": "crypto-invest@hongkong-fund.net"},
718
+ ),
719
+ PayOpsTask(
720
+ task_id="HARD-009",
721
+ difficulty="hard",
722
+ description=(
723
+ "Synthetic identity fraud: new business account with a flawless-looking "
724
+ "profile (perfect score, clean history) requests a large first outbound "
725
+ "wire within 10 days of opening."
726
+ ),
727
+ transaction_id="TXN-H009",
728
+ amount=19_500.00,
729
+ currency="USD",
730
+ sender="newco-llc@startup-bank.com",
731
+ receiver="vendor@offshore-services.biz",
732
+ transaction_type="transfer",
733
+ risk_score=0.35,
734
+ ml_confidence=0.40, # low confidence: model flagging identity inconsistency
735
+ flags=["new_account_10d", "perfect_score_anomaly", "large_first_wire"],
736
+ velocity_1h=1, velocity_24h=1,
737
+ avg_transaction_amount=None,
738
+ account_age_days=10,
739
+ country_risk="medium", kyc_status="verified", kyc_expiry_days=360,
740
+ previous_violations=0, previous_sars=0, counterparty_risk="unknown",
741
+ correct_action="escalate",
742
+ partial_credit_actions={"hold": 0.5, "flag": 0.4, "reject": 0.3},
743
+ requires_investigation={"verify_kyc", "inspect"},
744
+ kyc_reveal=(
745
+ "KYC documents re-examined: EIN was issued 11 days ago but the 'years "
746
+ "in business' field on the application states 4 years. Registered address "
747
+ "is a UPS store mailbox. Director SSN belongs to a real person with no "
748
+ "knowledge of this LLC. Synthetic identity confirmed."
749
+ ),
750
+ inspect_reveal=(
751
+ "Account ML confidence flagged as low: identity features are statistically "
752
+ "perfect (no noise in credit history, address, DOB fields) — consistent "
753
+ "with synthetically generated identity. Escalate for manual identity verification."
754
+ ),
755
+ key_flags=["new_account_10d", "perfect_score_anomaly"],
756
+ ),
757
+ PayOpsTask(
758
+ task_id="HARD-010",
759
+ difficulty="hard",
760
+ description=(
761
+ "Payroll diversion: HR system breach rerouted an employee's salary to "
762
+ "a newly added bank account 2 days ago. Employee has not been notified."
763
+ ),
764
+ transaction_id="TXN-H010",
765
+ amount=5_640.00,
766
+ currency="USD",
767
+ sender="payroll@employer-corp.com",
768
+ receiver="new-acct-redirect@routing-bank.com",
769
+ transaction_type="payroll",
770
+ risk_score=0.58,
771
+ ml_confidence=0.63,
772
+ flags=["direct_deposit_changed", "recent_account_change", "payroll_diversion_indicator"],
773
+ velocity_1h=1, velocity_24h=1,
774
+ avg_transaction_amount=5_640.0,
775
+ account_age_days=2_920,
776
+ country_risk="low", kyc_status="verified", kyc_expiry_days=180,
777
+ previous_violations=0, previous_sars=0, counterparty_risk="unknown",
778
+ correct_action="reject",
779
+ partial_credit_actions={"hold": 0.5, "escalate": 0.4, "flag": 0.3},
780
+ requires_investigation={"inspect", "contact_sender"},
781
+ inspect_reveal=(
782
+ "HR system audit: direct deposit account changed 2 days ago via an "
783
+ "unrecognized IP at 2:14 AM. Employee's primary device was offline at the "
784
+ "time. No 2FA challenge was issued. Account change was unauthorized."
785
+ ),
786
+ contact_reveal=(
787
+ "Reached employee by mobile: they confirm they did NOT change their "
788
+ "direct deposit account. Did not receive any email or SMS about the "
789
+ "change. HR system credentials were likely phished. Reject this payroll "
790
+ "disbursement and revert to prior account."
791
+ ),
792
+ key_flags=["direct_deposit_changed", "payroll_diversion_indicator"],
793
+ ),
794
  ]
795
 
796
 
797
  # =============================================================================
798
+ # CRITICAL (6 tasks — multi-step chains, regulatory stakes)
799
  # =============================================================================
800
 
801
  CRITICAL_TASKS: List[PayOpsTask] = [
 
924
  contact_reveal="CEO confirms they did NOT initiate this transfer. Account takeover confirmed.",
925
  key_flags=["geo_impossible_login", "account_takeover_indicator"],
926
  ),
927
+ PayOpsTask(
928
+ task_id="CRIT-005",
929
+ difficulty="critical",
930
+ description=(
931
+ "OFAC sanctions evasion: large USD payment routed through a UAE shell "
932
+ "company chain. Investigation reveals the ultimate beneficial owner is "
933
+ "on the OFAC Specially Designated Nationals (SDN) list."
934
+ ),
935
+ transaction_id="TXN-C005",
936
+ amount=490_000.00,
937
+ currency="USD",
938
+ sender="uae-trading@al-rashid-fze.ae",
939
+ receiver="intermediary@gulf-clearing-bv.nl",
940
+ transaction_type="transfer",
941
+ risk_score=0.69,
942
+ ml_confidence=0.60,
943
+ flags=["high_value", "cross_border", "shell_company_indicator", "sanctions_adjacent"],
944
+ velocity_1h=1, velocity_24h=2,
945
+ avg_transaction_amount=380_000.0,
946
+ account_age_days=180,
947
+ country_risk="high", kyc_status="verified", kyc_expiry_days=45,
948
+ previous_violations=0, previous_sars=0, counterparty_risk="watchlist",
949
+ correct_action="reject",
950
+ partial_credit_actions={"escalate": 0.5, "hold": 0.4},
951
+ requires_investigation={"inspect", "verify_kyc", "request_docs"},
952
+ regulatory_action=True,
953
+ chain_total=3,
954
+ inspect_reveal=(
955
+ "Corporate registry search: al-rashid-fze.ae incorporated 6 months ago. "
956
+ "Sole director: a nominee agent used by 47 other FZE shells. Beneficial "
957
+ "ownership records absent. Payment routed via 3 intermediate entities "
958
+ "before reaching the UAE sender — classic sanctions-evasion layering."
959
+ ),
960
+ kyc_reveal=(
961
+ "UBO lookup via SWIFT KYC Registry and Refinitiv: ultimate beneficial "
962
+ "owner resolves to Tariq Al-Mansouri, listed on OFAC SDN list since 2021 "
963
+ "(Ref: SDN-2021-04-19-TM). Transacting with this UBO violates 31 C.F.R. "
964
+ "§ 594. Reject immediately and file SAR."
965
+ ),
966
+ docs_reveal=(
967
+ "Provided trade invoice shows goods described as 'industrial equipment'. "
968
+ "HS code used (8479.89) does not match the attached packing list. "
969
+ "Likely misclassification to obscure dual-use goods export controls."
970
+ ),
971
+ key_flags=["shell_company_indicator", "sanctions_adjacent"],
972
+ network_graph={
973
+ "sender_shell": "al-rashid-fze.ae",
974
+ "intermediary": "gulf-clearing-bv.nl",
975
+ "ubo_sdn": "Tariq Al-Mansouri",
976
+ },
977
+ ),
978
+ PayOpsTask(
979
+ task_id="CRIT-006",
980
+ difficulty="critical",
981
+ description=(
982
+ "Correspondent banking alert: a downstream correspondent partner bank "
983
+ "was added to FinCEN's 311 Special Measures list yesterday. Payments "
984
+ "routed through them are now legally restricted."
985
+ ),
986
+ transaction_id="TXN-C006",
987
+ amount=215_000.00,
988
+ currency="USD",
989
+ sender="trade-desk@our-bank.com",
990
+ receiver="correspondent@partner-bank-latam.bz",
991
+ transaction_type="transfer",
992
+ risk_score=0.71,
993
+ ml_confidence=0.65,
994
+ flags=["fincen_311_alert", "correspondent_risk", "high_value", "cross_border"],
995
+ velocity_1h=3, velocity_24h=12,
996
+ avg_transaction_amount=200_000.0,
997
+ account_age_days=2_100,
998
+ country_risk="high", kyc_status="verified", kyc_expiry_days=20,
999
+ previous_violations=0, previous_sars=0, counterparty_risk="watchlist",
1000
+ correct_action="escalate",
1001
+ partial_credit_actions={"hold": 0.5, "reject": 0.4},
1002
+ requires_investigation={"inspect", "request_docs"},
1003
+ regulatory_action=True,
1004
+ chain_total=2,
1005
+ inspect_reveal=(
1006
+ "FinCEN Section 311 Special Measure imposed 2024-04-01 on partner-bank-latam.bz "
1007
+ "for 'primary money laundering concern'. Special Measure 5 prohibits US "
1008
+ "correspondent accounts and payable-through accounts. All in-flight wires "
1009
+ "must be escalated to Compliance before release — unilateral rejection "
1010
+ "also requires Compliance sign-off (may breach existing agreements)."
1011
+ ),
1012
+ docs_reveal=(
1013
+ "Correspondent agreement review: clause 12.3 requires 90-day wind-down "
1014
+ "notice before terminating the relationship. Legal and Compliance must "
1015
+ "jointly approve next steps. Escalate immediately — do not reject or "
1016
+ "approve without written senior Compliance authorisation."
1017
+ ),
1018
+ key_flags=["fincen_311_alert", "correspondent_risk"],
1019
+ network_graph={
1020
+ "our_bank": "trade-desk@our-bank.com",
1021
+ "correspondent": "partner-bank-latam.bz",
1022
+ },
1023
+ ),
1024
  ]
1025
 
1026
 
 
1063
  # ---------------------------------------------------------------------------
1064
  TASK_VARIANTS: Dict[str, List[Dict]] = {
1065
 
1066
+ # ── EASY ──────────────────────────────────────────────────────────────
1067
+
1068
+ "EASY-001": [
1069
+ # v1: account accessed from unknown device just before transfer → hold
1070
+ {
1071
+ "correct_action": "hold",
1072
+ "partial_credit_actions": {"flag": 0.6, "escalate": 0.3},
1073
+ "flags": ["unusual_login_location", "device_change"],
1074
+ "risk_score": 0.61,
1075
+ "inspect_reveal": (
1076
+ "Sender logged in from an unrecognized device in a different country "
1077
+ "18 minutes ago. Possible account takeover. All signals otherwise "
1078
+ "identical. Hold pending confirmation with account holder."
1079
+ ),
1080
+ },
1081
+ # v2: amount exceeds sender's daily limit for new payee → flag
1082
+ {
1083
+ "correct_action": "flag",
1084
+ "partial_credit_actions": {"hold": 0.6, "approve": 0.2},
1085
+ "flags": ["first_time_payee", "amount_spike"],
1086
+ "risk_score": 0.44,
1087
+ "inspect_reveal": (
1088
+ "Bob is a new payee — first time ever on this account. System "
1089
+ "limit for first-time payees is $25. Transfer amount $47.50 exceeds "
1090
+ "new-payee daily cap. Flag for standard new-payee challenge-and-confirm."
1091
+ ),
1092
+ },
1093
+ ],
1094
+
1095
+ "EASY-002": [
1096
+ # v1: signals are real but entity is a licensed MSB with OFAC authorisation → escalate
1097
+ {
1098
+ "correct_action": "escalate",
1099
+ "partial_credit_actions": {"reject": 0.4, "hold": 0.3},
1100
+ "inspect_reveal": (
1101
+ "Sender is a licensed Money Service Business (MSB) with a FinCEN "
1102
+ "registration number and a current OFAC sanctions-compliance programme. "
1103
+ "The 'sanctioned country' flag was triggered by destination country "
1104
+ "of receiver's beneficial owner, but this MSB holds a specific OFAC "
1105
+ "licence for this corridor. Escalate for senior review rather than "
1106
+ "unilateral reject."
1107
+ ),
1108
+ },
1109
+ # v2: additional money mule node detected → reject + SAR
1110
+ {
1111
+ "correct_action": "reject",
1112
+ "partial_credit_actions": {"escalate": 0.5},
1113
+ "regulatory_action": True,
1114
+ "inspect_reveal": (
1115
+ "Receiver account linked to 6 prior confirmed fraud rejections this "
1116
+ "quarter, totalling $58k. New network node in a known mule ring. "
1117
+ "All prior transfers drained offshore within 30 minutes. Reject "
1118
+ "immediately and file SAR."
1119
+ ),
1120
+ },
1121
+ ],
1122
+
1123
+ "EASY-003": [
1124
+ # v1: refund issued to a different card than the original purchase → flag
1125
+ {
1126
+ "correct_action": "flag",
1127
+ "partial_credit_actions": {"hold": 0.5, "escalate": 0.3},
1128
+ "flags": ["refund_mismatch", "different_card"],
1129
+ "risk_score": 0.52,
1130
+ "inspect_reveal": (
1131
+ "Original purchase was made on Visa ending 4412. Refund is being "
1132
+ "issued to Mastercard ending 7891 — a different card not on file for "
1133
+ "this order. Possible refund fraud. Flag for merchant verification "
1134
+ "before processing."
1135
+ ),
1136
+ },
1137
+ # v2: refund claimed on a fraudulent chargeback dispute → reject
1138
+ {
1139
+ "correct_action": "reject",
1140
+ "partial_credit_actions": {"flag": 0.5, "escalate": 0.3},
1141
+ "flags": ["chargeback_fraud", "dispute_open"],
1142
+ "risk_score": 0.68,
1143
+ "inspect_reveal": (
1144
+ "Customer filed a chargeback claiming non-receipt. Courier tracking "
1145
+ "shows package delivered and signed for 3 days ago. Customer's "
1146
+ "claimed delivery address matches the signature. Friendly fraud — "
1147
+ "reject refund and respond to chargeback with delivery evidence."
1148
+ ),
1149
+ },
1150
+ ],
1151
+
1152
+ "EASY-004": [
1153
+ # v1: card-cloning confirmed by bank intelligence → reject
1154
+ {
1155
+ "correct_action": "reject",
1156
+ "partial_credit_actions": {"escalate": 0.5, "flag": 0.4},
1157
+ "inspect_reveal": (
1158
+ "Cross-check with card-fraud intelligence feed: the card number "
1159
+ "involved in this ATM burst matches a batch of 340 card numbers "
1160
+ "from a point-of-sale skimming incident reported yesterday. Confirmed "
1161
+ "card-clone attack. Reject all withdrawals and block card immediately."
1162
+ ),
1163
+ },
1164
+ # v2: legitimate end-of-month cash distribution for a small cash-and-carry → approve
1165
+ {
1166
+ "correct_action": "approve",
1167
+ "partial_credit_actions": {"flag": 0.3},
1168
+ "inspect_reveal": (
1169
+ "Account holder is a licensed cash-and-carry wholesaler. Same "
1170
+ "end-of-month ATM burst pattern appears in 11 of the last 12 months — "
1171
+ "consistent with weekly cash float replenishment for their 4 shop "
1172
+ "locations. No fraud intelligence matches. Approve."
1173
+ ),
1174
+ },
1175
+ ],
1176
+
1177
+ "EASY-005": [
1178
+ # v1: loan account in arrears, payment covers only partial balance → flag
1179
+ {
1180
+ "correct_action": "flag",
1181
+ "partial_credit_actions": {"hold": 0.5, "approve": 0.3},
1182
+ "amount": 650.00,
1183
+ "flags": ["partial_payment", "loan_arrears"],
1184
+ "risk_score": 0.38,
1185
+ "inspect_reveal": (
1186
+ "Loan account 2 months in arrears. Required minimum payment is "
1187
+ "$1,247 but sender submitted $650. Partial payment does not clear "
1188
+ "the overdue balance. Flag for borrower contact and payment plan "
1189
+ "communication before crediting."
1190
+ ),
1191
+ },
1192
+ # v2: overpayment submitted — process normally but note discrepancy → approve
1193
+ {
1194
+ "correct_action": "approve",
1195
+ "partial_credit_actions": {"flag": 0.3},
1196
+ "amount": 2_500.00,
1197
+ "flags": ["overpayment"],
1198
+ "risk_score": 0.08,
1199
+ "inspect_reveal": (
1200
+ "Borrower submitted $2,500 vs scheduled $1,247. Overpayments are "
1201
+ "permitted under the loan agreement (clause 8.2) and reduce principal "
1202
+ "balance. No fraud indicators. Approve and apply excess to principal."
1203
+ ),
1204
+ },
1205
+ ],
1206
+
1207
+ "EASY-006": [
1208
+ # v1: genuine re-submission after first instance failed → approve
1209
+ {
1210
+ "correct_action": "approve",
1211
+ "partial_credit_actions": {"flag": 0.4},
1212
+ "flags": ["resubmission"],
1213
+ "risk_score": 0.18,
1214
+ "inspect_reveal": (
1215
+ "First submission (4 min ago) failed with a bank routing error "
1216
+ "(error code: ROUTE-001 — invalid sort code format). This is a valid "
1217
+ "resubmission with corrected routing. No double-charge risk. Approve."
1218
+ ),
1219
+ },
1220
+ # v2: vendor-side system error caused 8 duplicate submissions → hold
1221
+ {
1222
+ "correct_action": "hold",
1223
+ "partial_credit_actions": {"flag": 0.6},
1224
+ "velocity_1h": 8,
1225
+ "flags": ["duplicate_transaction", "velocity_breach", "api_error_burst"],
1226
+ "risk_score": 0.72,
1227
+ "inspect_reveal": (
1228
+ "Vendor payment API experienced a timeout loop at 14:02. 8 identical "
1229
+ "payment submissions in 5 minutes from the same API key. Only 1 "
1230
+ "should be processed. Hold all 8; contact vendor to identify the "
1231
+ "canonical intended payment before releasing any."
1232
+ ),
1233
+ },
1234
+ ],
1235
+
1236
  # ── MEDIUM ────────────────────────────────────────────────────────────
1237
 
1238
+ "MED-001": [
1239
+ # v1: CEO impersonation (BEC) — fraudulent wire instruction → reject
1240
+ {
1241
+ "correct_action": "reject",
1242
+ "partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
1243
+ "flags": ["high_value", "cross_border", "bec_indicator"],
1244
+ "risk_score": 0.74,
1245
+ "inspect_reveal": (
1246
+ "The CFO's email domain in the wire instructions is globalcorp.co "
1247
+ "(note: .co not .com). CFO's actual domain is globalcorp.com. "
1248
+ "Phone verification: real CFO is on holiday and did NOT authorise "
1249
+ "this transfer. Business Email Compromise confirmed — reject."
1250
+ ),
1251
+ },
1252
+ # v2: contract and counterparty fully verified → approve
1253
+ {
1254
+ "correct_action": "approve",
1255
+ "partial_credit_actions": {"escalate": 0.3, "flag": 0.2},
1256
+ "inspect_reveal": (
1257
+ "Vendor contract on file, signed 3 months ago (ref VC-EUR-0712). "
1258
+ "Receiver is an EU-licensed entity (UID DE315441892). Payment matches "
1259
+ "milestone 2 of 4. CFO confirmed by phone. Cross-border and amount "
1260
+ "are within normal operating parameters. Approve."
1261
+ ),
1262
+ },
1263
+ ],
1264
+
1265
+ "MED-002": [
1266
+ # v1: KYC re-check reveals UBO mismatch → escalate
1267
+ {
1268
+ "correct_action": "escalate",
1269
+ "partial_credit_actions": {"hold": 0.5, "flag": 0.3},
1270
+ "kyc_reveal": (
1271
+ "KYC renewal triggered a UBO re-check. Companies House filing shows "
1272
+ "the ultimate beneficial owner changed 3 months ago to a holding "
1273
+ "company domiciled in a secrecy jurisdiction, without bank "
1274
+ "notification. Material KYC change — escalate for enhanced due "
1275
+ "diligence before releasing any treasury transfers."
1276
+ ),
1277
+ },
1278
+ # v2: board resolution authorises emergency liquidity transfer → approve
1279
+ {
1280
+ "correct_action": "approve",
1281
+ "partial_credit_actions": {"hold": 0.4},
1282
+ "kyc_reveal": (
1283
+ "KYC renewal: both entities share the same UBO (verified). Pending "
1284
+ "status is purely administrative — renewal submitted 6 days ago, "
1285
+ "no change of ownership. Board resolution ref BR-2024-Q1-08 "
1286
+ "authorises this emergency liquidity transfer. Approve."
1287
+ ),
1288
+ },
1289
+ ],
1290
+
1291
  "MED-003": [
1292
+ # v1: unauthorised recurring billing → hold
1293
  {
1294
  "correct_action": "hold",
1295
  "partial_credit_actions": {"flag": 0.5, "escalate": 0.3},
 
1319
  "partial_credit_actions": {"flag": 0.5, "hold": 0.3},
1320
  "inspect_reveal": (
1321
  "Sender has made 7 crypto exchange payments in 5 days totalling $28k "
1322
+ "across 3 different platforms. Amounts stay just below exchange "
1323
+ "reporting limits. Pattern matches structuring. Escalate for AML review."
1324
  ),
1325
  },
1326
  # v2: clean recurring investment → approve
 
1386
  },
1387
  ],
1388
 
1389
+ "MED-007": [
1390
+ # v1: genuine bank account change, properly authorised → approve
1391
+ {
1392
+ "correct_action": "approve",
1393
+ "partial_credit_actions": {"hold": 0.5, "flag": 0.3},
1394
+ "docs_reveal": (
1395
+ "Signed bank-change authorisation letter provided on supplier "
1396
+ "letterhead, with notarised director signature matching KYC records. "
1397
+ "Called supplier finance director on known contact number — confirmed "
1398
+ "the account change. New account is registered to the same company. "
1399
+ "Approve."
1400
+ ),
1401
+ "contact_reveal": (
1402
+ "Original supplier contact verified the bank change was genuine — "
1403
+ "they switched banks last month due to a merger. Reference number "
1404
+ "SBC-2024-0312 matches their internal change management log."
1405
+ ),
1406
+ },
1407
+ # v2: multiple lookalike invoice redirect emails sent to same client → escalate
1408
+ {
1409
+ "correct_action": "escalate",
1410
+ "partial_credit_actions": {"hold": 0.5, "reject": 0.4},
1411
+ "docs_reveal": (
1412
+ "Client reports receiving 3 lookalike invoice redirect emails in 2 "
1413
+ "weeks from slightly different supplier domains. Two payments to "
1414
+ "prior redirect accounts were already processed (total €87k). This "
1415
+ "is a systematic BEC campaign targeting this client relationship. "
1416
+ "Escalate to fraud and financial crime team immediately."
1417
+ ),
1418
+ "contact_reveal": (
1419
+ "Genuine supplier finance director: they have sent zero bank-change "
1420
+ "notices this year. The lookalike domain was registered 3 weeks ago. "
1421
+ "Client's email system has been compromised. Escalate."
1422
+ ),
1423
+ },
1424
+ ],
1425
+
1426
+ "MED-008": [
1427
+ # v1: KYC passes, address verified — legitimate BNPL purchase → approve
1428
+ {
1429
+ "correct_action": "approve",
1430
+ "partial_credit_actions": {"flag": 0.4},
1431
+ "kyc_reveal": (
1432
+ "KYC recheck: address verification passed on second attempt — "
1433
+ "customer had entered a unit number incorrectly. Identity document "
1434
+ "authentic. Credit bureau confirms thin file is consistent with "
1435
+ "a first-time credit applicant (age 22). No fraud indicators. "
1436
+ "Approve BNPL transaction."
1437
+ ),
1438
+ },
1439
+ # v2: device fingerprint linked to BNPL fraud network → reject
1440
+ {
1441
+ "correct_action": "reject",
1442
+ "partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
1443
+ "kyc_reveal": (
1444
+ "Device fingerprint matches 14 prior BNPL fraud applications across "
1445
+ "3 platforms in the last 6 weeks. All 14 prior accounts defaulted on "
1446
+ "first instalment. Identity documents were synthetically generated. "
1447
+ "Reject and blacklist device fingerprint."
1448
+ ),
1449
+ },
1450
+ ],
1451
+
1452
  # ── HARD ──────────────────────────────────────────────────────────────
1453
 
1454
  "HARD-001": [
 
1589
  "Reject and file SAR immediately."
1590
  ),
1591
  },
1592
+ # v2: estate probate settlement, legitimate reactivation → escalate
1593
  {
1594
  "correct_action": "escalate",
1595
  "partial_credit_actions": {"flag": 0.6, "hold": 0.4},
 
1602
  },
1603
  ],
1604
 
1605
+ "HARD-007": [
1606
+ # v1: SIM was ported by legitimate account holder (travelling) → hold
1607
+ {
1608
+ "correct_action": "hold",
1609
+ "partial_credit_actions": {"escalate": 0.5, "reject": 0.4},
1610
+ "inspect_reveal": (
1611
+ "Telecom records: SIM swap was initiated via the carrier's verified "
1612
+ "app using biometric authentication. Account holder is overseas and "
1613
+ "replaced a broken SIM abroad. No third-party involvement detected. "
1614
+ "New withdrawal address is a cold storage wallet previously linked "
1615
+ "to the account. Hold briefly for confirmation call."
1616
+ ),
1617
+ "contact_reveal": (
1618
+ "Reached account holder on a backup email. They confirm the SIM swap "
1619
+ "and the withdrawal are legitimate. They are in Japan and broke their "
1620
+ "phone. Hold approved — confirm by secure callback before releasing."
1621
+ ),
1622
+ },
1623
+ # v2: no response from account holder, third-party SIM swap confirmed → reject
1624
+ {
1625
+ "correct_action": "reject",
1626
+ "partial_credit_actions": {"hold": 0.4, "escalate": 0.5},
1627
+ "contact_reveal": (
1628
+ "Account holder unreachable on all registered contact methods. "
1629
+ "Carrier fraud team confirmed: SIM swap was initiated via a social "
1630
+ "engineering call to a customer service agent — not by the account "
1631
+ "holder. No biometric authentication used. Third-party SIM swap "
1632
+ "confirmed. Reject and lock account."
1633
+ ),
1634
+ },
1635
+ ],
1636
+
1637
+ "HARD-008": [
1638
+ # v1: customer is aware it's high-risk investment but not a scam → escalate
1639
+ {
1640
+ "correct_action": "escalate",
1641
+ "partial_credit_actions": {"reject": 0.4, "hold": 0.4},
1642
+ "contact_reveal": (
1643
+ "Customer is fully aware of the risk and has a history of crypto "
1644
+ "investing. The 'romantic partner' is actually a friend from a "
1645
+ "verified online investment forum. Prior transfers were legitimate "
1646
+ "investments that returned profit. However, the escalating pattern "
1647
+ "and offshore destination warrant senior review. Escalate."
1648
+ ),
1649
+ "inspect_reveal": (
1650
+ "Transaction history: prior 3 payments were invested and partially "
1651
+ "returned. Receiver fund has a verifiable online presence (2 years). "
1652
+ "However, the most recent dormancy of returns and amount escalation "
1653
+ "match late-stage pig-butchering conversion. Escalate for human review."
1654
+ ),
1655
+ },
1656
+ # v2: scam confirmed, customer is distressed victim → reject
1657
+ {
1658
+ "correct_action": "reject",
1659
+ "partial_credit_actions": {"hold": 0.4, "escalate": 0.3},
1660
+ "contact_reveal": (
1661
+ "Customer broke down on the call. They have now been blocked by the "
1662
+ "'romantic partner' and cannot reach them. They realise they have been "
1663
+ "scammed. All prior funds ($5,500) are lost. Reject this transfer, "
1664
+ "file SAR, and refer customer to scam recovery support."
1665
+ ),
1666
+ },
1667
+ ],
1668
+
1669
+ "HARD-009": [
1670
+ # v1: new tech entrepreneur, thin file is legitimate → approve
1671
+ {
1672
+ "correct_action": "approve",
1673
+ "partial_credit_actions": {"hold": 0.5, "escalate": 0.4},
1674
+ "kyc_reveal": (
1675
+ "KYC deep-dive: EIN issued 11 days ago but LLC was pre-registered "
1676
+ "90 days ago (formation date vs EIN issuance differ — normal for new "
1677
+ "companies). Registered address is a co-working space (common for "
1678
+ "startups). Director SSN cross-check passsed. Wire is to a verified "
1679
+ "software vendor (AWS partner). Thin file is genuine, not synthetic. "
1680
+ "Approve."
1681
+ ),
1682
+ "inspect_reveal": (
1683
+ "ML confidence flag was fired due to thin data, not anomalous data. "
1684
+ "No features match synthetic identity profile (all values have "
1685
+ "natural variance). The 'perfect score anomaly' flag was a false "
1686
+ "positive from the rule engine. Safe to approve."
1687
+ ),
1688
+ },
1689
+ # v2: identity confirmed as synthetic via government database check → reject
1690
+ {
1691
+ "correct_action": "reject",
1692
+ "partial_credit_actions": {"escalate": 0.4, "hold": 0.3},
1693
+ "kyc_reveal": (
1694
+ "IRS TIN matching service: EIN does not match the business name in "
1695
+ "the IRS database. SSN belongs to a deceased individual. Address is "
1696
+ "a virtual office with no physical occupancy. Synthetic identity "
1697
+ "confirmed. Reject all transactions and report to FinCEN."
1698
+ ),
1699
+ },
1700
+ ],
1701
+
1702
+ "HARD-010": [
1703
+ # v1: employee did change their account (legitimate, notified HR verbally) → approve
1704
+ {
1705
+ "correct_action": "approve",
1706
+ "partial_credit_actions": {"hold": 0.5, "flag": 0.4},
1707
+ "inspect_reveal": (
1708
+ "HR audit: account change was requested by the employee at HR's "
1709
+ "'walk-in' desk 3 days ago. The change was logged manually by an "
1710
+ "HR admin and entered into the system at 2:10 AM during the overnight "
1711
+ "batch. Unusual timestamp is from the batch processor, not an "
1712
+ "attacker. Legitimate change confirmed."
1713
+ ),
1714
+ "contact_reveal": (
1715
+ "Employee confirms: they switched to a new bank and submitted the "
1716
+ "form in person. Was not aware any notification would be sent. "
1717
+ "Batch processing time was confusing but legitimate. Approve."
1718
+ ),
1719
+ },
1720
+ # v2: HR credentials were phished, second employee already victimised → reject
1721
+ {
1722
+ "correct_action": "reject",
1723
+ "partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
1724
+ "inspect_reveal": (
1725
+ "HR system audit: same rogue IP (185.220.101.x — known Tor exit node) "
1726
+ "changed direct deposit accounts for 3 employees overnight. Two "
1727
+ "employees have already confirmed they did not make changes. Payroll "
1728
+ "diversion campaign in progress. Reject all affected disbursements "
1729
+ "and freeze HR system access."
1730
+ ),
1731
+ },
1732
+ ],
1733
+
1734
  # ── CRITICAL ──────────────────────────────────────────────────────────
1735
 
1736
  "CRIT-001": [
 
1770
  ],
1771
 
1772
  "CRIT-002": [
1773
+ # v1: same SME multiple business accounts — coincidental threshold → hold
1774
  {
1775
  "correct_action": "hold",
1776
  "partial_credit_actions": {"reject": 0.5, "escalate": 0.4},
 
1812
  "Hong Kong and the destination country. No discrepancies."
1813
  ),
1814
  },
1815
+ # v2: criminal-scale TBML reject
1816
  {
1817
  "correct_action": "reject",
1818
  "partial_credit_actions": {"escalate": 0.5, "flag": 0.3},
 
1831
  ],
1832
 
1833
  "CRIT-004": [
1834
+ # v1: CEO delegated to CFO who was in Lagos hold for confirmation
1835
  {
1836
  "correct_action": "hold",
1837
  "partial_credit_actions": {"escalate": 0.5, "reject": 0.4},
 
1847
  "Hold briefly to verify the papertrail, then release."
1848
  ),
1849
  },
1850
+ # v2: CEO unreachable, audit trail ambiguous escalate
1851
  {
1852
  "correct_action": "escalate",
1853
  "partial_credit_actions": {"reject": 0.5, "hold": 0.4},
 
1865
  ),
1866
  },
1867
  ],
1868
+
1869
+ "CRIT-005": [
1870
+ # v1: shell layers cleared, legitimate commodity trade → approve
1871
+ {
1872
+ "correct_action": "approve",
1873
+ "partial_credit_actions": {"escalate": 0.5, "hold": 0.4},
1874
+ "inspect_reveal": (
1875
+ "UAE FZE registration traced through 3 layers: ultimate beneficial "
1876
+ "owner is a listed European industrial group (Frankfurt Stock Exchange: "
1877
+ "DXI.DE). No SDN or PEP matches. Shell structure is standard for "
1878
+ "legitimate UAE free-zone trading entities. All FATF risk factors "
1879
+ "cleared by AML team. Approve."
1880
+ ),
1881
+ "kyc_reveal": (
1882
+ "UBO lookup: al-Rashid FZE is a subsidiary of Deutsche Xport Industries "
1883
+ "AG, a publicly listed entity. No sanctions exposure. KYC pack complete "
1884
+ "and verified. Trade invoice amount matches published commodity prices."
1885
+ ),
1886
+ "docs_reveal": (
1887
+ "Trade invoice and shipping manifest verified. HS code 8479.89 matches "
1888
+ "industrial mixing equipment — no dual-use export restrictions apply. "
1889
+ "All documentation clean. Approve."
1890
+ ),
1891
+ },
1892
+ # v2: SDN match confirmed at deeper UBO level → reject
1893
+ {
1894
+ "correct_action": "reject",
1895
+ "partial_credit_actions": {"escalate": 0.4, "hold": 0.3},
1896
+ "inspect_reveal": (
1897
+ "Shell chain traced: al-rashid-fze.ae → Gibraltar holding → BVI "
1898
+ "nominee → Tariq Al-Mansouri (OFAC SDN since 2021, Ref SDN-2021-04-19). "
1899
+ "3-layer obfuscation structure is classic sanctions evasion. Reject "
1900
+ "and report to OFAC within 10 business days."
1901
+ ),
1902
+ "kyc_reveal": (
1903
+ "UBO confirmed as Tariq Al-Mansouri via FinScan premium SDN database "
1904
+ "cross-reference. Transacting with this UBO violates 31 C.F.R. § 594. "
1905
+ "Reject immediately and file SAR with FinCEN."
1906
+ ),
1907
+ },
1908
+ ],
1909
+
1910
+ "CRIT-006": [
1911
+ # v1: FinCEN notice is a false positive — different bank with same name → approve
1912
+ {
1913
+ "correct_action": "approve",
1914
+ "partial_credit_actions": {"hold": 0.5, "escalate": 0.4},
1915
+ "inspect_reveal": (
1916
+ "FinCEN 311 action reviewed: the targeted bank is 'Global Partner Bank "
1917
+ "of Belize', ABA 082901987. Our correspondent is 'Partner Bank of Latin "
1918
+ "America, Belize', ABA 082901462 — a distinct entity. The alert was "
1919
+ "triggered by a partial name match. Our correspondent is not subject "
1920
+ "to the 311 measure. Approve with a compliance memo on file."
1921
+ ),
1922
+ "docs_reveal": (
1923
+ "SWIFT BIC PTBZBZBZ confirmed as Partner Bank of Latin America — "
1924
+ "no sanctions linkage. Legal confirmed no restriction applies to "
1925
+ "this specific correspondent relationship. Approve this payment and "
1926
+ "update the name-match rule to prevent future false positives."
1927
+ ),
1928
+ },
1929
+ # v2: FinCEN 311 applies, must cease all correspondent transactions → escalate
1930
+ {
1931
+ "correct_action": "escalate",
1932
+ "partial_credit_actions": {"hold": 0.5, "reject": 0.4},
1933
+ "inspect_reveal": (
1934
+ "FinCEN 311 Special Measure 5 confirmed as applicable to our "
1935
+ "correspondent (partner-bank-latam.bz, SWIFT PTBZBZBZ). All US "
1936
+ "dollar correspondent and payable-through accounts must be closed "
1937
+ "within 30 days. In-flight transactions require Compliance authorisation. "
1938
+ "Escalate immediately — unilateral rejection may breach our "
1939
+ "contractual wind-down obligations."
1940
+ ),
1941
+ "docs_reveal": (
1942
+ "Correspondent agreement clause 12.3 requires 90-day written wind-down "
1943
+ "notice. FinCEN has set a 30-day compliance deadline creating a "
1944
+ "legal conflict. Legal and Compliance must jointly manage the "
1945
+ "resolution. Escalate with full documentation."
1946
+ ),
1947
+ },
1948
+ ],
1949
  }
1950
 
1951
 
validate.py CHANGED
@@ -150,7 +150,7 @@ def check_openenv_yaml():
150
  ("API_BASE_URL", "API_BASE_URL env var declared"),
151
  ("MODEL_NAME", "MODEL_NAME env var declared"),
152
  ("HF_TOKEN", "HF_TOKEN env var declared"),
153
- ("count: 20", "tasks count=20"),
154
  ("inference.py", "inference script reference"),
155
  ("approve", "approve action"),
156
  ("reject", "reject action"),
 
150
  ("API_BASE_URL", "API_BASE_URL env var declared"),
151
  ("MODEL_NAME", "MODEL_NAME env var declared"),
152
  ("HF_TOKEN", "HF_TOKEN env var declared"),
153
+ ("count: 30", "tasks count=30"),
154
  ("inference.py", "inference script reference"),
155
  ("approve", "approve action"),
156
  ("reject", "reject action"),