README.md

---
library_name: sklearn
license: mit
language:
- en
tags:
- fraud-detection
- agent-security
- mcp
- agentic-ai
- payment-security
- behavioral-detection
- random-forest
- session-graph
pipeline_tag: text-classification
---

# acb-guard-session-graph-rf

A RandomForest classifier trained on 20-dimensional session graph features extracted from AI agent MCP payment sessions. The first application of session graph feature extraction to MCP commerce call sequences.

**Key result: F1=0.64, FPR=0%** on the AgentCommerceBench behavioral attack layer — trained on clean sessions only, with zero knowledge of specific attack patterns.

This is the behavioral detection baseline in the [AgentCommerceBench](https://github.com/BuildWithGordonAI/agentcommercebench) pipeline, deployed as the L2 layer between the L1 keyword guard and L3 commerce policy enforcement.

## Access

This model is **gated**. Request access above. We grant access for research, defensive security, and evaluation purposes.

## What it detects

Autonomous AI agents operating on the Gordon MCP payment rail issue two types of calls:
- `gordon_find_service` — catalog lookup, discovers vendors
- `gordon_authorize` — payment authorization, transfers USDC on-chain

This model detects **behavioral sequence attacks** — attacks where every individual call looks legitimate, but the session-level pattern reveals compromise. These are invisible to keyword scanners and rate limiters.

**Example — B3 Reconnaissance Sweep**: An attacker-controlled agent issues 15 `gordon_find_service` probes before a single $0.50 USDC `gordon_authorize` to an attacker wallet. No injected text anywhere. The `find_authorize_ratio` feature reaches 15.0 vs. a clean-session mean of ~1.0 — 46 standard deviations above normal. Caught at probe 4.

## The 20 Features

Each agent session is mapped to a 20-dimensional feature vector extracted from the MCP call sequence — no raw text, no logs, just the event structure:

| Index | Feature | What it detects |
|-------|---------|-----------------|
| 0 | `n_events` | Session length |
| 1 | `n_find_service` | Total catalog lookups |
| 2 | `n_authorize` | Total payment authorizations |
| 3 | `n_settle` | Settlement events |
| 4 | `find_authorize_ratio` | **Recon signal** — 15.0 in B3, ~1.0 in clean sessions |
| 5 | `n_distinct_services` | **Catalog sweep** — 15 unique vendors in B3, ~1 clean |
| 6 | `service_diversity` | Shannon entropy of vendor distribution |
| 7 | `total_amount_usdc` | Session spend total |
| 8 | `max_amount_usdc` | Largest single transaction |
| 9 | `amount_std_usdc` | **Ratcheting signal** (B2 — escalating transaction amounts) |
| 10 | `amount_max_ratio` | Max vs. session mean (spike detection) |
| 11 | `timing_gap_mean_s` | Average inter-event gap |
| 12 | `timing_gap_std_s` | **Burst signal** — high variance = irregular probing cadence |
| 13 | `timing_gap_min_s` | Minimum gap — floor of burst probing |
| 14 | `n_categories` | Number of distinct service categories used |
| 15 | `dominant_category_frac` | **Scope violation signal** — low purity = category drift |
| 16 | `n_a2a_transfers` | Agent-to-agent transfer count |
| 17 | `a2a_amount_usdc` | Total A2A transfer volume |
| 18 | `has_override_keyword` | **L1 text signal** — SYSTEM:, ignore previous instructions, etc. |
| 19 | `has_b64_blob` | **L1 text signal** — base64-encoded payload detection |

Features 18–19 are L1 text signals included so the model can also flag payload injection attacks (A1–A7), making it a superset of pure keyword filtering for session-level scoring.

## Benchmark Results

Evaluated on AgentCommerceBench — 15 attack scenarios across 3 layers:

| Detector | F1 | FPR | L1 TPR | L2 TPR | L3 TPR | Notes |
|----------|----|-----|--------|--------|--------|-------|
| velocity_check | 0.12 | 0% | 0% | 14% | 0% | Rate-limit only |
| keyword_filter | 0.57 | 0% | 100% | 0% | 0% | Blind to all behavioral attacks |
| isolation_forest | 0.57 | 0% | 20% | 57% | 100% | Unsupervised; misses payload attacks |
| llm_text_safety (Bedrock) | 0.53 | 46% | 100% | 86% | 100% | Unusable FPR in production |
| **session_graph_rf (this)** | **0.64** | **0%** | 60% | 57% | 100% | **Behavioral baseline — 0% FPR** |
| gordon_+seq (full pipeline) | 0.93 | 0% | 100% | 75% | 100% | L1+L2+L3 combined |

**Real Gordon production sessions (held-out): 81% catch rate.**

**Blind red team (BT2 — novel behavioral sequences, no access to detector code):** 6/6 caught. Both novel attacks activate `find_authorize_ratio` and `amount_std` features that rank highest in SGRF's Gini importances — the behavioral signal generalizes to unseen attack variants that share the same structural mechanism.

## Training

- **Data**: 500 clean sessions calibrated from 503 real Gordon production transactions — **no attack examples in training**
- **Method**: The model learns what *normal* looks like per agent persona. Attack detection is anomaly scoring over clean-session distribution, not supervised classification over attack labels.
- **Decision threshold**: `predict_proba[1] >= 0.70` → BLOCK
- **Inference pattern**: Re-scored after every MCP event (streaming), not once at session end

## Usage

```python
import joblib
import numpy as np

# Load weights
clf = joblib.load("session_graph_rf.joblib")

# Load feature extractor from this repo
from session_graph import session_to_features

# Score a session incrementally after each new MCP event
events = [...]  # list of Event objects (see harness/simulate/schema.py)

for i, event in enumerate(events):
    history  = events[:i]
    features = session_to_features(history + [event])  # shape (1, 20)
    proba    = clf.predict_proba(features)[0][1]        # P(fraud)
    decision = "BLOCK" if proba >= 0.70 else "ALLOW"
    print(f"Event {i+1}: P(fraud)={proba:.3f}  {decision}")
```

### Feature vector format

```python
from session_graph import session_to_features

# events: list of dicts or Event objects with fields:
#   action_type  — "FIND_SERVICE" or "AUTHORIZE"
#   amount_units — integer, USDC micro-units (1 USDC = 1,000,000)
#   vendor       — str, wallet address or service_id
#   original_request — dict, raw MCP payload (used for L1 text features)
#   timestamp    — datetime or float seconds

features = session_to_features(events)
# Returns: np.ndarray shape (1, 20), dtype float64
# Column order matches index 0-19 in the feature table above
```

## Files in this repo

| File | Description |
|------|-------------|
| `session_graph_rf.joblib` | Trained RandomForest weights (scikit-learn, joblib format) |
| `session_graph.py` | Feature extraction + detector class (`session_to_features`, `SessionGraphModel`) |

## Related

- Dataset: [withgordon/agentcommercebench](https://huggingface.co/datasets/withgordon/agentcommercebench)
- QLoRA model (graph-conditioned): [withgordon/acb-guard-qwen25-7b-graph](https://huggingface.co/withgordon/acb-guard-qwen25-7b-graph)
- GitHub: [BuildWithGordonAI/agentcommercebench](https://github.com/BuildWithGordonAI/agentcommercebench)

## Citation

```bibtex
@software{agentcommercebench2026,
  title        = {AgentCommerceBench: A Benchmark for Fraud Detection in AI Agent Payment Systems},
  author       = {{Gordon AI}},
  year         = {2026},
  url          = {https://github.com/BuildWithGordonAI/agentcommercebench},
}
```