spend-copilot-4b / README.md
sriram1983007's picture
Point links to the stablemode organization
07e7d4f verified
|
Raw History Blame Contribute Delete
8.45 kB
metadata
license: cc-by-nc-4.0
base_model: Qwen/Qwen3.5-4B
datasets:
  - stablemode/a2a-payments-sandbox
language:
  - en
pipeline_tag: text-generation
tags:
  - agentic-payments
  - x402
  - stablecoin
  - usdc
  - ai-agents
  - payment-policy
  - stablemode
  - lora
extra_gated_heading: Request access to Stablemode Spend Copilot
extra_gated_description: >-
  Free for research and evaluation under CC BY-NC 4.0. Access is approved
  automatically. Commercial use needs a separate license from Stablemode.
extra_gated_button_content: Get access
extra_gated_fields:
  Name: text
  Work email: text
  Company or institution: text
  Role: text
  Intended use:
    type: select
    options:
      - Research
      - Evaluating agent payment guardrails
      - Building an agent product
      - Education
      - label: Other
        value: other
  I will not use this model for commercial purposes without a license from Stablemode: checkbox
model-index:
  - name: stablemode-spend-copilot-4b
    results:
      - task:
          type: text-generation
          name: Agent payment policy decisions
        dataset:
          name: Stablemode A2A Payments (Sandbox)
          type: stablemode/a2a-payments-sandbox
          split: test
        metrics:
          - type: accuracy
            name: Decision accuracy
            value: 0.9888
          - type: f1
            name: Macro F1 (pay / decline / ask_owner)
            value: 0.959
          - type: unsafe_pay_rate
            name: Unsafe pay rate (lower is better)
            value: 0
          - type: over_block_rate
            name: Over-block rate (lower is better)
            value: 0

Stablemode Spend Copilot (4B)

A small open model that decides whether an AI agent should pay another agent: pay, decline, or ask the owner. It explains why in one sentence and says what to change when a payment is blocked.

It's a LoRA fine-tune of Qwen3.5-4B (Apache 2.0), merged into full weights. It was trained on stablemode-a2a-payments-sandbox, 3,000 real API outcomes from the live Stablemode sandbox.

Results

Metric (448 held-out cases) Base Qwen3.5-4B Spend Copilot
Valid JSON answers 100.0% 100.0%
Decision accuracy 65.0% 98.9%
Macro F1 (pay / decline / ask_owner) 49.4% 95.9%
Decline category correct 0.0% 100.0%
Unsafe pay: said "pay" when it should decline 6.2% 0.0%
Over-block: said "decline" when it should pay 0.0% 0.0%
Decision Test cases Base F1 Spend Copilot F1
pay 213 0.922 0.995
decline 211 0.535 0.993
ask_owner 24 0.027 0.889

What changed with fine-tuning: the base model already writes valid JSON. Without training, though, it lets 6.2% of payments that should be blocked go through. It never uses Stablemode's decline categories, and it almost never escalates to the owner. After fine-tuning, it made zero unsafe payments and zero wrong blocks on the test set. Its remaining mistakes are all on the borderline "ask the owner" calls.

What it does

Input: one JSON "situation":

  • the payment and its x402-style requirements
  • the payer's owner policy (status, per-payment cap, daily budget, approved payees)
  • what the payer has spent today
  • its history with this payee
  • the request's status, when relevant

Output: JSON with these fields:

  • decision: pay, decline or ask_owner
  • category: e.g. within_policy, payee_not_approved, per_payment_cap_exceeded, daily_budget_exceeded, wrong_payer, already_paid, request_expired, budget_nearly_used, price_spike
  • reason: one sentence an owner can read
  • suggested_fix: what to change, or null

Usage

import json, torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "stablemode/spend-copilot-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

SYSTEM = "You are Stablemode Spend Copilot. An AI agent wants to make a stablecoin payment to another agent. Using the payer's owner policy, its spend so far today, and its history with this payee, decide: \"pay\", \"decline\", or \"ask_owner\" (allowed by policy but unusual enough that a human should confirm). Reply with JSON only: {\"decision\", \"category\", \"reason\", \"suggested_fix\"}. Keep the reason to one sentence an owner can read. suggested_fix is null when nothing needs changing."
situation = {'payment': {'payer': 'Translator',
             'payee': 'Data Broker',
             'amount_usd': 6.2,
             'memo': 'Company filings bundle',
             'resource': 'https://broker.example/filings',
             'via': 'x402 payment request'},
 'x402_requirements': {'x402Version': 1,
                       'scheme': 'exact',
                       'network': 'stablemode-sandbox',
                       'maxAmountRequired': '6200000',
                       'asset': 'USDC',
                       'payTo': 'agent:5',
                       'resource': 'https://broker.example/filings',
                       'description': 'Company filings bundle',
                       'requestId': 812,
                       'expiresAt': '2026-11-20T10:15:00Z'},
 'payer_policy': {'status': 'active',
                  'per_payment_cap_usd': 4.0,
                  'daily_budget_usd': 250.0,
                  'approved_payees': ['Compute Provider'],
                  'policy_version': 1},
 'spend_today': {'spent_usd': 31.4, 'remaining_usd': 218.6},
 'history_with_payee': {'settled_payments': 0, 'median_amount_usd': None, 'last_amount_usd': None}}

prompt = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM}, {"role": "user", "content": json.dumps(situation)}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# {"decision": "decline", "category": "payee_not_approved", "reason": "Data Broker is not on Translator's owner-approved payee list.", ...}

The LoRA adapter alone (170 MB) is in adapter/. Load it on top of Qwen/Qwen3.5-4B with PEFT.

Training

  • Data: 2,253 training, 299 validation and 448 held-out test examples, split by decision. ask_owner examples were oversampled 3× in training.
  • Method: LoRA (rank 32, all attention and MLP projections), 2 epochs, learning rate 1.5e-4 with a cosine schedule, and loss only on the answer.
  • Compute: one A100 80 GB on Hugging Face Jobs. Training took about 16 minutes; the whole job, including both evaluations, took about 23 minutes and cost about $1.
  • Code: scripts/ holds the dataset builder, training, evaluation and the full job script, so the run is reproducible.

Limitations: please read

  • Sandbox data. It was trained on a simulated agent economy from the Stablemode sandbox, not real payments. Treat these scores as performance on that distribution.
  • Most decisions follow fixed rules. Pay and decline labels come from Stablemode's own policy engine, so the model is largely learning those rules. In production, keep Stablemode's policy check as the hard gate and use the model for explanations, fix suggestions and escalation.
  • "Ask the owner" labels are heuristics: a first large payment to a new payee, a price at least 2.5× the usual, or a payment that would use at least 85% of the daily budget. This is the class where it still makes mistakes (F1 0.89), and it gets meaningfully better only with real owner feedback.
  • Text-only. It was loaded and trained as a text model; Qwen3.5's vision features are not included.
  • Not compliance or fraud screening. It doesn't check sanctions, prompt injection or counterparty risk.

License

CC BY-NC 4.0: free for research, education and evaluation, with attribution. Commercial use needs a license from Stablemode; open a discussion in the Community tab. The base model, Qwen3.5-4B, is Apache 2.0.


Stablemode was founded by Amin (@aminheid). Spend Copilot is by Sriram Ramakrishnan (@sriram1983007-dev).