Model Card for zeroproof-ecommerce-1b

Release: v1.2

Frontier-approaching e-commerce intent detection in a 1B model. Given a live customer conversation, zeroproof-ecommerce-1b returns a single JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option.

On a hard, held-out benchmark it reaches 75.3% macro-averaged intent-type accuracy. On the 412-row frontier comparison subset it reaches 71.4%, 89% of GPT-5's 80.2%, at $0.18 per 1M output tokens and about a second per response.

Model Details

Model Description

• Developed by: Zero Proof Labs
• Model type: E-commerce payment-intent classifier; structured JSON output over seven intent types
• Language: English
• License: Gemma (inherited from the base model)
• Finetuned from: google/gemma-3-1b-it, via 4-bit QLoRA

Model Sources

• Repository: https://huggingface.co/zero-proof-ai/zeroproof-ecommerce-1b
• Training data: https://huggingface.co/datasets/zero-proof-ai/ecommerce-intent

Uses

Direct Use

Drop-in intent detection for e-commerce and payments support. Feed the conversation so far; the model returns one JSON object per message: whether an actionable payment intent is present, which of seven types it is, and the extracted details. Cheap and fast enough to call on every turn of every conversation.

Out-of-Scope Use

English-language e-commerce and payments only. It is a narrow intent classifier, not a general assistant, and should not be the sole authority for executing a payment without a downstream verification step.

Bias, Risks, and Limitations

Trained and evaluated on synthetic role-play conversations from a multi-agent marketplace simulation, labeled in a separate model pass under a locked labeling policy; labels are model-assigned and carry residual noise, and live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, and transfers; checkout and bill-pay are the focus of the current data round.

Recommendations

Pair the model with a verification layer before any payment executes, and re-benchmark on your own traffic before relying on the reported numbers in production.

How to Get Started with the Model

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")
tok   = AutoTokenizer.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")

Prompt it with the system prompt shipped in prompt_template.txt plus the conversation so far, and decode greedily (temperature=0).

Output format

The model returns one JSON object per message:

{"intent_detected": true, "core_type": "reverse",
 "details": {"action": "refund", "merchant": "Amazon", "amount": 81.40, "reason_code": "scam"},
 "confidence": 0.85, "reason": "Customer wants a refund on a suspicious charge.",
 "source_message_seqs": [2]}

core_type is one of spend, send, exchange, recur, bill, reverse, or none, and each type carries its own details fields.

Training Details

Training Data

The differentiator is the data. Zero Proof Labs builds e-commerce intent training data as randomized conversational simulations: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, simulated turn by turn between two independently drawn models. Generation is label-blind (the generating models never see the intent schema), labels are assigned in a separate pass, and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage.

This release was fine-tuned on 17,144 conversations. The held-out evaluation set contains 1,977 conversations and has no conversation or identifier overlap with the training split.

Training Procedure

Training Hyperparameters

• Method: 4-bit QLoRA (via unsloth)
• Epochs: 1
• Effective batch size: 16
• Learning rate: 2e-4
• Sequence length: 12288 (long conversations fit)

Evaluation

Testing Data, Factors & Metrics

Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored deterministically against gold: intent type by exact match, order details as the fraction of gold fields reproduced (partial credit per field). The frontier panel is scored on a 412-row balanced subset of the same eval.

Results

• Fine-tuning takes intent-type accuracy from 14.4% to 75.3% on the public six-action rubric, macro-averaged across the 1,977-row held-out evaluation.
• Intent detection reaches 80.7%, and structured order-detail extraction reaches 66.3%, both macro-averaged on the same public rubric.
• Action-level intent-type accuracy is 71.8% for bill payment and 78.5% for recurring payments.
• On the no-action boundary, 9.5% of true no-action conversations are marked actionable.

Technical Specifications

Model Architecture and Objective

A LoRA adapter over gemma-3-1b-it, trained response-only to emit one structured intent object per turn.

Compute Infrastructure

Served as an OpenAI-compatible endpoint (base + adapter) under vLLM. Measured on a Modal L4 GPU: $0.18 per 1M output tokens at batched capacity and ~1s per response (p50 984 ms, p95 1,191 ms), 55 to 140x below frontier list prices.

Model Card Contact

Zero Proof Labs, https://huggingface.co/zero-proof-ai