Upgrade model card: badges, metrics, examples, citations, cross-references

d1fee90 verified about 1 month ago

7.22 kB

library_name: onnx
pipeline_tag: text-classification
license: apache-2.0
language:
  - en
tags:
  - intentguard
  - guardrails
  - llm-safety
  - content-moderation
  - legal
  - deberta-v2
  - onnx-runtime
  - intent-classification
  - chatbot-security
model-index:
  - name: intentguard-legal
    results:
      - task:
          type: text-classification
          name: Intent Classification
        metrics:
          - name: Accuracy
            type: accuracy
            value: 97.9
          - name: Legitimate Block Rate
            type: accuracy
            value: 0
          - name: Off-Topic Pass Rate
            type: accuracy
            value: 0.5

IntentGuard — Legal & Compliance

Production-ready vertical intent classifier for LLM chatbot guardrails. Classifies user messages as allow, deny, or abstain to keep legal chatbots on-topic and within professional boundaries.

Research Article | perfecXion.ai | Finance Model | Healthcare Model | Legal Model

IntentGuard Model Family

Model	Vertical	Accuracy	Off-Topic Pass Rate	Link
intentguard-finance	Financial Services	99.6%	0.00%	perfecXion/intentguard-finance
intentguard-healthcare	Healthcare & Clinical	98.9%	0.98%	perfecXion/intentguard-healthcare
intentguard-legal	Legal & Compliance	97.9%	0.50%	This model

Overview

The Problem

Legal chatbots carry significant liability risk — unauthorized practice of law, confidentiality breaches, and jurisdictional issues demand strict topic boundaries. An LLM that happily answers questions about celebrity gossip or sports while branded as a legal assistant undermines professional credibility and trust.

The Solution

IntentGuard uses a tiny DeBERTa-v3-xsmall model (22M parameters, 2.5MB quantized) to classify user intent in <30ms on CPU with three-way classification:

Allow — On-topic legal query, pass to the LLM
Deny — Off-topic, block with a polite redirect
Abstain — Ambiguous, escalate to secondary classifier or human review

Performance

Metric	Value
Overall Accuracy	97.9%
Legitimate Block Rate	0.00%
Off-Topic Pass Rate	0.50%
p99 Latency (CPU)	<30ms
Model Size (ONNX INT8)	2.5MB
Base Parameters	22M (DeBERTa-v3-xsmall)
Expected Calibration Error	<0.03

Model Details

Property	Value
Architecture	DeBERTa-v3-xsmall (fine-tuned for 3-way classification)
Format	ONNX (INT8 quantized)
Version	1.0
Vertical	Legal (Law & Compliance)
GPU Required	No — runs on CPU
Publisher	perfecXion.ai

Core Topics (Allow)

Contracts, litigation, employment law, intellectual property, criminal law, family law, real estate law, immigration, corporate law, compliance, privacy law, civil rights, estate planning, bankruptcy

Hard Exclusions (Deny)

Sports, entertainment, cooking, gaming, celebrity gossip, fashion, travel/leisure, fiction writing, relationship advice

Usage

Python (ONNX Runtime)

import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

tokenizer = AutoTokenizer.from_pretrained("perfecXion/intentguard-legal")
session = ort.InferenceSession("model.onnx")

text = "What are the requirements for filing a patent application?"
inputs = tokenizer(text, return_tensors="np", max_length=128, truncation=True, padding="max_length")

logits = session.run(None, {
    "input_ids": inputs["input_ids"],
    "attention_mask": inputs["attention_mask"]
})[0]

labels = ["allow", "deny", "abstain"]
prediction = labels[np.argmax(logits)]
confidence = float(np.max(np.exp(logits) / np.sum(np.exp(logits))))

print(f"Intent: {prediction} (confidence: {confidence:.3f})")
# Output: Intent: allow (confidence: 0.996)

Docker

docker pull ghcr.io/perfecxion/intentguard:legal-1.0
docker run -p 8080:8080 ghcr.io/perfecxion/intentguard:legal-1.0

curl -X POST http://localhost:8080/v1/classify \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "What are my rights as a tenant?"}]}'

pip

pip install intentguard

from intentguard import IntentGuard
guard = IntentGuard.load("legal")
result = guard.classify("What are the requirements for filing a patent?")
print(result)  # Intent(label='allow', confidence=0.996)

Example Classifications

User Message	Predicted	Confidence	Correct?
"What are the requirements for filing a patent?"	allow	0.996	✅
"Can my landlord evict me without notice?"	allow	0.995	✅
"Who won the Super Bowl?"	deny	0.999	✅
"Tell me a joke"	deny	0.996	✅
"Is GDPR compliance required for US companies?"	allow	0.989	✅
"What's the best recipe for pasta?"	deny	0.998	✅

Citation

@misc{thornton2025intentguard,
  title={IntentGuard: A Production-Grade Vertical Intent Classifier for LLM Guardrails},
  author={Thornton, Scott},
  year={2025},
  publisher={perfecXion.ai},
  url={https://perfecxion.ai/articles/intentguard-vertical-intent-classifier-llm-guardrails.html},
  note={Model: https://huggingface.co/perfecXion/intentguard-legal}
}

Quality Metrics

Metric	Result
Accuracy (Legal vertical)	97.9%
Legitimate Block Rate	0.00%
Off-Topic Pass Rate	0.50%
Expected Calibration Error	<0.03
ONNX INT8 Quantization	Validated
CPU Inference (p99)	<30ms

License

Apache 2.0

perfecXion
/

intentguard-legal