license: apache-2.0
language:
- de
- en
base_model:
- patronus-studio/husky-sight-tool-type-classifier
pipeline_tag: text-classification
tags:
- tool-use
- ai-agents
- function-calling
- mcp
- tool-security
- security
- llm-security
- ai-safety
- ai-agent-security
- patronus
- multilingual
- modernbert
- onnx
- quantized
- edge
Model Card for Husky Sight Tool Type Classifier Edge
Quantized ONNX builds of Husky Sight Tool Type Classifier for local / edge inference.
This repository contains the size- and memory-optimized ONNX exports of the FP32 model. For intended uses, class definitions, training data, benchmarks, and limitations, see the main model card.
Variants
| Variant | Path | Quantization |
|---|---|---|
| FP16 | onnx/fp16/model.onnx |
FP16 weights |
| INT8 | onnx/int8/model.onnx |
Post-training dynamic INT8 (MatMul/Gemm) |
| INT8 + INT4 embeddings | onnx/int8_int4_embeddings/model.onnx |
Dynamic INT8 plus 4-bit weight-only token embeddings |
The exact quantization parameters are recorded in onnx/quantization_manifest.json.
Benchmark
Quantization parity on the unified_v3 held-out test split (n = 2914; max_length 128, CPU):
| Variant | Accuracy | F1 (macro) | Δ F1 vs FP32 |
|---|---|---|---|
| FP32 reference (main repo) | 0.9568 | 0.9567 | — |
| FP16 | 0.9557 | 0.9554 | -0.0013 |
| INT8 | 0.9574 | 0.9573 | +0.0006 |
| INT8 + INT4 embeddings | 0.9550 | 0.9548 | -0.0019 |
Measured CPU latency (batch 64): ~18.89 ms/text for the INT8+INT4 build.
Full details in metrics/quant_bench.json.
Usage
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer
model_id = "patronus-studio/husky-sight-tool-type-classifier-edge"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSequenceClassification.from_pretrained(model_id, subfolder="onnx/int8_int4_embeddings")
inputs = tokenizer("...", return_tensors="pt")
logits = model(**inputs).logits
Post-process the logits as described in the main model card (softmax over the logits, argmax).
License
Released under the Apache License 2.0; the full
text is included as LICENSE. Derived from
Husky Sight Tool Type Classifier and ultimately from
jhu-clsp/mmBERT-small (MIT License); upstream
notices are retained.
Patronus Ark
This model is built to run inside Patronus Ark, Patronus' open-source on-device AI-security scanning library (L1 native rules → L2 NTDB cascade → L3 transformer). Ark is not publicly released yet — a repository link will be added here at launch.
🛡️ Patronus Protect
Brought to you by Patronus Protect — a local AI firewall that secures every AI interaction, including prompts, tools and documents, before it reaches your models.
Try it for free at patronus.studio.