patronus-studio/husky-sight-tool-type-classifier-edge

Model Card for Husky Sight Tool Type Classifier Edge

Quantized ONNX builds of Husky Sight Tool Type Classifier for local / edge inference.

This repository contains the size- and memory-optimized ONNX exports of the FP32 model. For intended uses, class definitions, training data, benchmarks, and limitations, see the main model card.

Variants

Variant	Path	Quantization
FP16	onnx/fp16/model.onnx	FP16 weights
INT8	onnx/int8/model.onnx	Post-training dynamic INT8 (MatMul/Gemm)
INT8 + INT4 embeddings	onnx/int8_int4_embeddings/model.onnx	Dynamic INT8 plus 4-bit weight-only token embeddings

The exact quantization parameters are recorded in onnx/quantization_manifest.json.

Benchmark

Quantization parity on the unified_v3 held-out test split (n = 2914; max_length 128, CPU):

Variant	Accuracy	F1 (macro)	Δ F1 vs FP32
FP32 reference (main repo)	0.9568	0.9567	—
FP16	0.9557	0.9554	-0.0013
INT8	0.9574	0.9573	+0.0006
INT8 + INT4 embeddings	0.9550	0.9548	-0.0019

Measured CPU latency (batch 64): ~18.89 ms/text for the INT8+INT4 build.

Full details in metrics/quant_bench.json.

Usage

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer

model_id = "patronus-studio/husky-sight-tool-type-classifier-edge"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSequenceClassification.from_pretrained(model_id, subfolder="onnx/int8_int4_embeddings")

inputs = tokenizer("...", return_tensors="pt")
logits = model(**inputs).logits

Post-process the logits as described in the main model card (softmax over the logits, argmax).

License

Released under the Apache License 2.0; the full text is included as LICENSE. Derived from Husky Sight Tool Type Classifier and ultimately from jhu-clsp/mmBERT-small (MIT License); upstream notices are retained.

Patronus Ark

This model is built to run inside Patronus Ark, Patronus' open-source on-device AI-security scanning library (L1 native rules → L2 NTDB cascade → L3 transformer). Ark is not publicly released yet — a repository link will be added here at launch.

---

🛡️ Patronus Protect

Brought to you by Patronus Protect — a local AI firewall that secures every AI interaction, including prompts, tools and documents, before it reaches your models.

Try it for free at patronus.studio.