---
license: apache-2.0
tags:
- document-classification
- topic-classification
- dlp
- data-loss-prevention
- security
- llm-security
- ai-safety
- ai-agent-security
- patronus
- multilingual
- modernbert
- onnx
- quantized
- edge
base_model: patronus-studio/orca-sonar-document-classifier
pipeline_tag: text-classification
language:
- de
- en
---

# Model Card for Orca Sonar Document Classifier Edge

**Edge/quantized ONNX variant of [`patronus-studio/orca-sonar-document-classifier`](https://huggingface.co/patronus-studio/orca-sonar-document-classifier).** Private / internal.

Quantized with the Patronus RunPod ONNX method (`runpod/quantize_single_heads.py`):
`quantize_dynamic` for INT8 linears + `MatMulNBitsQuantizer` (4-bit, block 128) for the
embedding. The parent FP32/FP16 model is unchanged; this repo only adds the small variants.

7-class document topic classifier (mmBERT-small / ModernBERT). Metrics measured on the held-out document-classifier test set; FP32 reproduces the parent card (macro-F1 0.978).

## Variants (measured, essentially lossless)

| Variante | Größe | macro-F1 | Δ vs FP32 | Accuracy |
|---|---|---|---|---|
| fp16 | 282 MB | 0.9783 | +0.0000 | 0.9783 |
| int8 | 142 MB | 0.9793 | +0.0011 | 0.9792 |
| **int8_int4_embeddings** | **96 MB** | **0.9790** | **+0.0007** | 0.9792 |

**Recommended: `onnx/int8_int4_embeddings/model.onnx`** — ~5.5× smaller than FP32 at ~0 quality loss.

## Files
- `onnx/fp16/model.onnx`, `onnx/int8/model.onnx`, `onnx/int8_int4_embeddings/model.onnx`
- `config.json`, `tokenizer.json`, `tokenizer_config.json` (from the parent)
- `metrics/quant_bench.json` — the full FP32-vs-quant benchmark

## Usage
```python
import onnxruntime as ort, numpy as np
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("patronus-studio/orca-sonar-document-classifier-edge")
sess = ort.InferenceSession("onnx/int8_int4_embeddings/model.onnx", providers=["CPUExecutionProvider"])
enc = tok("your text", truncation=True, max_length=256, padding="max_length", return_tensors="np")
logits = sess.run(["logits"], {"input_ids": enc["input_ids"].astype(np.int64),
                               "attention_mask": enc["attention_mask"].astype(np.int64)})[0]
```

## License

This model is released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
A copy of the license is included as `LICENSE` in this repository.

## Patronus Ark

This model is built to run inside **Patronus Ark**, Patronus' open-source on-device
AI-security scanning library (L1 native rules → L2 NTDB cascade → L3 transformer).
Ark is not publicly released yet — a repository link will be added here at launch.

---

## 🛡️ Patronus Protect

Brought to you by **[Patronus Protect](https://patronus.studio)** — a local AI firewall that
secures every AI interaction (prompts, tools, documents) before it reaches your models.
**Try it for free at [patronus.studio](https://patronus.studio).**