Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -36,9 +36,11 @@ language:
|
|
| 36 |
- tr
|
| 37 |
---
|
| 38 |
|
| 39 |
-
#
|
| 40 |
|
| 41 |
-
The `toxicity` detector for [
|
|
|
|
|
|
|
| 42 |
|
| 43 |
This card is generated from the evaluation and export artifacts of the training run, so every number on it is reproducible from this repository rather than asserted.
|
| 44 |
|
|
@@ -59,6 +61,57 @@ This number is not decoration. Read at the 0.5 default that looked reasonable, s
|
|
| 59 |
- At the 0.5 default: 0.989
|
| 60 |
- At the calibrated 0.81: 0.989
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
## Per language
|
| 63 |
|
| 64 |
Per language rather than an aggregate, because an aggregate across 26 languages hides the tail and the tail is the point.
|
|
|
|
| 36 |
- tr
|
| 37 |
---
|
| 38 |
|
| 39 |
+
# toxicity[border]
|
| 40 |
|
| 41 |
+
The `toxicity` detector for [border](https://github.com/flowx-ai/border), an embeddable library that inspects the text going into and coming out of an LLM and returns a structured decision plus an audit-grade evidence record.
|
| 42 |
+
|
| 43 |
+
`flowxai/toxicity` on the hub. It is one detector of 28, and it is not a general purpose toxicity classifier: it was trained for this library's policy, is read at the operating point below, and reports through the evidence record rather than returning a bare score.
|
| 44 |
|
| 45 |
This card is generated from the evaluation and export artifacts of the training run, so every number on it is reproducible from this repository rather than asserted.
|
| 46 |
|
|
|
|
| 61 |
- At the 0.5 default: 0.989
|
| 62 |
- At the calibrated 0.81: 0.989
|
| 63 |
|
| 64 |
+
## How to use it
|
| 65 |
+
|
| 66 |
+
Through the library, which is what this model is for. It loads the artifact below, applies the operating point above, and returns a decision with an evidence record rather than a bare score.
|
| 67 |
+
|
| 68 |
+
```sh
|
| 69 |
+
pip install flowx-border
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
```yaml
|
| 73 |
+
# policy.yaml
|
| 74 |
+
policy_id: default
|
| 75 |
+
version: 1
|
| 76 |
+
|
| 77 |
+
detectors:
|
| 78 |
+
toxicity:
|
| 79 |
+
enabled: true
|
| 80 |
+
on_fail: flag
|
| 81 |
+
threshold: 0.81
|
| 82 |
+
```
|
| 83 |
+
|
| 84 |
+
```python
|
| 85 |
+
from flowx_border import load_policy, scan_input, scan_output
|
| 86 |
+
|
| 87 |
+
policy = load_policy("policy.yaml")
|
| 88 |
+
|
| 89 |
+
decision = scan_input(user_text, policy)
|
| 90 |
+
decision = scan_output(model_answer, policy)
|
| 91 |
+
|
| 92 |
+
print(decision.verdict) # allow | flag | redact | block
|
| 93 |
+
print([f.label for f in decision.findings if f.detector_id == "toxicity"])
|
| 94 |
+
print(decision.evidence.record_id)
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
This detector reads the input and output side, so `scan_input` and `scan_output` is where it fires. It is T2, so it runs on the standard path and can be disabled per policy. Its budget is 225 ms at 87 tokens on one CPU thread.
|
| 98 |
+
|
| 99 |
+
The weights are fetched once and cached, and a scan needs no network after that. Nothing here calls out to a hosted model, and the evidence record carries hashes rather than your text.
|
| 100 |
+
|
| 101 |
+
### Without the library
|
| 102 |
+
|
| 103 |
+
The artifact is plain ONNX, so it will load in `onnxruntime` directly. Two things you then own yourself, and they are the reason the library exists: the operating point above is not in the graph, and neither is the chunking. Inputs longer than the trained window have to be split and recombined, or the scores past it are extrapolation.
|
| 104 |
+
|
| 105 |
+
```python
|
| 106 |
+
import onnxruntime as ort
|
| 107 |
+
from huggingface_hub import hf_hub_download
|
| 108 |
+
from tokenizers import Tokenizer
|
| 109 |
+
|
| 110 |
+
repo = "flowxai/toxicity"
|
| 111 |
+
session = ort.InferenceSession(hf_hub_download(repo, "onnx/model.int8.onnx"))
|
| 112 |
+
tokenizer = Tokenizer.from_file(hf_hub_download(repo, "tokenizer.json"))
|
| 113 |
+
```
|
| 114 |
+
|
| 115 |
## Per language
|
| 116 |
|
| 117 |
Per language rather than an aggregate, because an aggregate across 26 languages hides the tail and the tail is the point.
|