xiaoxiae commited on
Commit
b1c4e35
·
verified ·
1 Parent(s): 3db92b8

add README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - tr
5
+ library_name: onnx
6
+ pipeline_tag: token-classification
7
+ base_model: akdeniz27/bert-base-turkish-cased-ner
8
+ tags:
9
+ - pii
10
+ - ner
11
+ - token-classification
12
+ - bert
13
+ - turkish
14
+ - onnx
15
+ - int8
16
+ - onnxruntime
17
+ ---
18
+
19
+ # pii-ner-model
20
+
21
+ Dynamic-INT8 **ONNX** export of [`akdeniz27/bert-base-turkish-cased-ner`](https://huggingface.co/akdeniz27/bert-base-turkish-cased-ner)
22
+ (BERTurk, MIT). It detects free-text PII — **names and addresses** — that a deterministic
23
+ regex masker can't catch, and runs **in-process via `onnxruntime` (no torch)**.
24
+
25
+ Freya's voice agent loads it for freeform-PII redaction (`src/privacy/ner.py`,
26
+ `LocalPiiDetector`); the agent image fetches this repo at build into `PII_NER_MODEL_DIR`.
27
+ NER is optional + fail-open and gated per-agent by `privacy_config.mask_pii`.
28
+
29
+ ## Files
30
+
31
+ | file | what |
32
+ |------|------|
33
+ | `model.int8.onnx` | dynamic-INT8-quantized BERTurk token-classification model (~106 MB) |
34
+ | `tokenizer.json` | Rust-tokenizer config for the `onnxruntime` path |
35
+ | `config.json` | `id2label` map for decode |
36
+ | `export_model.py` | the offline recipe that produced the artifacts (not used at runtime) |
37
+
38
+ ## Labels
39
+
40
+ 7-class BIO: `O`, `B-PER`/`I-PER`, `B-ORG`/`I-ORG`, `B-LOC`/`I-LOC`. Downstream mapping:
41
+ `PER -> NAME`, `LOC -> ADDRESS`; `ORG` is dropped.
42
+
43
+ ## Quality
44
+
45
+ Validated on Turkish: names F1 ~1.00 (cased) / ~0.93–0.95 (ASR-style lowercase). INT8 is
46
+ effectively lossless vs fp32 on cased text. Addresses (`LOC`) are weaker on lowercase ASR text.
47
+
48
+ ## Regenerating
49
+
50
+ Needs torch + `optimum[onnxruntime]` (not runtime deps):
51
+
52
+ ```bash
53
+ pip install torch --index-url https://download.pytorch.org/whl/cpu
54
+ pip install "optimum[onnxruntime]" transformers
55
+ python export_model.py --model akdeniz27/bert-base-turkish-cased-ner --out /tmp/pii-ner
56
+ # then copy model_quantized.onnx -> model.int8.onnx, plus tokenizer.json + config.json
57
+ ```
58
+
59
+ ## License
60
+
61
+ MIT — same as the base model. See `LICENSE`. Base model: `akdeniz27/bert-base-turkish-cased-ner`.