f1k commited on
Commit
28ef330
·
verified ·
1 Parent(s): 40d13f0

Katip Mask Base · tr v2: CC BY-SA 4.0, credit BERTurk and Turkish Wiki NER

Browse files
Files changed (1) hide show
  1. README.md +23 -0
README.md CHANGED
@@ -18,6 +18,29 @@ BERTurk (`dbmdz/bert-base-turkish-cased`) fine-tuned to tag **person names, addr
18
  art. 6 health terms** in Turkish legal documents, as the on-device model for
19
  [Katip Mask](https://huggingface.co/spaces/f1k/katip-kvkk-anonymization). Exported to ONNX (fp32 and int8) for transformers.js.
20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  **v2 (13 Nov 2026)** adds 17,967 real Turkish sentences (Turkish Wiki NER, person names only) to the
22
  synthetic training documents. v1 found every name in Katip's synthetic test sets but only 41% of names in
23
  real Turkish sentences; v2 finds 90%.
 
18
  art. 6 health terms** in Turkish legal documents, as the on-device model for
19
  [Katip Mask](https://huggingface.co/spaces/f1k/katip-kvkk-anonymization). Exported to ONNX (fp32 and int8) for transformers.js.
20
 
21
+ ### KVKK-Bench (v0.1.2)
22
+
23
+ | # | System | Overall | Turkish names (court) | Foreign names (court) | Everyday text | Forms ◇ | Size MB |
24
+ |---|---|---|---|---|---|---|---|
25
+ | 1 | [Katip Mask Base · tr v2](https://huggingface.co/f1k/katip-mask-base) | 96.1 | 99.9 | 97.7 | 84.8 | 98.3 | 111.1 |
26
+ | 2 | [XLM-RoBERTa Large NER](https://huggingface.co/FacebookAI/xlm-roberta-large-finetuned-conll03-english) | 95.9 | 97.6 | 97.5 | 89.9 | 41.6 | 2235.4 |
27
+ | 3 | [XLM-RoBERTa Base Turkish NER](https://huggingface.co/akdeniz27/xlm-roberta-base-turkish-ner) | 95.7 | 98.5 | 98.2 | 86.3 | – | 1109.8 |
28
+ | 4 | [Katip rules + GLiNER2 · tr fine-tuned](https://github.com/fikrifirat/katip/blob/legal-study/lab/colab/gliner2_katip_recipe.ipynb) | 95.5 | 98.7 | 97.3 | 86.0 | 98.2 | 1228.7 |
29
+ | 5 | [GLiNER2 PII Multi](https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi) | 95.0 | 98.1 | 98.2 | 83.9 | 43.2 | 1228.4 |
30
+ | 6 | [Katip Mask Mini · tr v2](https://huggingface.co/f1k/katip-mask-mini) | 93.0 | 99.1 | 97.2 | 73.4 | 97.5 | 31.4 |
31
+ | 7 | [BERTurk NER](https://huggingface.co/savasy/bert-base-turkish-ner-cased) | 91.6 | 97.4 | 91.7 | 77.1 | 38.3 | 440.1 |
32
+ | 8 | [GLiNER Multi PII v1](https://huggingface.co/urchade/gliner_multi_pii-v1) | 91.3 | 93.1 | 92.9 | 85.2 | – | 1155.8 |
33
+ | 9 | [Katip Mask Nano · tr v1.1](https://huggingface.co/f1k/katip-mask-nano) | 85.9 | 94.4 | 95.5 | 55.2 | 96.6 | 2.9 |
34
+ | 10 | [ModernBERT-TR PII NER](https://huggingface.co/ytu-ce-cosmos/modernbert-tr-pii-ner) | 85.3 | 99.0 | 96.9 | 39.4 | – | 597.5 |
35
+ | 11 | [Qwen2.5 1.5B Instruct (zero-shot)](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) | 57.5 | 59.1 | 57.6 | 53.7 | – | 3087 |
36
+ | 12 | [Privacy Filter Multilingual v2](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2) | 57.2 | 59.8 | 57.4 | 50.7 | – | 2799.2 |
37
+ | 13 | [Privacy Filter](https://huggingface.co/openai/privacy-filter) | 51.3 | 58.0 | 58.0 | 27.9 | – | 2799 |
38
+ | 14 | [Piiranha v1](https://huggingface.co/iiiorg/piiranha-v1-detect-personal-information) | 49.8 | 41.4 | 61.5 | 59.4 | – | 1112.9 |
39
+ | 15 | [Presidio](https://github.com/microsoft/presidio) | 45.0 | 39.3 | 44.0 | 60.4 | 36.1 | 11 |
40
+ | 16 | [OpenMed PII Turkish Small](https://huggingface.co/OpenMed/OpenMed-PII-Turkish-ClinicalE5-Small-33M-v1) | 19.1 | 11.8 | 18.8 | 37.5 | – | 133 |
41
+
42
+ KVKK-Bench v0.1.2, scores out of 100, higher is better. *Overall* is the weighted F2 score for finding names in real text: Yargıtay decisions with Turkish and foreign names, and everyday Wikipedia sentences. *Forms* ◇ covers T.C. kimlik, tax numbers, IBANs and addresses on 306 synthetic petitions and bar forms; it is not in Overall, and blank means not run. Rows named Katip include Katip's rules. Live table and method: https://huggingface.co/spaces/f1k/kvkk-bench
43
+
44
  **v2 (13 Nov 2026)** adds 17,967 real Turkish sentences (Turkish Wiki NER, person names only) to the
45
  synthetic training documents. v1 found every name in Katip's synthetic test sets but only 41% of names in
46
  real Turkish sentences; v2 finds 90%.