bogdanraduta commited on
Commit
b74f209
·
verified ·
1 Parent(s): 68207cd

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +122 -0
README.md ADDED
@@ -0,0 +1,122 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: onnx
4
+ pipeline_tag: text-classification
5
+ tags:
6
+ - guardrails
7
+ - onnx
8
+ - multilingual
9
+ - topic_scope
10
+ language:
11
+ - az
12
+ - bg
13
+ - cs
14
+ - da
15
+ - de
16
+ - el
17
+ - en
18
+ - es
19
+ - et
20
+ - fi
21
+ - fr
22
+ - ga
23
+ - hr
24
+ - hu
25
+ - it
26
+ - lt
27
+ - lv
28
+ - mt
29
+ - nl
30
+ - pl
31
+ - pt
32
+ - ro
33
+ - sk
34
+ - sl
35
+ - sv
36
+ - tr
37
+ ---
38
+
39
+ # flowxai/topic-scope
40
+
41
+ The `topic_scope` detector for [flowx-border](https://github.com/flowx-ai/border), an embeddable library that inspects the text going into and coming out of an LLM and returns a structured decision plus an audit-grade evidence record.
42
+
43
+ This card is generated from the evaluation and export artifacts of the training run, so every number on it is reproducible from this repository rather than asserted.
44
+
45
+ ## What it is
46
+
47
+ - **Base model**: FacebookAI/xlm-roberta-base
48
+ - **Head**: single_label_classification
49
+ - **Labels**: not recorded
50
+ - **Artifact**: `onnx/model.int8.onnx`, 533 MB, opset 17
51
+ - **Trained at**: 96 tokens
52
+
53
+ ## Operating point
54
+
55
+ This head is read with argmax and has no threshold.
56
+
57
+ ## Per language
58
+
59
+ Per language rather than an aggregate, because an aggregate across 26 languages hides the tail and the tail is the point.
60
+
61
+ | Language | Support | P | R | F1 | Note |
62
+ |---|---|---|---|---|---|
63
+ | `az` Azerbaijani | 0 | 0.000 | 0.000 | 0.000 | |
64
+ | `bg` Bulgarian | 0 | 0.000 | 0.000 | 0.000 | |
65
+ | `cs` Czech | 0 | 0.000 | 0.000 | 0.000 | |
66
+ | `da` Danish | 0 | 0.000 | 0.000 | 0.000 | |
67
+ | `de` German | 0 | 0.000 | 0.000 | 0.000 | |
68
+ | `el` Greek | 0 | 0.000 | 0.000 | 0.000 | |
69
+ | `en` English | 0 | 0.000 | 0.000 | 0.000 | |
70
+ | `es` Spanish | 0 | 0.000 | 0.000 | 0.000 | |
71
+ | `et` Estonian | 0 | 0.000 | 0.000 | 0.000 | |
72
+ | `fi` Finnish | 0 | 0.000 | 0.000 | 0.000 | |
73
+ | `fr` French | 0 | 0.000 | 0.000 | 0.000 | |
74
+ | `ga` Irish | 0 | 0.000 | 0.000 | 0.000 | |
75
+ | `hr` Croatian | 0 | 0.000 | 0.000 | 0.000 | |
76
+ | `hu` Hungarian | 0 | 0.000 | 0.000 | 0.000 | |
77
+ | `it` Italian | 0 | 0.000 | 0.000 | 0.000 | |
78
+ | `lt` Lithuanian | 0 | 0.000 | 0.000 | 0.000 | |
79
+ | `lv` Latvian | 0 | 0.000 | 0.000 | 0.000 | |
80
+ | `mt` Maltese | 0 | 0.000 | 0.000 | 0.000 | not in base model pretraining |
81
+ | `nl` Dutch | 0 | 0.000 | 0.000 | 0.000 | |
82
+ | `pl` Polish | 0 | 0.000 | 0.000 | 0.000 | |
83
+ | `pt` Portuguese | 0 | 0.000 | 0.000 | 0.000 | |
84
+ | `ro` Romanian | 0 | 0.000 | 0.000 | 0.000 | |
85
+ | `sk` Slovak | 0 | 0.000 | 0.000 | 0.000 | |
86
+ | `sl` Slovenian | 0 | 0.000 | 0.000 | 0.000 | |
87
+ | `sv` Swedish | 0 | 0.000 | 0.000 | 0.000 | |
88
+ | `tr` Turkish | 0 | 0.000 | 0.000 | 0.000 | |
89
+
90
+ ### Weakest languages
91
+
92
+ Published rather than dropped. A coverage table with the bad rows removed is not a coverage table.
93
+
94
+ - `az` Azerbaijani: F1 0.000
95
+ - `bg` Bulgarian: F1 0.000
96
+ - `cs` Czech: F1 0.000
97
+
98
+ ## Quantisation
99
+
100
+ The published artifact is INT8, and **only the embedding table is quantised**.
101
+
102
+ Quantising everything is what most examples do and it does not work for this base model. Measured on 300 real test texts at the detector's own threshold:
103
+
104
+ | Recipe | Size | Mean logit drift | Decisions changed |
105
+ |---|---|---|---|
106
+ | all ops (the usual default) | 279 MB | 0.68 | 51 / 300 |
107
+ | MatMul only | 856 MB | 0.64 | 48 / 300 |
108
+ | **Gather only, what ships here** | **535 MB** | **0.0036** | **0 / 300** |
109
+
110
+ The embedding table carries the whole size win at no accuracy cost, while quantising the encoder MatMuls changes one decision in six to save 256 MB. XLM-RoBERTa has large activation outliers and per-tensor dynamic quantisation of activations is exactly what they defeat.
111
+
112
+ For this artifact specifically: **? of 200 decisions differ** from the fp32 checkpoint, mean logit drift 0.0000, read as `argmax`. A quantised model that answers differently is a different detector, so this is measured rather than assumed.
113
+
114
+ ## Limitations
115
+
116
+ - **Synthetic training data.** Generated natively per language, never translated from English, so the sentence structure is the target language's own. It is still synthetic, and a production distribution will differ.
117
+ - **Maltese is absent from XLM-RoBERTa's pretraining set.** That is a fact about the base model, and it is not an explanation for a weak score. This card said "no amount of data fixes that" until 2026-08-14, which this project's own measurement disproves: the `nsfw` detector scored 0.000 in Maltese, was blamed on the base model, and went to 1.000 with perfect precision and recall when its corpus went from 2 positives per language to 10. Nothing about the model changed. So where a language scores badly here, read the support column first.
118
+ - **This is not a compliance product.** It produces evidence about controls that were applied. It does not make anyone compliant with anything, and the obligations under the EU AI Act sit with the provider or deployer of a system, not with a model or a library.
119
+
120
+ ## Licence
121
+
122
+ Apache-2.0, declared in the metadata above as well as here, so that a tool reading the repository can attest it rather than a human having to read prose.