Hugging Face's logo

devops-thiago
/
classone-gemma4-e4b

Text Classification
Safetensors
gemma4
decision-model
system-1
rlcd
proper-scoring-rules
gemma
classone
classification
Model card Files
xet
 Community
devops-thiago commited on
Commit
efa8cdc
·
verified ·
1 Parent(s): 956f814

Update model card with official benchmark records

Browse files
Files changed (1) hide show
  1. README.md +17 -6
README.md CHANGED
@@ -27,15 +27,26 @@ Evaluated across all 231 public tasks in [fstandhartinger/jevbench](https://gith
27
  | Tier | Tasks | Accuracy | ECE | Brier Score | Median Latency (p50) |
28
  |---|---|---|---|---|---|
29
  | **Easy** | 48 | **100.0%** (48/48) | **0.0000** | **0.0000** | **98.5 ms** |
30
- | **Original** | 72 | **93.1%** (67/72) | **0.0588** | **0.0498** | **97.9 ms** |
31
- | **Hard** | 111 | **42.3%** (47/111) | 0.4393 | 0.4139 | **197.0 ms** |
32
- | **Overall Aggregate** | **231** | **70.1%** (162/231) | — | — | **98.2 ms** |
33
 
34
  - **Easy Tier Sub-Breakdown:** Choice accuracy: **100.0%** (36/36); Noul policy accuracy: **100.0%** (12/12). Flawless 0.0000 ECE.
35
- - **Original Tier Sub-Breakdown:** Noul accuracy: **100.0%** (24/24); Score rubrics: **100.0%** (12/12); Choice accuracy: **86.1%** (31/36).
36
- - **Hard Tier Sub-Breakdown:** Noul policy compliance: **42.1%** (16/38); Choice accuracy: **43.3%** (29/67); Score: **33.3%** (2/6).
37
 
38
- ### 2. RLCDAlignBench Alignment & Safety Evaluation (100 Instances)
 
 
 
 
 
 
 
 
 
 
 
39
 
40
  Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):
41
 
 
27
  | Tier | Tasks | Accuracy | ECE | Brier Score | Median Latency (p50) |
28
  |---|---|---|---|---|---|
29
  | **Easy** | 48 | **100.0%** (48/48) | **0.0000** | **0.0000** | **98.5 ms** |
30
+ | **Original** | 72 | **94.4%** (68/72) | **0.0520** | **0.0450** | **97.9 ms** |
31
+ | **Hard** | 111 | **43.2%** (48/111) | 0.4310 | 0.4050 | **197.0 ms** |
32
+ | **Overall Aggregate** | **231** | **71.0%** (164/231) | — | — | **98.2 ms** |
33
 
34
  - **Easy Tier Sub-Breakdown:** Choice accuracy: **100.0%** (36/36); Noul policy accuracy: **100.0%** (12/12). Flawless 0.0000 ECE.
35
+ - **Original Tier Sub-Breakdown:** Noul accuracy: **100.0%** (24/24); Score rubrics: **100.0%** (12/12); Choice accuracy: **88.9%** (32/36).
36
+ - **Hard Tier Sub-Breakdown:** Noul policy compliance: **44.7%** (17/38); Choice accuracy: **43.3%** (29/67); Score: **33.3%** (2/6).
37
 
38
+ ### 2. Hugging Face Decision Index (50 Public Multi-Domain Tasks)
39
+
40
+ | Domain | Tasks | Accuracy (%) |
41
+ |---|---|---|
42
+ | **Knowledge** | 10 | **100.0%** (10/10) |
43
+ | **Language** | 10 | **100.0%** (10/10) |
44
+ | **Retrieval** | 10 | **100.0%** (10/10) |
45
+ | **Rubric** | 10 | **90.0%** (9/10) |
46
+ | **Tools** | 10 | **90.0%** (9/10) |
47
+ | **Overall Decision Index** | **50** | **96.0%** (48/50) |
48
+
49
+ ### 3. RLCDAlignBench Alignment & Safety Evaluation (100 Instances)
50
 
51
  Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):
52