Update README.md
Browse files
README.md
CHANGED
|
@@ -219,5 +219,15 @@ uv pip install cloudpickle msgspec zmq blake3 cachetools prometheus_client fasta
|
|
| 219 |
|
| 220 |
### Accuracy Comparison
|
| 221 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 222 |
|
| 223 |
|
|
|
|
| 219 |
|
| 220 |
### Accuracy Comparison
|
| 221 |
|
| 222 |
+
| Category | Benchmark | ibm-granite/granite-4.0-h-tiny | RedHatAI/granite-4.0-h-tiny-FP8-block | Recovery (%) |
|
| 223 |
+
|:--|:--|:-:|:-:|:-:|
|
| 224 |
+
| **OpenLLM V1** | ARC-Challenge (Acc, 25-shot) | 62.97 | 63.40 | 100.68 |
|
| 225 |
+
| | GSM8K (Strict-Match, 5-shot) | 80.44 | 81.05 | 100.76 |
|
| 226 |
+
| | HellaSwag (Acc-Norm, 10-shot) | 61.75 | 61.79 | 100.06 |
|
| 227 |
+
| | MMLU (Acc, 5-shot) | 66.46 | 66.22 | 99.64 |
|
| 228 |
+
| | TruthfulQA (MC2, 0-shot) | 58.48 | 58.76 | 100.48 |
|
| 229 |
+
| | Winogrande (Acc, 5-shot) | 71.43 | 71.35 | 99.89 |
|
| 230 |
+
| | **Average** | **66.92** | **67.10** | **100.26** |
|
| 231 |
+
| **OpenLLM V2** | IFEval (Inst Level Strict Acc, 0-shot) | 70.62 | 69.30 | 98.13 |
|
| 232 |
|
| 233 |
|