Hugging Face's logo

RedHatAI
/
granite-3.1-2b-instruct-quantized.w8a8

Text Generation
Transformers
Safetensors
English
granite
w8a8
int8
vllm
conversational
8-bit precision
compressed-tensors
Model card Files
xet
Community
nm-research commited on
Commit
613a7f4
·
verified ·
1 Parent(s): 0d328c8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -214,12 +214,12 @@ evalplus.evaluate \
214
  | Math-Hard (Exact-Match, 4-shot) | 8.66 | 8.04 |
215
  | GPQA (Acc-Norm, 0-shot) | 28.30 | 27.60 |
216
  | MUSR (Acc-Norm, 0-shot) | 35.12 | 34.58 |
217
- | MMLU-Pro (Acc, 5-shot) | 26.87 | |
218
- | **Average Score** | **35.17** | **** |
219
- | **Recovery** | **100.00** | **** |
220
 
221
  #### HumanEval pass@1 scores
222
  | Metric | ibm-granite/granite-3.1-2b-instruct | neuralmagic-ent/granite-3.1-2b-instruct-quantized.w8a8 |
223
  |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
224
- | HumanEval Pass@1 | 53.40 | 0.549 |
225
 
 
214
  | Math-Hard (Exact-Match, 4-shot) | 8.66 | 8.04 |
215
  | GPQA (Acc-Norm, 0-shot) | 28.30 | 27.60 |
216
  | MUSR (Acc-Norm, 0-shot) | 35.12 | 34.58 |
217
+ | MMLU-Pro (Acc, 5-shot) | 26.87 | 26.89 |
218
+ | **Average Score** | **35.17** | **34.61** |
219
+ | **Recovery** | **100.00** | **98.40** |
220
 
221
  #### HumanEval pass@1 scores
222
  | Metric | ibm-granite/granite-3.1-2b-instruct | neuralmagic-ent/granite-3.1-2b-instruct-quantized.w8a8 |
223
  |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
224
+ | HumanEval Pass@1 | 53.40 | 54.9 |
225