Hugging Face's logo

RedHatAI
/
Mistral-Small-24B-Instruct-2501-quantized.w8a8

Text Generation
Safetensors
mistral
mistral-small
quantized
W8A8
vllm
conversational
text-generation-inference
compressed-tensors
8-bit precision
Model card Files
xet
Community
alexmarques commited on
Commit
4634361
·
verified ·
1 Parent(s): 6ea8427

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -8
README.md CHANGED
@@ -211,11 +211,11 @@ lm_eval \
211
 
212
  | Metric | mistralai/Mistral-Small-24B-Instruct-2501 | nm-testing/Mistral-Small-24B-Instruct-2501-quantized.w8a8 |
213
  |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
214
- | ARC-Challenge (Acc-Norm, 25-shot) | 72.18 | |
215
- | GSM8K (Strict-Match, 5-shot) | 90.14 | |
216
- | HellaSwag (Acc-Norm, 10-shot) | 85.05 | |
217
- | MMLU (Acc, 5-shot) | 80.69 | |
218
- | TruthfulQA (MC2, 0-shot) | 65.55 | |
219
- | Winogrande (Acc, 5-shot) | 83.11 | |
220
- | **Average Score** | **79.45** | **** |
221
- | **Recovery (%)** | **100.00** | **** |
 
211
 
212
  | Metric | mistralai/Mistral-Small-24B-Instruct-2501 | nm-testing/Mistral-Small-24B-Instruct-2501-quantized.w8a8 |
213
  |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
214
+ | ARC-Challenge (Acc-Norm, 25-shot) | 72.18 | 68.86 |
215
+ | GSM8K (Strict-Match, 5-shot) | 90.14 | 90.00 |
216
+ | HellaSwag (Acc-Norm, 10-shot) | 85.05 | 85.06 |
217
+ | MMLU (Acc, 5-shot) | 80.69 | 80.25 |
218
+ | TruthfulQA (MC2, 0-shot) | 65.55 | 65.69 |
219
+ | Winogrande (Acc, 5-shot) | 83.11 | 81.69 |
220
+ | **Average Score** | **79.45** | **78.59** |
221
+ | **Recovery (%)** | **100.00** | **98.92** |