RedHatAI
/

Mistral-Small-24B-Instruct-2501-quantized.w4a16

Text Generation

text-generation-inference

compressed-tensors

Model card Files Files and versions

Eldar Kurtic commited on Jan 31, 2025

Commit

eeeeb77

·

1 Parent(s): 534f197

fix typo

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -210,7 +210,7 @@ lm_eval \
 #### OpenLLM Leaderboard V1 evaluation scores
-| Metric                                   | mistralai/Mistral-Small-24B-Instruct-2501             | nm-testing/Mistral-Small-24B-Instruct-2501-quantized.w4a16 |
 |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
 | ARC-Challenge (Acc-Norm, 25-shot)       |  72.18                            |    71.16                                    |
 | GSM8K (Strict-Match, 5-shot)            |  90.14                            |    89.69                                     |
@@ -223,7 +223,7 @@ lm_eval \
 #### OpenLLM Leaderboard V2 evaluation scores
-| Metric                                                   | mistralai/Mistral-Small-24B-Instruct-2501             | nm-testing/Mistral-Small-24B-Instruct-2501-quantized.w4a16 |
 |---------------------------------------------------------|:---------------------------------:|:-------------------------------------------:|
 | IFEval (Inst-and-Prompt Level Strict Acc, 0-shot)       |     73.27                        |    74.37                                    |
 | BBH (Acc-Norm, 3-shot)                                  |     45.18                        |    45.15                                       |

 #### OpenLLM Leaderboard V1 evaluation scores
+| Metric                                   | mistralai/Mistral-Small-24B-Instruct-2501             | neuralmagic/Mistral-Small-24B-Instruct-2501-quantized.w4a16 |
 |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
 | ARC-Challenge (Acc-Norm, 25-shot)       |  72.18                            |    71.16                                    |
 | GSM8K (Strict-Match, 5-shot)            |  90.14                            |    89.69                                     |
 #### OpenLLM Leaderboard V2 evaluation scores
+| Metric                                                   | mistralai/Mistral-Small-24B-Instruct-2501             | neuralmagic/Mistral-Small-24B-Instruct-2501-quantized.w4a16 |
 |---------------------------------------------------------|:---------------------------------:|:-------------------------------------------:|
 | IFEval (Inst-and-Prompt Level Strict Acc, 0-shot)       |     73.27                        |    74.37                                    |
 | BBH (Acc-Norm, 3-shot)                                  |     45.18                        |    45.15                                       |