Update README.md
Browse files
README.md
CHANGED
|
@@ -68,7 +68,7 @@ This model was created with [llm-compressor](https://github.com/vllm-project/llm
|
|
| 68 |
|
| 69 |
|
| 70 |
```bash
|
| 71 |
-
python quantize.py --model_path ibm-granite/granite-3.1-8b-instruct --quant_path "output_dir/granite-3.1-8b-instruct-quantized.w4a16" --calib_size 1024 --dampening_frac 0.
|
| 72 |
```
|
| 73 |
|
| 74 |
|
|
@@ -192,18 +192,18 @@ evalplus.evaluate \
|
|
| 192 |
|
| 193 |
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 194 |
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|
| 195 |
-
| ARC-Challenge (Acc-Norm, 25-shot) | 66.81 | 66.
|
| 196 |
-
| GSM8K (Strict-Match, 5-shot) | 64.52 |
|
| 197 |
-
| HellaSwag (Acc-Norm, 10-shot) | 84.18 | 83.
|
| 198 |
-
| MMLU (Acc, 5-shot) | 65.52 |
|
| 199 |
-
| TruthfulQA (MC2, 0-shot) | 60.57 | 60.
|
| 200 |
-
| Winogrande (Acc, 5-shot) | 80.19 | 78.
|
| 201 |
-
| **Average Score** | **70.30** | **
|
| 202 |
-
| **Recovery** | **100.00** | **99.
|
| 203 |
|
| 204 |
#### HumanEval pass@1 scores
|
| 205 |
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 206 |
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|
| 207 |
-
| HumanEval Pass@1 | 71.00 | 70.
|
| 208 |
|
| 209 |
|
|
|
|
| 68 |
|
| 69 |
|
| 70 |
```bash
|
| 71 |
+
python quantize.py --model_path ibm-granite/granite-3.1-8b-instruct --quant_path "output_dir/granite-3.1-8b-instruct-quantized.w4a16" --calib_size 1024 --dampening_frac 0.1 --observer mse --actorder static
|
| 72 |
```
|
| 73 |
|
| 74 |
|
|
|
|
| 192 |
|
| 193 |
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 194 |
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|
| 195 |
+
| ARC-Challenge (Acc-Norm, 25-shot) | 66.81 | 66.81 |
|
| 196 |
+
| GSM8K (Strict-Match, 5-shot) | 64.52 | 65.66 |
|
| 197 |
+
| HellaSwag (Acc-Norm, 10-shot) | 84.18 | 83.62 |
|
| 198 |
+
| MMLU (Acc, 5-shot) | 65.52 | 64.25 |
|
| 199 |
+
| TruthfulQA (MC2, 0-shot) | 60.57 | 60.17 |
|
| 200 |
+
| Winogrande (Acc, 5-shot) | 80.19 | 78.37 |
|
| 201 |
+
| **Average Score** | **70.30** | **69.81** |
|
| 202 |
+
| **Recovery** | **100.00** | **99.31** |
|
| 203 |
|
| 204 |
#### HumanEval pass@1 scores
|
| 205 |
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 206 |
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|
| 207 |
+
| HumanEval Pass@1 | 71.00 | 70.50 |
|
| 208 |
|
| 209 |
|