Update README.md

README.md

@@ -43,7 +43,7 @@ from transformers import AutoTokenizer
 from vllm import LLM, SamplingParams
 
 max_model_len, tp_size = 4096, 1
-model_name = "neuralmagic-ent/granite-3.1-2b-base-quantized.w4a16"
+model_name = "neuralmagic/granite-3.1-2b-base-quantized.w4a16"
 tokenizer = AutoTokenizer.from_pretrained(model_name)
 llm = LLM(model=model_name, tensor_parallel_size=tp_size, max_model_len=max_model_len, trust_remote_code=True)
 sampling_params = SamplingParams(temperature=0.3, max_tokens=256, stop_token_ids=[tokenizer.eos_token_id])
@@ -66,6 +66,8 @@ vLLM also supports OpenAI-compatible serving. See the [documentation](https://do
 
 This model was created with [llm-compressor](https://github.com/vllm-project/llm-compressor) by running the code snippet below. 
 
+<details>
+<summary>Model Creation Code</summary>
 
 ```bash
 python quantize.py --model_path ibm-granite/granite-3.1-2b-base --quant_path "output_dir/granite-3.1-2b-base-quantized.w4a16" --calib_size 1024 --dampening_frac 0.01 --observer mse
@@ -144,16 +146,20 @@ oneshot(
 model.save_pretrained(quant_path, save_compressed=True)
 tokenizer.save_pretrained(quant_path)
 ```
+</details>
 
 ## Evaluation
 
-The model was evaluated on OpenLLM Leaderboard [V1](https://huggingface.co/spaces/open-llm-leaderboard-old/open_llm_leaderboard) and on [HumanEval](https://github.com/neuralmagic/evalplus), using the following commands:
+The model was evaluated on OpenLLM Leaderboard [V1](https://huggingface.co/spaces/open-llm-leaderboard-old/open_llm_leaderboard), OpenLLM Leaderboard [V2](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/) and on [HumanEval](https://github.com/neuralmagic/evalplus), using the following commands:
+
+<details>
+<summary>Evaluation Commands</summary>
 
 OpenLLM Leaderboard V1:
 ```
 lm_eval \
   --model vllm \
-  --model_args pretrained="neuralmagic-ent/granite-3.1-2b-base-quantized.w4a16",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,gpu_memory_utilization=0.8,enable_chunked_prefill=True,trust_remote_code=True \
+  --model_args pretrained="neuralmagic/granite-3.1-2b-base-quantized.w4a16",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,gpu_memory_utilization=0.8,enable_chunked_prefill=True,trust_remote_code=True \
   --tasks openllm \
   --write_out \
   --batch_size auto \
@@ -165,7 +171,7 @@ lm_eval \
 ##### Generation
 ```
 python3 codegen/generate.py \
-  --model neuralmagic-ent/granite-3.1-2b-base-quantized.w4a16 \
+  --model neuralmagic/granite-3.1-2b-base-quantized.w4a16 \
   --bs 16 \
   --temperature 0.2 \
   --n_samples 50 \
@@ -175,32 +181,79 @@ python3 codegen/generate.py \
 ##### Sanitization
 ```
 python3 evalplus/sanitize.py \
-  humaneval/neuralmagic-ent--granite-3.1-2b-base-quantized.w4a16_vllm_temp_0.2
+  humaneval/neuralmagic--granite-3.1-2b-base-quantized.w4a16_vllm_temp_0.2
 ```
 ##### Evaluation
 ```
 evalplus.evaluate \
   --dataset humaneval \
-  --samples humaneval/neuralmagic-ent--granite-3.1-2b-base-quantized.w4a16_vllm_temp_0.2-sanitized
+  --samples humaneval/neuralmagic--granite-3.1-2b-base-quantized.w4a16_vllm_temp_0.2-sanitized
 ```
+</details>
 
 ### Accuracy
 
-#### OpenLLM Leaderboard V1 evaluation scores
-
-| Metric                                  | ibm-granite/granite-3.1-2b-base             | neuralmagic-ent/granite-3.1-2b-base-quantized.w4a16 |
-|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
-| ARC-Challenge (Acc-Norm, 25-shot)       | 53.75                             | 51.96                                            |
-| GSM8K (Strict-Match, 5-shot)            | 47.84                             | 42.53                                             |
-| HellaSwag (Acc-Norm, 10-shot)           | 77.94                             | 75.38                                            |
-| MMLU (Acc, 5-shot)                      | 52.88                             | 51.09                                             |
-| TruthfulQA (MC2, 0-shot)                | 39.04                             | 41.35                                            |
-| Winogrande (Acc, 5-shot)                | 74.43                             | 74.27                                            |
-| **Average Score**                       | **57.65**                         | **56.10**                                           |
-| **Recovery**                            | **100.00**                        | **97.31**                                            |
-
-#### HumanEval pass@1 scores
+<table>
+  <thead>
+    <tr>
+      <th>Category</th>
+      <th>Metric</th>
+      <th>ibm-granite/granite-3.1-2b-base</th>
+      <th>neuralmagic/granite-3.1-2b-base-quantized.w4a16</th>
+      <th>Recovery (%)</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td rowspan="7"><b>OpenLLM Leaderboard V1</b></td>
+      <td>ARC-Challenge (Acc-Norm, 25-shot)</td>
+      <td>53.75</td>
+      <td>51.96</td>
+      <td>96.67</td>
+    </tr>
+    <tr>
+      <td>GSM8K (Strict-Match, 5-shot)</td>
+      <td>47.84</td>
+      <td>42.53</td>
+      <td>88.89</td>
+    </tr>
+    <tr>
+      <td>HellaSwag (Acc-Norm, 10-shot)</td>
+      <td>77.94</td>
+      <td>75.38</td>
+      <td>96.71</td>
+    </tr>
+    <tr>
+      <td>MMLU (Acc, 5-shot)</td>
+      <td>52.88</td>
+      <td>51.09</td>
+      <td>96.61</td>
+    </tr>
+    <tr>
+      <td>TruthfulQA (MC2, 0-shot)</td>
+      <td>39.04</td>
+      <td>41.35</td>
+      <td>105.93</td>
+    </tr>
+    <tr>
+      <td>Winogrande (Acc, 5-shot)</td>
+      <td>74.43</td>
+      <td>74.27</td>
+      <td>99.78</td>
+    </tr>
+    <tr>
+      <td><b>Average Score</b></td>
+      <td><b>57.65</b></td>
+      <td><b>56.10</b></td>
+      <td><b>97.31</b></td>
+    </tr>
+    <tr>
+      <td rowspan="2"><b>HumanEval</b></td>
+      <td>HumanEval Pass@1</td>
+      <td>30.00</td>
+      <td>29.80</td>
+      <td><b>99.33</b></td>
+    </tr>
+  </tbody>
+</table>
 
-| Metric                                  | ibm-granite/granite-3.1-2b-base             | neuralmagic-ent/granite-3.1-2b-base-quantized.w4a16 |
-|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
-| HumanEval Pass@1                        | 30.00                             | 0.298                                            |