TheStageAI
/

Elastic-Qwen2.5-7B-Instruct

Text Generation

Model card Files Files and versions

Update README.md

#9

by hinairo - opened May 28

base: refs/heads/main

←

from: refs/pr/9

Discussion Files changed

Files changed (1) hide show

README.md +6 -6

README.md CHANGED Viewed

@@ -192,7 +192,7 @@ We have used the `lm_eval` library to validate the models. For each model size (
 We measured TPS (tokens per second) for each model size using 100 input tokens and 300 output tokens.
-![Latency Benchmarking](https://cdn.thestage.ai/production/cms_file_upload/1773422585-3065316c-5c07-4430-befb-61daac95f712/Elastic_Qwen2.5_7B_Instruct_latency.png)
 ### Latency Benchmark Results
@@ -202,11 +202,11 @@ Tokens per second for different model sizes on various GPUs.
 | **GPU/Model Size**| **S**| **M**| **L**| **XL**| **Original**| **W8A8_int8** |
  | ---  | ---  | ---  | ---  | ---  | ---  | ---  |
-| **H100** | 184 | 177 | 157 | 138 | 62 | 201 |
-| **L40s** | 72 | 67 | 57 | 48 | 42 | 78 |
-| **B200** | 239 | 232 | 216 | 199 | 114 | N/A |
-| **GeForce RTX 5090** | 141 | N/A | N/A | N/A | 66 | N/A |
-| **GeForce RTX 4090** | 95 | N/A | N/A | N/A | 45 | N/A |
 ## Benchmarking Methodology

 We measured TPS (tokens per second) for each model size using 100 input tokens and 300 output tokens.
+![Latency Benchmarking](https://cdn.thestage.ai/production/cms_file_upload/1779128526-dac1999d-d2a4-45e7-ba9d-8a7c498549c7/Elastic_Qwen2.5_7B_Instruct_latency.png)
 ### Latency Benchmark Results
 | **GPU/Model Size**| **S**| **M**| **L**| **XL**| **Original**| **W8A8_int8** |
  | ---  | ---  | ---  | ---  | ---  | ---  | ---  |
+| **H100** | 202 | 191 | 168 | 147 | 62 | 201 |
+| **L40s** | 74 | 69 | 58 | 49 | 42 | 78 |
+| **B200** | 274 | 272 | 269 | 248 | 114 | N/A |
+| **GeForce RTX 5090** | 150 | N/A | N/A | N/A | 66 | N/A |
+| **GeForce RTX 4090** | 96 | N/A | N/A | N/A | 45 | N/A |
 ## Benchmarking Methodology