RedHatAI
/

DeepSeek-R1-Distill-Qwen-1.5B-quantized.w8a8

Text Generation

text-generation-inference

8-bit precision

compressed-tensors

Model card Files Files and versions

nm-research commited on Feb 20, 2025

Commit

78f3866

·

verified ·

1 Parent(s): 5a39a4c

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -292,7 +292,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
 <summary>Benchmarking Command</summary>
 ```
-guidellm --model neuralmagic/DeepSeek-R1-Distill-Qwen-1.5B-quantized.w4a16 --target "http://localhost:8000/v1" --data-type emulated --data "prompt_tokens=<prompt_tokens>,generated_tokens=<generated_tokens>" --max seconds 360 --backend aiohttp_server
 ```
 </details>

 <summary>Benchmarking Command</summary>
 ```
+guidellm --model neuralmagic/DeepSeek-R1-Distill-Qwen-1.5B-quantized.w8a8 --target "http://localhost:8000/v1" --data-type emulated --data "prompt_tokens=<prompt_tokens>,generated_tokens=<generated_tokens>" --max seconds 360 --backend aiohttp_server
 ```
 </details>