OPEA
/

DeepSeek-R1-int4-AutoRound-awq-asym

4-bit precision

Model card Files Files and versions

cicdatopea commited on Mar 17, 2025

Commit

839af37

·

verified ·

1 Parent(s): fc57cb6

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -22,7 +22,7 @@ To serve using vLLM with 8x 80GB GPUs, use the following command:
 ```sh
 VLLM_WORKER_MULTIPROC_METHOD=spawn python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 12345 --max-model-len 65536 --max-num-batched-tokens 65536 --trust-remote-code --tensor-parallel-size 8 --gpu-memory-utilization 0.97 --dtype bfloat16 --served-model-name deepseek-reasoner --model OPEA/DeepSeek-R1-int4-asym-AutoRound-awq
 ```
-You can download the wheel [cognitivecomputations](https://huggingface.co/cognitivecomputations) built for PyTorch 2.6, Python 3.12 by clicking [here](https://huggingface.co/x2ray/wheels/resolve/main/vllm-0.7.3.dev187%2Bg0ff1a4df.d20220101.cu126-cp312-cp312-linux_x86_64.whl).
 ~~~python
 import requests

 ```sh
 VLLM_WORKER_MULTIPROC_METHOD=spawn python -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 12345 --max-model-len 65536 --max-num-batched-tokens 65536 --trust-remote-code --tensor-parallel-size 8 --gpu-memory-utilization 0.97 --dtype bfloat16 --served-model-name deepseek-reasoner --model OPEA/DeepSeek-R1-int4-asym-AutoRound-awq
 ```
+You can download the wheel built by [cognitivecomputations](https://huggingface.co/cognitivecomputations) for PyTorch 2.6 and Python 3.12 by clicking [here](https://huggingface.co/x2ray/wheels/resolve/main/vllm-0.7.3.dev187%2Bg0ff1a4df.d20220101.cu126-cp312-cp312-linux_x86_64.whl).
 ~~~python
 import requests