Upload README.md
Browse files
README.md
CHANGED
|
@@ -68,19 +68,19 @@ We provide dequantized model weights in bfloat16 to run DeepSeek-R1 on NVIDIA GP
|
|
| 68 |
|
| 69 |
```bash
|
| 70 |
pip install modelscope
|
| 71 |
-
modelscope download --model
|
| 72 |
```
|
| 73 |
|
| 74 |
### Download the FlagOS image
|
| 75 |
|
| 76 |
```bash
|
| 77 |
-
docker pull
|
| 78 |
```
|
| 79 |
|
| 80 |
### Start the inference service
|
| 81 |
|
| 82 |
```bash
|
| 83 |
-
docker run -itd --name flagrelease_nv --privileged --gpus all --net=host --ipc=host --device=/dev/infiniband --shm-size 512g --ulimit memlock=-1 -v
|
| 84 |
|
| 85 |
docker exec -it flagrelease_nv /bin/bash
|
| 86 |
|
|
@@ -95,6 +95,7 @@ cd FlagGems
|
|
| 95 |
pip install .
|
| 96 |
cd ../
|
| 97 |
```
|
|
|
|
| 98 |
### Modify the configuration
|
| 99 |
|
| 100 |
```bash
|
|
@@ -140,7 +141,7 @@ cd FlagScale/examples/deepseek_r1/conf/serve
|
|
| 140 |
- serve_id: vllm_model
|
| 141 |
engine: vllm
|
| 142 |
engine_args:
|
| 143 |
-
model: /
|
| 144 |
tensor_parallel_size: 8
|
| 145 |
pipeline_parallel_size: 4
|
| 146 |
gpu_memory_utilization: 0.9
|
|
@@ -161,7 +162,7 @@ pip install .
|
|
| 161 |
### Serve
|
| 162 |
|
| 163 |
```
|
| 164 |
-
flagscale serve
|
| 165 |
```
|
| 166 |
|
| 167 |
#
|
|
@@ -180,8 +181,8 @@ We warmly welcome global developers to join us:
|
|
| 180 |
Scan the QR code below to add our WeChat group
|
| 181 |
send "FlagRelease"
|
| 182 |
|
| 183 |
-

|
| 185 |
|
| 186 |
# License
|
| 187 |
|
| 188 |
+
The weights of this model are based on deepseek-ai/DeepSeek-R1 and are open-sourced under the Apache 2.0 License: https://www.apache.org/licenses/LICENSE-2.0.txt.
|