YummyYum commited on
Commit
e89d4a5
·
verified ·
1 Parent(s): 69575f3

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -7
README.md CHANGED
@@ -68,19 +68,19 @@ We provide dequantized model weights in bfloat16 to run DeepSeek-R1 on NVIDIA GP
68
 
69
  ```bash
70
  pip install modelscope
71
- modelscope download --model <Model Name> --local_dir <Cache Path>
72
  ```
73
 
74
  ### Download the FlagOS image
75
 
76
  ```bash
77
- docker pull <IMAGE>
78
  ```
79
 
80
  ### Start the inference service
81
 
82
  ```bash
83
- docker run -itd --name flagrelease_nv --privileged --gpus all --net=host --ipc=host --device=/dev/infiniband --shm-size 512g --ulimit memlock=-1 -v <CKPT_PATH>:<CKPT_PATH> flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:deepseek-flagos-nvidia /bin/bash
84
 
85
  docker exec -it flagrelease_nv /bin/bash
86
 
@@ -95,6 +95,7 @@ cd FlagGems
95
  pip install .
96
  cd ../
97
  ```
 
98
  ### Modify the configuration
99
 
100
  ```bash
@@ -140,7 +141,7 @@ cd FlagScale/examples/deepseek_r1/conf/serve
140
  - serve_id: vllm_model
141
  engine: vllm
142
  engine_args:
143
- model: /models/deepseek_r1 # path of weight of deepseek r1
144
  tensor_parallel_size: 8
145
  pipeline_parallel_size: 4
146
  gpu_memory_utilization: 0.9
@@ -161,7 +162,7 @@ pip install .
161
  ### Serve
162
 
163
  ```
164
- flagscale serve <Model>
165
  ```
166
 
167
  #
@@ -180,8 +181,8 @@ We warmly welcome global developers to join us:
180
  Scan the QR code below to add our WeChat group
181
  send "FlagRelease"
182
 
183
- ![WeChat](https://cdn-uploads.huggingface.co/production/uploads/673326280dbcb3477ecc2af6/aETN9Zswqts2P9YLrizrz.png)
184
 
185
  # License
186
 
187
- This project and related model weights are licensed under the MIT License.
 
68
 
69
  ```bash
70
  pip install modelscope
71
+ modelscope download --model deepseek-ai/DeepSeek-R1 --local_dir /nfs/DeepSeek-R1
72
  ```
73
 
74
  ### Download the FlagOS image
75
 
76
  ```bash
77
+ docker pull flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:deepseek-flagos-nvidia
78
  ```
79
 
80
  ### Start the inference service
81
 
82
  ```bash
83
+ docker run -itd --name flagrelease_nv --privileged --gpus all --net=host --ipc=host --device=/dev/infiniband --shm-size 512g --ulimit memlock=-1 -v /nfs:/nfs flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:deepseek-flagos-nvidia /bin/bash
84
 
85
  docker exec -it flagrelease_nv /bin/bash
86
 
 
95
  pip install .
96
  cd ../
97
  ```
98
+
99
  ### Modify the configuration
100
 
101
  ```bash
 
141
  - serve_id: vllm_model
142
  engine: vllm
143
  engine_args:
144
+ model: /nfs/deepseek_r1 # path of weight of deepseek r1
145
  tensor_parallel_size: 8
146
  pipeline_parallel_size: 4
147
  gpu_memory_utilization: 0.9
 
162
  ### Serve
163
 
164
  ```
165
+ flagscale serve deepseek_r1
166
  ```
167
 
168
  #
 
181
  Scan the QR code below to add our WeChat group
182
  send "FlagRelease"
183
 
184
+ ![WeChat](image/group.png)
185
 
186
  # License
187
 
188
+ The weights of this model are based on deepseek-ai/DeepSeek-R1 and are open-sourced under the Apache 2.0 License: https://www.apache.org/licenses/LICENSE-2.0.txt.