Update README.md
Browse files
README.md
CHANGED
|
@@ -37,7 +37,7 @@ We introduce two innovative techniques: Gating Logit Normalization, which enhanc
|
|
| 37 |
Skywork-MoE demonstrates comparable or superior performance to models with more parameters or more activated parameters, such as Grok-1, DBRX, Mistral 8*22, and Deepseek-V2.
|
| 38 |
|
| 39 |
# News and Updates
|
| 40 |
-
* 2024.6.3 We release the **Skywork-MoE-
|
| 41 |
|
| 42 |
# Table of contents
|
| 43 |
|
|
@@ -49,22 +49,15 @@ Skywork-MoE demonstrates comparable or superior performance to models with more
|
|
| 49 |
- [🤝Contact Us and Citation](#Contact-Us-and-Citation)
|
| 50 |
|
| 51 |
|
| 52 |
-
# Download URL
|
| 53 |
-
|
| 54 |
-
| | HuggingFace Model | ModelScope Model | Wisemodel Model |
|
| 55 |
-
|:-------:|:-----------:|:-----------------------------:|:-----------------------------:|
|
| 56 |
-
| **Skywork-MoE-base** | 🤗 [Skywork-MoE-base](https://github.com/SkyworkAI/Skywork-MoE) | 🤖[Skywork-MoE-base](https://www.modelscope.cn/models/skywork/Skywork-MoE-base) | 👾[Skywork-MoE-base](https://wisemodel.cn/models/Skywork/Skywork-MoE-base) |
|
| 57 |
-
| **Skywork-MoE-Base-FP8** | 🤗 [Skywork-MoE-Base-FP8](https://github.com/SkyworkAI/Skywork-MoE) | 🤖 | 👾 |
|
| 58 |
-
|
| 59 |
# Benchmark Results
|
| 60 |
-
We evaluated Skywork-MoE-
|
| 61 |
<img src="misc/skywork_moe_base_evaluation.png" alt="Image" width="600" height="280">
|
| 62 |
|
| 63 |
# Demonstration of Hugging Face Model Inference
|
| 64 |
|
| 65 |
## Base Model Inference
|
| 66 |
|
| 67 |
-
We can perform inference for the Skywork-MoE-
|
| 68 |
|
| 69 |
```python
|
| 70 |
|
|
@@ -100,35 +93,23 @@ comming soon...
|
|
| 100 |
|
| 101 |
## Quickstart with vLLM
|
| 102 |
|
| 103 |
-
We provide a method to quickly deploy the Skywork-
|
| 104 |
-
|
| 105 |
-
Under fp8 precision you can run Skywork-Moe-base with just only 8*4090.
|
| 106 |
|
| 107 |
You can get the source code in [`vllm`](https://github.com/SkyworkAI/vllm)
|
| 108 |
|
| 109 |
-
You can get the fp8 model in [`Skywork-MoE-Base-FP8`](https://huggingface.co/Skywork/Skywork-MoE-Base-FP8)
|
| 110 |
|
| 111 |
### Based on local environment
|
| 112 |
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
``` shell
|
| 116 |
-
# for cuda12.1
|
| 117 |
-
pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121
|
| 118 |
-
# for cuda12.4
|
| 119 |
-
pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu124
|
| 120 |
-
```
|
| 121 |
-
|
| 122 |
-
Some other dependencies also need to be installed:
|
| 123 |
|
| 124 |
```shell
|
| 125 |
pip3 install xformers vllm-flash-attn
|
| 126 |
```
|
| 127 |
|
| 128 |
-
Then clone the [`vllm`](https://github.com/SkyworkAI/vllm) provided by skywork
|
| 129 |
|
| 130 |
``` shell
|
| 131 |
-
git clone https://github.com/SkyworkAI/vllm.git
|
| 132 |
cd vllm
|
| 133 |
```
|
| 134 |
|
|
@@ -138,7 +119,7 @@ Then compile and install vllm:
|
|
| 138 |
MAX_JOBS=8 python3 setup.py install
|
| 139 |
```
|
| 140 |
|
| 141 |
-
###
|
| 142 |
|
| 143 |
You can use the docker image provided by skywork to run vllm directly:
|
| 144 |
|
|
@@ -149,7 +130,7 @@ docker pull registry.cn-wulanchabu.aliyuncs.com/triple-mu/skywork-moe-vllm:v1
|
|
| 149 |
Then start the container and set the model path and working directory.
|
| 150 |
|
| 151 |
```shell
|
| 152 |
-
model_path="Skywork/Skywork-MoE-Base
|
| 153 |
workspace=${PWD}
|
| 154 |
|
| 155 |
docker run \
|
|
@@ -162,19 +143,19 @@ docker run \
|
|
| 162 |
--privileged=true \
|
| 163 |
--ulimit stack=67108864 \
|
| 164 |
--ipc=host \
|
| 165 |
-
-v ${model_path}:/Skywork-MoE-Base
|
| 166 |
-v ${workspace}:/workspace \
|
| 167 |
registry.cn-wulanchabu.aliyuncs.com/triple-mu/skywork-moe-vllm:v1
|
| 168 |
```
|
| 169 |
|
| 170 |
-
Now, you can run the Skywork
|
| 171 |
|
| 172 |
### Text Completion
|
| 173 |
|
| 174 |
``` python
|
| 175 |
from vllm import LLM, SamplingParams
|
| 176 |
|
| 177 |
-
model_path = '/
|
| 178 |
prompts = [
|
| 179 |
"The president of the United States is",
|
| 180 |
"The capital of France is",
|
|
|
|
| 37 |
Skywork-MoE demonstrates comparable or superior performance to models with more parameters or more activated parameters, such as Grok-1, DBRX, Mistral 8*22, and Deepseek-V2.
|
| 38 |
|
| 39 |
# News and Updates
|
| 40 |
+
* 2024.6.3 We release the **Skywork-MoE-Base** model.
|
| 41 |
|
| 42 |
# Table of contents
|
| 43 |
|
|
|
|
| 49 |
- [🤝Contact Us and Citation](#Contact-Us-and-Citation)
|
| 50 |
|
| 51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
# Benchmark Results
|
| 53 |
+
We evaluated Skywork-MoE-Base model on various popular benchmarks, including C-Eval, MMLU, CMMLU, GSM8K, MATH and HumanEval.
|
| 54 |
<img src="misc/skywork_moe_base_evaluation.png" alt="Image" width="600" height="280">
|
| 55 |
|
| 56 |
# Demonstration of Hugging Face Model Inference
|
| 57 |
|
| 58 |
## Base Model Inference
|
| 59 |
|
| 60 |
+
We can perform inference for the Skywork-MoE-Base (16x13B size) model using HuggingFace on 8xA100/A800 or higher GPU hardware configurations.
|
| 61 |
|
| 62 |
```python
|
| 63 |
|
|
|
|
| 93 |
|
| 94 |
## Quickstart with vLLM
|
| 95 |
|
| 96 |
+
We provide a method to quickly deploy the Skywork-MoE-Base model based on vllm.
|
|
|
|
|
|
|
| 97 |
|
| 98 |
You can get the source code in [`vllm`](https://github.com/SkyworkAI/vllm)
|
| 99 |
|
|
|
|
| 100 |
|
| 101 |
### Based on local environment
|
| 102 |
|
| 103 |
+
Some dependencies need to be installed:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
```shell
|
| 106 |
pip3 install xformers vllm-flash-attn
|
| 107 |
```
|
| 108 |
|
| 109 |
+
Then clone the [`vllm`](https://github.com/SkyworkAI/vllm) provided by skywork:
|
| 110 |
|
| 111 |
``` shell
|
| 112 |
+
git clone https://github.com/SkyworkAI/vllm.git
|
| 113 |
cd vllm
|
| 114 |
```
|
| 115 |
|
|
|
|
| 119 |
MAX_JOBS=8 python3 setup.py install
|
| 120 |
```
|
| 121 |
|
| 122 |
+
### Based on docker
|
| 123 |
|
| 124 |
You can use the docker image provided by skywork to run vllm directly:
|
| 125 |
|
|
|
|
| 130 |
Then start the container and set the model path and working directory.
|
| 131 |
|
| 132 |
```shell
|
| 133 |
+
model_path="Skywork/Skywork-MoE-Base"
|
| 134 |
workspace=${PWD}
|
| 135 |
|
| 136 |
docker run \
|
|
|
|
| 143 |
--privileged=true \
|
| 144 |
--ulimit stack=67108864 \
|
| 145 |
--ipc=host \
|
| 146 |
+
-v ${model_path}:/Skywork-MoE-Base \
|
| 147 |
-v ${workspace}:/workspace \
|
| 148 |
registry.cn-wulanchabu.aliyuncs.com/triple-mu/skywork-moe-vllm:v1
|
| 149 |
```
|
| 150 |
|
| 151 |
+
Now, you can run the Skywork-MoE-Base model for fun!
|
| 152 |
|
| 153 |
### Text Completion
|
| 154 |
|
| 155 |
``` python
|
| 156 |
from vllm import LLM, SamplingParams
|
| 157 |
|
| 158 |
+
model_path = 'Skywork/Skywork-MoE-Base'
|
| 159 |
prompts = [
|
| 160 |
"The president of the United States is",
|
| 161 |
"The capital of France is",
|