Update README.md
Browse files
README.md
CHANGED
|
@@ -49,7 +49,7 @@ which are licensed under different terms:
|
|
| 49 |
|
| 50 |
| |**Solar-Open-100B**|**Nota MoE Quantization (Ours)**|**AutoRound**|
|
| 51 |
|--- | --- | --- | --- |
|
| 52 |
-
|PPL (WikiText-2)↓|6.06 |**
|
| 53 |
|MMLU-Pro↑ |73.91 |**62.53** |61.56 |
|
| 54 |
|GPQA-Diamond↑ |58.08 |**45.96** |42.42 |
|
| 55 |
|General Evaluation Benchmarks |75.77 |**73.94** |73.74 |
|
|
@@ -82,7 +82,7 @@ Step 2: Install Solar Open's optimized vLLM
|
|
| 82 |
pip install vllm==0.17.0
|
| 83 |
```
|
| 84 |
|
| 85 |
-
Step 3: Overwrite the two files (solar_open.py and registry.py) in the
|
| 86 |
|
| 87 |
Step 4: Start the vLLM server (For 1GPUs)
|
| 88 |
```bash
|
|
|
|
| 49 |
|
| 50 |
| |**Solar-Open-100B**|**Nota MoE Quantization (Ours)**|**AutoRound**|
|
| 51 |
|--- | --- | --- | --- |
|
| 52 |
+
|PPL (WikiText-2)↓|6.06 |**6.90** |7.22 |
|
| 53 |
|MMLU-Pro↑ |73.91 |**62.53** |61.56 |
|
| 54 |
|GPQA-Diamond↑ |58.08 |**45.96** |42.42 |
|
| 55 |
|General Evaluation Benchmarks |75.77 |**73.94** |73.74 |
|
|
|
|
| 82 |
pip install vllm==0.17.0
|
| 83 |
```
|
| 84 |
|
| 85 |
+
Step 3: Overwrite the two files (solar_open.py and registry.py) in the `patches` folder of the repository containing the model weights into the `vllm/model_executor/models` directory inside the folder where vLLM is installed (typically lib/python3.xx/site-packages).
|
| 86 |
|
| 87 |
Step 4: Start the vLLM server (For 1GPUs)
|
| 88 |
```bash
|