typos
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ To run on vllm, follow vllm & DeepGemm install instructions at https://docs.vllm
|
|
| 16 |
|
| 17 |
Serve on 4 B200s:
|
| 18 |
```
|
| 19 |
-
vllm serve "
|
| 20 |
```
|
| 21 |
|
| 22 |
## Evals
|
|
@@ -24,9 +24,9 @@ vllm serve "bdellabe/DeepSeek-V3.2-NVFP4-FP8-BLOCK" --tensor-parallel-size 2 --d
|
|
| 24 |
### gsm8k
|
| 25 |
This checkpoint achieves over 99% recovery of the original `deepseek-ai/DeepSeek-V3.2` checkpoint's [0.9553 score on gsm8k](https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3_2.html#gsm8k):
|
| 26 |
```
|
| 27 |
-
lm_eval --model local-completions --model_args "model=
|
| 28 |
Requesting API: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1319/1319 [04:01<00:00, 5.46it/s]
|
| 29 |
-
local-completions ({'model': '
|
| 30 |
|Tasks|Version| Filter |n-shot| Metric | |Value| |Stderr|
|
| 31 |
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|
| 32 |
|gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.953|± |0.0058|
|
|
@@ -45,8 +45,8 @@ local-completions ({'model': '/mnt/data/brian-dellabetta/DeepSeek-V3.2-W4A16', '
|
|
| 45 |
### aime
|
| 46 |
This checkpoint achieves roughly 86% recovery of the original checkpoint's [0.933 score on aime25](https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3_2.html#aime25)
|
| 47 |
```
|
| 48 |
-
lm_eval --model local-chat-completions --model_args "model=
|
| 49 |
-
local-chat-completions ({'model': '
|
| 50 |
|Tasks |Version|Filter|n-shot| Metric | |Value| |Stderr|
|
| 51 |
|------|------:|------|-----:|-----------|---|----:|---|-----:|
|
| 52 |
|aime25| 0|none | 0|exact_match|↑ | 0.8|± |0.0743|
|
|
|
|
| 16 |
|
| 17 |
Serve on 4 B200s:
|
| 18 |
```
|
| 19 |
+
vllm serve "RedHatAI/DeepSeek-V3.2-NVFP4-FP8-BLOCK" --tensor-parallel-size 2 --data-parallel-size=2 --enable-expert-parallel --tokenizer-mode deepseek_v32 --reasoning-parser deepseek_v3
|
| 20 |
```
|
| 21 |
|
| 22 |
## Evals
|
|
|
|
| 24 |
### gsm8k
|
| 25 |
This checkpoint achieves over 99% recovery of the original `deepseek-ai/DeepSeek-V3.2` checkpoint's [0.9553 score on gsm8k](https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3_2.html#gsm8k):
|
| 26 |
```
|
| 27 |
+
lm_eval --model local-completions --model_args "model=RedHatAI/DeepSeek-V3.2-NVFP4-FP8-BLOCK,base_url=http://0.0.0.0:8000/v1/completions,max_length=8192,tokenized_requests=False,tokenizer_backend=None,num_concurrent=32" --tasks gsm8k --num_fewshot 5
|
| 28 |
Requesting API: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1319/1319 [04:01<00:00, 5.46it/s]
|
| 29 |
+
local-completions ({'model': 'RedHatAI/DeepSeek-V3.2-NVFP4-FP8-BLOCK', 'base_url': 'http://0.0.0.0:8000/v1/completions', 'max_length': 8192, 'tokenized_requests': False, 'tokenizer_backend': None, 'num_concurrent': 32}), gen_kwargs: ({}), limit: None, num_fewshot: 5, batch_size: 1
|
| 30 |
|Tasks|Version| Filter |n-shot| Metric | |Value| |Stderr|
|
| 31 |
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|
| 32 |
|gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.953|± |0.0058|
|
|
|
|
| 45 |
### aime
|
| 46 |
This checkpoint achieves roughly 86% recovery of the original checkpoint's [0.933 score on aime25](https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3_2.html#aime25)
|
| 47 |
```
|
| 48 |
+
lm_eval --model local-chat-completions --model_args "model=RedHatAI/DeepSeek-V3.2-NVFP4-FP8-BLOCK,base_url=http://0.0.0.0:8000/v1/chat/completions,tokenized_requests=False,tokenizer_backend=None,num_concurrent=20,timeout=5000,max_length=72768" --tasks aime25 --apply_chat_template --gen_kwargs '{"temperature":1.0,"max_gen_toks":65536,"top_p":0.95,"chat_template_kwargs":{"thinking":true}}' --log_samples --output_path "aime25_ds32"
|
| 49 |
+
local-chat-completions ({'model': 'RedHatAI/DeepSeek-V3.2-NVFP4-FP8-BLOCK', 'base_url': 'http://0.0.0.0:8000/v1/chat/completions', 'tokenized_requests': False, 'tokenizer_backend': None, 'num_concurrent': 20, 'timeout': 5000, 'max_length': 72768}), gen_kwargs: ({'temperature': 1.0, 'max_gen_toks': 65536, 'top_p': 0.95, 'chat_template_kwargs': {'thinking': True}}), limit: None, num_fewshot: None, batch_size: 1
|
| 50 |
|Tasks |Version|Filter|n-shot| Metric | |Value| |Stderr|
|
| 51 |
|------|------:|------|-----:|-----------|---|----:|---|-----:|
|
| 52 |
|aime25| 0|none | 0|exact_match|↑ | 0.8|± |0.0743|
|