---
license: mit
library_name: transformers
---
# DeepSeek-V4-Flash-0731
Technical Report👁️
> **Note (this fork).** This is **not** a new or re-trained model. The weights are
> the official **DeepSeek-V4-Flash-0731**, byte-for-byte unchanged. The **only**
> modification is setting `num_experts_per_tok` from `6` to **`4`** in `config.json`
> (plus a one-line vLLM weight-loading shim to make it load). Everything else added
> here is **analysis**: a `top_k=4` vs `top_k=6` accuracy/speed comparison and the
> statistical evidence behind recommending `top_k=4`. See
> [Expert Routing: `top_k=4` vs `top_k=6`](#expert-routing-top_k4-vs-top_k6-recommended-top_k4).
## Introduction
**DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as [DeepSeek-V4-Flash-DSpark](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark), i.e. it comes with a speculative decoding module attached.
DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
| Benchmark | DeepSeek-V4-Flash-0731 | DeepSeek-V4-Flash (Preview) | DeepSeek-V4-Pro (Preview) | GLM-5.2 | Opus-4.8 |
| :--- | :---: | :---: | :---: | :---: | :---: |
| Terminal Bench 2.1 | 82.7 | 61.8 | 72.1 | 81.0 | 85.0 |
| NL2Repo | 54.2 | 39.4 | 38.5 | 48.9 | 69.7 |
| Cybergym | 76.7 | 38.7 | 52.7 | - | 83.1 |
| DeepSWE | 54.4 | 7.3 | 12.8 | 46.2 | 58.0 |
| Toolathlon-Verified | 70.3 | 49.7 | 55.9 | 59.9 | 76.2 |
| Agents' Last Exam | 25.2 | 15.8 | 16.5 | 23.8 | 25.7 |
| AutomationBench Public | 25.1 | 10.8 | 12.8 | 12.9 | 27.2 |
| DSBench-FullStack † | 68.7 | 37.0 | 41.8 | 61.8 | 71.6 |
| DSBench-Hard † | 59.6 | 25.8 | 31.1 | 54.5 | 71.7 |
Notes:
1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
## Chat Template
This release does not include a Jinja-format chat template. Instead, we provide a dedicated `encoding` folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the [`encoding`](encoding/README.md) folder for full documentation.
The `reasoning_effort` parameter now supports three levels — `low`, `high`, and `max` — which control how much deliberation the model spends before answering.
A brief example:
```python
from encoding_dsv4 import encode_messages, parse_message_from_completion_text
messages = [
{"role": "user", "content": "hello"},
{"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
{"role": "user", "content": "1+1=?"}
]
# messages -> string
prompt = encode_messages(messages, thinking_mode="thinking", reasoning_effort="max")
# string -> tokens
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-0731")
tokens = tokenizer.encode(prompt)
```
## How to Run with vLLM
DSpark speculative decoding is enabled with a single flag — add --speculative-config with method: dspark to your vLLM launch command:
`--speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'`
For example, the command below serves the model with vLLM on a single 4×GB300 node.
See the [vLLM recipe](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Flash?hardware=b300&features=tool_calling,reasoning) for detailed instructions and other hardware configurations.
```bash
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
--trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
--data-parallel-size 4 --enable-expert-parallel \
--moe-backend deep_gemm_mega_moe \
--attention-config '{"use_fp4_indexer_cache": true}' \
--speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'
```
## Expert Routing: `top_k=4` vs `top_k=6` (Recommended: **`top_k=4`**)
DeepSeek-V4-Flash-0731 ships with `num_experts_per_tok=6` (6 of 256 routed experts
activated per token, ~13B active params). We additionally evaluated
`num_experts_per_tok=4` (~11B active params) and **recommend `top_k=4` as the
default**: it is measurably faster at **statistically indistinguishable accuracy**.
### Why `top_k=4`
1. **~15% fewer activated parameters, for free.** Routing to 4 experts instead of
6 drops per-token active params from ~13B to ~11B. Shared experts and attention
are unchanged, so quality is preserved (see measurements below).
2. **Faster inference (~13–18%).** Fewer expert FFN computations plus less
gather/scatter and softmax overhead in the router. Measured end-to-end:
HumanEval wall-time ~15% lower, per-token generation ~13% faster.
3. **Power-of-2 dispatch alignment.** `6` is not a power of two; MoE dispatch,
warp scheduling, and memory alignment on the GPU are more efficient when the
expert count aligns to a power of two (`4`), improving tensor-core utilization
for the dispatch/combine shapes.
4. **No accuracy regression.** On our internal SWE-bench-Lite and HumanEval runs
the difference between `top_k=4` and `top_k=6` is within run-to-run noise
(details below).
### Single-GPU test environment (used for the numbers below)
The measurements were produced on a **single NVIDIA B300 (SXM6, ~275 GiB)** — no
data/expert parallelism and **DSpark speculative decoding disabled** (the official
multi-GPU launch in [How to Run with vLLM](#how-to-run-with-vllm) enables DSpark;
that is orthogonal to the routing comparison here). Exact launch command:
```bash
CUDA_HOME=$HOME/.local/lib/python3.11/site-packages/nvidia/cu13 \
VLLM_USE_FLASHINFER_SAMPLER=0 \
CUDA_VISIBLE_DEVICES=0 \
vllm serve /path/to/DeepSeek-V4-Flash-0731 \
--served-model-name dsv4 --port 18002 \
--trust-remote-code --kv-cache-dtype fp8 \
--max-model-len 32768 --gpu-memory-utilization 0.85 \
--reasoning-parser deepseek_v4 --tool-call-parser deepseek_v4 \
--enable-auto-tool-choice
```
Notes for single-GPU:
- **No `--speculative-config`** — DSpark is left off, so throughput numbers reflect
the base model. (DSpark is a decode-speed optimization and does not change which
tasks pass; it can be re-enabled independently.)
- `--max-model-len 32768` fits the KV cache comfortably in 275 GiB alongside the
fp8+MXFP4 weights; raise it if you have headroom.
- The `tid2eid` shim from [How to enable `top_k=4`](#how-to-enable-top_k4) must be
applied before serving with `num_experts_per_tok=4`.
- Weight load takes ~6–9 min (fp8/fp4 MoE autotuning on first start).
**Test harnesses:** HumanEval via `/v1/chat/completions` (code-fence extraction,
`temperature=0.1`); SWE-bench-Lite via the [mini-swe-agent](https://github.com/SWE-agent/mini-swe-agent)
minimal text-based agent (no Docker; each repo in an isolated `uv` venv), with the
official code-agent sampling `temperature=1.0, top_p=0.95`.
### Measured accuracy — the difference is within noise
All numbers below are from the single-B300 setup above on the native
fp8+MXFP4 weights.
**HumanEval** (pass@1, 164 problems, thinking mode, `temperature=0.1`):
| Config | Pass@1 | Note |
| :--- | :---: | :--- |
| `top_k=4` | **92.1% / 91.5% / 92.1%** (3 reps: 151 / 150 / 151) | run-to-run spread ±1 problem |
| `top_k=6` | 90.9% (149) | within the ±1-problem noise band |
**SWE-bench-Lite** (n=86 subset, mini-swe-agent harness, no Docker,
official code-agent sampling `temperature=1.0, top_p=0.95` unless noted):
| Config | Resolved | Rate | Sampling |
| :--- | :---: | :---: | :--- |
| `top_k=6` | 37/86 | 43.0% | default |
| `top_k=4` (rep 1) | 38/86 | 44.2% | default |
| `top_k=4` (rep 2) | 38/86 | 44.2% | default |
| `top_k=4` (official) | **39/86** | **45.3%** | `t=1.0, p=0.95` |
Two-proportion z-test `top_k=4` vs `top_k=6`: **z ≈ 0.15–0.31 (not significant)**.
Repeating the *same* `top_k=4` config flips ~14–17 of the 86 instances per pair of
runs (≈31% of the ever-solved union) purely from MoE-routing / fp8-kernel /
batching non-determinism. Aggregated over 4 runs: **23 instances always pass**
(stable core), **37 always fail**, and **26 are coin-flips**. In other words, the
per-task differences between `top_k=4` and `top_k=6` are the same magnitude as
`top_k=4` versus itself — i.e. **noise, not a capability gap**. `top_k=4` gets the
speed and parameter savings at no measurable accuracy cost.
> On knowledge-heavy multiple-choice benchmarks (e.g. MMLU-Pro) narrower routing
> can even help slightly; on code generation `top_k=6` may hold a fraction of a
> point. Both directions are inside the noise band on our runs — treat them as
> equivalent in quality.
### How to enable `top_k=4`
**Step 1 — set the config.** In `config.json`:
```json
"num_experts_per_tok": 4
```
**Step 2 — patch vLLM weight loading (required).** The checkpoint's `tid2eid`
tensor (the hash-based expert-routing lookup table) was trained at `top_k=6`, so
it has shape `[vocab_size, 6]`. With `num_experts_per_tok=4` the model allocates a
`[vocab_size, 4]` parameter, and loading fails with:
```
AssertionError: Attempted to load weight (torch.Size([129280, 6]))
into parameter (torch.Size([129280, 4]))
```
Fix it by slicing the checkpoint tensor to the first `top_k` columns during load.
In `vllm/models/deepseek_v4/nvidia/model.py`, inside `load_weights`, in the
final `else` branch just before the `weight_loader(param, loaded_weight)` call:
```python
param = params_dict[name]
# top_k override: checkpoint's tid2eid is [vocab, 6] (trained at top_k=6);
# slice to the config's top_k columns so it matches the allocated parameter.
if "tid2eid" in name and loaded_weight.shape != param.shape:
loaded_weight = loaded_weight[:, :param.shape[1]].contiguous()
weight_loader = getattr(param, "weight_loader", default_weight_loader)
weight_loader(param, loaded_weight)
```
The `[:, :param.shape[1]]` slice keeps the highest-priority expert columns and is
a no-op when the shapes already match (`top_k=6`), so the patch is safe to leave
in place for both configurations. No other weights change — total parameters,
routing method (`noaux_tc`), shared experts, and attention are all identical.
> Note: this is a weight-loading shim, not a re-training of the router table.
> The hash table's remaining 4 columns are the same top entries used at `top_k=6`,
> which is why accuracy is preserved.
## How to Run Locally
Please refer to the [inference](inference/README.md) folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos.
For local deployment, we recommend setting the sampling parameters to `temperature = 1.0`, with `top_p = 0.95` for agentic scenarios and `top_p = 1.0` otherwise. For the `high` and `max` reasoning effort levels, we recommend a maximum output length of **384K** tokens.
## License
This repository and the model weights are licensed under the [MIT License](LICENSE).
## Citation
```
@misc{deepseekai2026deepseekv4,
title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
author={DeepSeek-AI},
year={2026},
}
```
