| --- |
| license: mit |
| base_model: deepseek-ai/DeepSeek-V4-Flash |
| library_name: mlx |
| tags: |
| - mlx |
| - mtp |
| - speculative-decoding |
| pipeline_tag: text-generation |
| --- |
| |
| # ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16 |
|
|
| MTP drafter for speculative decoding with DeepSeek-V4-Flash on Apple Silicon (mlx-vlm), in **bf16** (3.4 GB). |
| This is a drafter, not a chat model — it is only useful as `--draft-model`. |
|
|
| ## Provenance (self-split from official weights) |
| - Source: [`deepseek-ai/DeepSeek-V4-Flash`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) (license: mit), |
| whose `mtp.*` tensors live in a dedicated shard (`model-00046-of-00046.safetensors`) |
| - Tool: `mlx-vlm 0.6.8` — |
| `python -m mlx_vlm.speculative.drafters.deepseek_v4_mtp.split --model deepseek-ai/DeepSeek-V4-Flash --output .` |
| - Result: 45 tensors — `decoder.*` (35), `hc_head` (3), `e_proj` / `h_proj` (2 each), `enorm`, `hnorm`, `norm` |
|
|
| ## Usage |
|
|
| Pair it with a DeepSeek-V4-Flash body and run the MTP round loop (`--draft-kind mtp`): |
|
|
| ```bash |
| mlx_vlm.server --model ToPo-ToPo/DeepSeek-V4-Flash-0731-mlx-4bit \ |
| --draft-model ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16 --draft-kind mtp |
| ``` |
|
|
| Speculative decoding is lossless, so the body's outputs are unchanged. |
|
|
| This drafter also works with the newer `-0731` body even though it is split from the earlier release — the two |
| share `hidden_size` 4096. The MTP module bundled inside `-0731` itself is a different architecture |
| (three modules built around `main_proj`) that mlx-vlm 0.6.8 does not implement. |
|
|
