---
license: mit
base_model: deepseek-ai/DeepSeek-V4-Flash
library_name: mlx
tags:
- mlx
- mtp
- speculative-decoding
pipeline_tag: text-generation
---

# ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16

MTP drafter for speculative decoding with DeepSeek-V4-Flash on Apple Silicon (mlx-vlm), in **bf16** (3.4 GB).
This is a drafter, not a chat model — it is only useful as `--draft-model`.

## Provenance (self-split from official weights)
- Source: [`deepseek-ai/DeepSeek-V4-Flash`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) (license: mit),
  whose `mtp.*` tensors live in a dedicated shard (`model-00046-of-00046.safetensors`)
- Tool: `mlx-vlm 0.6.8` —
  `python -m mlx_vlm.speculative.drafters.deepseek_v4_mtp.split --model deepseek-ai/DeepSeek-V4-Flash --output .`
- Result: 45 tensors — `decoder.*` (35), `hc_head` (3), `e_proj` / `h_proj` (2 each), `enorm`, `hnorm`, `norm`

## Usage

Pair it with a DeepSeek-V4-Flash body and run the MTP round loop (`--draft-kind mtp`):

```bash
mlx_vlm.server --model ToPo-ToPo/DeepSeek-V4-Flash-0731-mlx-4bit \
               --draft-model ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16 --draft-kind mtp
```

Speculative decoding is lossless, so the body's outputs are unchanged.

This drafter also works with the newer `-0731` body even though it is split from the earlier release — the two
share `hidden_size` 4096. The MTP module bundled inside `-0731` itself is a different architecture
(three modules built around `main_proj`) that mlx-vlm 0.6.8 does not implement.