README.md

metadata

license: mit
base_model: deepseek-ai/DeepSeek-V4-Flash
library_name: mlx
tags:
  - mlx
  - mtp
  - speculative-decoding
pipeline_tag: text-generation

ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16

MTP drafter for speculative decoding with DeepSeek-V4-Flash on Apple Silicon (mlx-vlm), in bf16 (3.4 GB). This is a drafter, not a chat model — it is only useful as --draft-model.

Provenance (self-split from official weights)

• Source: deepseek-ai/DeepSeek-V4-Flash (license: mit), whose mtp.* tensors live in a dedicated shard (model-00046-of-00046.safetensors)
• Tool: mlx-vlm 0.6.8 — python -m mlx_vlm.speculative.drafters.deepseek_v4_mtp.split --model deepseek-ai/DeepSeek-V4-Flash --output .
• Result: 45 tensors — decoder.* (35), hc_head (3), e_proj / h_proj (2 each), enorm, hnorm, norm

Usage

Pair it with a DeepSeek-V4-Flash body and run the MTP round loop (--draft-kind mtp):

mlx_vlm.server --model ToPo-ToPo/DeepSeek-V4-Flash-0731-mlx-4bit \
               --draft-model ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16 --draft-kind mtp

Speculative decoding is lossless, so the body's outputs are unchanged.

This drafter also works with the newer -0731 body even though it is split from the earlier release — the two share hidden_size 4096. The MTP module bundled inside -0731 itself is a different architecture (three modules built around main_proj) that mlx-vlm 0.6.8 does not implement.