README.md

---
license: mit
base_model: 0xSero/DeepSeek-V4-Flash-0731-REAP
tags:
  - speculative-decoding
  - draft-head
  - dspark
  - mtp
  - jetson
  - edge-inference
library_name: safetensors
pipeline_tag: text-generation
---

# DeepSeek-V4-Flash-0731-REAP + DSpark MTP draft head — complete checkpoint

A **drop-in replacement** for [`0xSero/DeepSeek-V4-Flash-0731-REAP`](https://huggingface.co/0xSero/DeepSeek-V4-Flash-0731-REAP)
with a fine-tuned DSpark MTP draft head merged in. Download it and you have the whole model: no
patching step, no separate head file to graft on.

**Every emitted token is bit-identical to base autoregressive decode.** The speed-up is lossless by
construction — speculation changes how tokens are *produced*, never which tokens they are — and that
invariant is checked on every run by a gate that compares the speculative path against plain AR.

## What is actually different from the base checkpoint

Three files. Shards **46, 47 and 48** carry the MTP draft-head tensors; shards 1–45, the tokenizer,
and the config are byte-identical to the base. If you already have the base checkpoint, the only new
weights here are ~6.7 GB of head shards.

| | |
|---|---|
| provenance | `deepseek-ai/DeepSeek-V4-Flash-0731` @ `9e165c30` → REAP prune (256 → **160** experts/layer) → this head fine-tune |
| head trained on | 1,472 self-generated sequences, deficit-weighted, `a_ce` 0.1 / `a_tv` 0.9 |
| draft block width | **5** (the checkpoint's own `dspark_block_size`) |
| quantisation | native MXFP4 experts, FP8 attention — unchanged from base |

## Measured speculative decode

Measured on a **Jetson AGX Thor (`sm_110a`, 122.8 GiB unified)** with a from-scratch pure-CUDA
inference server. Every figure is from a logged run, not an estimate.

| | measured |
|---|---|
| suite mean `tau` | **3.8413** of a 5-wide draft (77 % of the ceiling) |
| suite mean throughput | **28.38 tok/s** |
| base AR decode | 14.61 tok/s → speculation is **1.94×** |
| vs the stock shipped head | **22.66 → 28.38 tok/s, +25.3 %** |
| on copy-heavy agentic prompts | up to **34.97 tok/s, 2.51×** |

### Acceptance by workload, and where this head gives ground

`tau` is tokens committed per target forward. The head was selected on the **mean**, and the mean
hides a real trade — published here rather than buried:

| category | stock head | **this head** | |
|---|---:|---:|---|
| long_context | 5.00 | 4.53 | **−0.47** |
| agentic_format | 4.41 | 4.16 | **−0.25** |
| multi_turn | 4.06 | 4.83 | +0.77 |
| code_edit | 4.08 | 4.18 | +0.10 |
| short_factual | 3.11 | 3.87 | +0.76 |
| reasoning | 1.82 | 3.57 | +1.75 |
| explanation | 1.70 | 3.00 | +1.30 |
| code_gen | 1.86 | 2.59 | +0.73 |
| **mean** | 3.2550 | **3.8413** | **+18.0 %** |

The gain is concentrated in the *constructive* categories — reasoning nearly doubles — and it is
paid for on the two *reconstructive* ones the stock head was already best at. Under this project's
own release rule that is a **failure of 3 of 6 category floors**, and the rule is stated so that a
downstream user can predict their own workload instead of inheriting a mean.

**If your workload is dominated by long-context reconstruction, the stock head may serve you better.**
For mixed agentic work — tool calls, multi-turn, code edits, reasoning — this head is a clear win.

## Task accuracy

Measured through the same server, `effort=low`, 24k token budget, temperature 1.0 / top-p 0.95.

| benchmark | score | n |
|---|---:|---:|
| AIME 2024 | **91.7 %** [80.0, 100.0] | 60 |
| GPQA-Diamond | **83.8 %** [78.1, 88.3] | 198 |
| MMLU-Pro | **74.7 %** [67.2, 81.0] | 150 |
| BFCL (tool calling) | **86.2 %** | 240 |
| BFCL-Live | **78.7 %** | 508 |
| LiveCodeBench | 46.9 % [39.6, 54.2] *(8k budget, 59 % truncated — not quotable)* | 175 |

**A caveat that matters more than any single number.** GPQA at a 24k budget was *pre-registered* at
87–93 % before the run, with a stated falsification threshold of 85 %. It landed at **83.8 %** — the
prediction was wrong, and the reason is a finding about this pruned model: **19 of 51 items that
exhaust an 8k budget never terminate even at 24k**, and the ones that do finish score 78.1 %, not the
93.9 % that items terminating inside 8k reach. Prompts needing long chains are *qualitatively*
harder here, not merely longer. Budget your context accordingly.

## Context

| context | KV format | fits |
|---:|---|---|
| 32,768 | fp32 rows | yes, 119.9 / 122.8 GiB |
| **131,072** | packed fp8 + ue8m0 | **yes, 117.0 / 122.8 GiB** |
| 262,144 | packed | allocates at 121.8 / 122.8 — **too little headroom to serve** |

KV packing is **bit-exact** (720 B/row vs 2048; the fp32 path already stored e4m3-representable
values in four bytes) and costs ~12 % of prefill throughput, so it is worth enabling only when the
context needs it.

## Limitations, stated plainly

- **Measured on one machine.** Every number above comes from a single Jetson AGX Thor. Throughput
  will differ on other hardware; accuracy should not.
- **Single-stream.** The server these numbers came from serves one request at a time by design. This
  is not a multi-tenant serving checkpoint.
- **The eval battery is partial.** The suite is complete; the 24k extension covers 4 of 8 tasks.
  LiveCodeBench, MATH-500, HumanEval and SciCode are at the 8k budget and several are truncation-
  limited, which is why LCB is marked not quotable above.
- **This is a REAP-pruned model.** 160 of 256 experts per layer. Comparisons against unpruned
  DeepSeek-V4-Flash are not like-for-like.

## Licence

MIT, following the base checkpoint. The DeepSeek copyright notice is retained in `LICENSE`. This
repository adds only the fine-tuned draft-head tensors in shards 46–48.