| --- |
| license: mit |
| base_model: deepseek-ai/DeepSeek-V4-Flash-0731 |
| tags: |
| - gguf |
| - deepseek |
| - rocm |
| - strix-halo |
| - quantized |
| --- |
| |
| # DeepSeek-V4-Flash-0731 — Option 1: adaptive down-experts (ROCmFP3-MIX) |
|
|
| The same bytes as the uniform ROCmFPX baseline, holding a measurably better model. |
|
|
| The 43 down-projection expert tensors move from **qtype 104** (`Q3_0_ROCMFPX`, uniform) to |
| **qtype 105** (`Q3_1_ROCMFP3_MIX`, adaptive). Both are **14 bytes per 32 weights**, so the |
| file is byte-for-byte the same size, reads the same bytes per token, and costs the kernels |
| the same work. Nothing else in the model changes: attention, dense, gate/up, norms and |
| embeddings are identical to the baseline. |
|
|
| ## What "adaptive" means here |
|
|
| A uniform format compiles one interpretation into the format itself — the same codebook for |
| every block in the model. An adaptive format moves that interpretation into the artifact: |
| each expert carries its own small learned table, chosen by a per-expert search against |
| Hessian-weighted sensitivity rather than by a rule applied to the whole tensor. |
|
|
| Those tables are ~375 KB against ~102 GB. That asymmetry is the whole trick — a per-expert |
| choice costs almost nothing to record and buys a much better fit, because expert weight |
| distributions genuinely differ. |
|
|
| ## Measured quality |
|
|
| From the quality-transfer ledger that selected this assignment (fake-quant, against the |
| unquantized model, 2 seeds x 2 corpora x 5 policies): |
|
|
| | | uniform (P0) | adaptive | |
| |---|---|---| |
| | wikitext2 PPL | 3.8573 | **3.6760** | |
| | wikitext2 KL | 0.2157 | **0.0953** | |
| | c4 PPL | 12.5342 | **11.5245** | |
| | c4 KL | 0.2064 | **0.0677** | |
|
|
| **Aggregate: KL -57.0%, perplexity -5.6%, at identical footprint.** |
|
|
| KL is the number that matters most: it measures divergence from the *unquantized* model, so |
| it says this artifact reproduces the original's output distribution roughly twice as |
| faithfully as the uniform recipe does — using the same bytes. |
|
|
| Two honest caveats: |
|
|
| - **96.4% weighted expert coverage.** The calibration corpus never routed to some experts; |
| those fall back to the conservative uniform rung rather than being guessed at. The gains |
| above are earned on the covered majority. |
| - These are **fake-quant research measurements** against the native model, not end-to-end |
| serving benchmarks. Throughput and speculative-decode figures are reported separately, |
| under one protocol with the prompt mix stated. |
|
|
| ## Single file — no sidecar |
|
|
| Earlier adaptive artifacts shipped as a model **plus a required `.p4mix.bin` companion**: |
| separate the two and the tensors are undecodable. The learned tables are now carried inside |
| the GGUF, as a verbatim byte blob in the KV block: |
|
|
| ``` |
| deepseek4.p4mix.sidecar 375,320 bytes |
| ``` |
|
|
| Loaders prefer the embedded copy and fall back to a loose file, so existing two-file |
| artifacts keep working unchanged. |
|
|
| ## Requirements |
|
|
| Needs a server build with the qtype-105 decode kernels — upstream `lucebox` plus the |
| adaptive series, on branch |
| [`feat/ds4-adaptive-on-upstream`](https://github.com/GeometricAGI/lucebox-hub/tree/feat/ds4-adaptive-on-upstream) |
| (upstream main + 20 reviewable commits, full suite green). A stock build without those |
| kernels will not decode qtype 105. |
|
|
| Pair with |
| [DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF](https://huggingface.co/Geometric-AI/DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF) |
| for speculative decode, and compare against |
| [DeepSeek-V4-Flash-0731-ROCMFPX](https://huggingface.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCMFPX), |
| the uniform baseline this is measured against. |
|
|
| ## Integrity |
|
|
| ``` |
| sha256 a431f955c639107d3f34ab4cc65edc806038588a0c258abd0bc3e9074b908096 |
| file ds4-0731-opt1.gguf (102,321,006,592 bytes) |
| ``` |
|
|
| That is **+375,392 bytes** over the uniform baseline — the embedded codebooks and their KV |
| overhead, and the entire size cost of the adaptive encoding. |
|
|
| ## License |
|
|
| MIT, inherited from the base model. |
|
|
| > **Checksum correction (2026-08-05).** An earlier revision of this card listed |
| > `accabb4c…`, which was the hash of a pre-release build, not of the published file. |
| > A correct download failed that check. The value above (`a431f955…`) is the real one |
| > and matches the git-LFS oid and `x-linked-etag`. Thanks to @hycris for catching it. |
|
|
| ## Serving: keep the model's six routed experts |
|
|
| **Do not pass `--ds4-expert-top-k 4` with this artifact.** That flag keeps only four of the |
| six routed experts and renormalizes. On the uniform ROCMFPX artifact it is harmless; on this |
| adaptive one it produces degenerate output. |
|
|
| Measured 2026-08-05, exact-copy fidelity over 20 identifiers x 3 repeats at temperature 0 |
| (the model is asked to echo a line of Python already present in its prompt, so anything but a |
| verbatim copy is a defect): |
|
|
| | | top-k 6 (model default) | top-k 4 | |
| |---|---|---| |
| | this artifact (adaptive) | 95.0% | **60.0%** | |
| | uniform ROCMFPX | 100% | 100% | |
|
|
| The failures are not degraded paraphrases -- they are `" 0 0 0 0 0 ..."`, repeated markdown |
| fragments and empty strings, on prompts as trivial as "repeat this line". Deterministic, and |
| reproducible at two context sizes and with fused decode on or off. |
|
|
| Some published example commands (including in this project's own `server/docs/DS4.md` until |
| today) included `--ds4-expert-top-k 4`. Omit it, or pass `0`, to use the model default. |
|
|
