README.md

---
license: mit
base_model: deepseek-ai/DeepSeek-V4-Flash-0731
tags:
  - gguf
  - deepseek
  - rocm
  - strix-halo
  - quantized
---

# DeepSeek-V4-Flash-0731 — Option 1: adaptive down-experts (ROCmFP3-MIX)

The same bytes as the uniform ROCmFPX baseline, holding a measurably better model.

The 43 down-projection expert tensors move from **qtype 104** (`Q3_0_ROCMFPX`, uniform) to
**qtype 105** (`Q3_1_ROCMFP3_MIX`, adaptive). Both are **14 bytes per 32 weights**, so the
file is byte-for-byte the same size, reads the same bytes per token, and costs the kernels
the same work. Nothing else in the model changes: attention, dense, gate/up, norms and
embeddings are identical to the baseline.

## What "adaptive" means here

A uniform format compiles one interpretation into the format itself — the same codebook for
every block in the model. An adaptive format moves that interpretation into the artifact:
each expert carries its own small learned table, chosen by a per-expert search against
Hessian-weighted sensitivity rather than by a rule applied to the whole tensor.

Those tables are ~375 KB against ~102 GB. That asymmetry is the whole trick — a per-expert
choice costs almost nothing to record and buys a much better fit, because expert weight
distributions genuinely differ.

## Measured quality

From the quality-transfer ledger that selected this assignment (fake-quant, against the
unquantized model, 2 seeds x 2 corpora x 5 policies):

| | uniform (P0) | adaptive |
|---|---|---|
| wikitext2 PPL | 3.8573 | **3.6760** |
| wikitext2 KL | 0.2157 | **0.0953** |
| c4 PPL | 12.5342 | **11.5245** |
| c4 KL | 0.2064 | **0.0677** |

**Aggregate: KL -57.0%, perplexity -5.6%, at identical footprint.**

KL is the number that matters most: it measures divergence from the *unquantized* model, so
it says this artifact reproduces the original's output distribution roughly twice as
faithfully as the uniform recipe does — using the same bytes.

Two honest caveats:

- **96.4% weighted expert coverage.** The calibration corpus never routed to some experts;
  those fall back to the conservative uniform rung rather than being guessed at. The gains
  above are earned on the covered majority.
- These are **fake-quant research measurements** against the native model, not end-to-end
  serving benchmarks. Throughput and speculative-decode figures are reported separately,
  under one protocol with the prompt mix stated.

## Single file — no sidecar

Earlier adaptive artifacts shipped as a model **plus a required `.p4mix.bin` companion**:
separate the two and the tensors are undecodable. The learned tables are now carried inside
the GGUF, as a verbatim byte blob in the KV block:

```
deepseek4.p4mix.sidecar   375,320 bytes
```

Loaders prefer the embedded copy and fall back to a loose file, so existing two-file
artifacts keep working unchanged.

## Requirements

Needs a server build with the qtype-105 decode kernels — upstream `lucebox` plus the
adaptive series, on branch
[`feat/ds4-adaptive-on-upstream`](https://github.com/GeometricAGI/lucebox-hub/tree/feat/ds4-adaptive-on-upstream)
(upstream main + 20 reviewable commits, full suite green). A stock build without those
kernels will not decode qtype 105.

Pair with
[DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF](https://huggingface.co/Geometric-AI/DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF)
for speculative decode, and compare against
[DeepSeek-V4-Flash-0731-ROCMFPX](https://huggingface.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCMFPX),
the uniform baseline this is measured against.

## Integrity

```
sha256  a431f955c639107d3f34ab4cc65edc806038588a0c258abd0bc3e9074b908096
file    ds4-0731-opt1.gguf  (102,321,006,592 bytes)
```

That is **+375,392 bytes** over the uniform baseline — the embedded codebooks and their KV
overhead, and the entire size cost of the adaptive encoding.

## License

MIT, inherited from the base model.

> **Checksum correction (2026-08-05).** An earlier revision of this card listed
> `accabb4c…`, which was the hash of a pre-release build, not of the published file.
> A correct download failed that check. The value above (`a431f955…`) is the real one
> and matches the git-LFS oid and `x-linked-etag`. Thanks to @hycris for catching it.

## Serving: keep the model's six routed experts

**Do not pass `--ds4-expert-top-k 4` with this artifact.** That flag keeps only four of the
six routed experts and renormalizes. On the uniform ROCMFPX artifact it is harmless; on this
adaptive one it produces degenerate output.

Measured 2026-08-05, exact-copy fidelity over 20 identifiers x 3 repeats at temperature 0
(the model is asked to echo a line of Python already present in its prompt, so anything but a
verbatim copy is a defect):

| | top-k 6 (model default) | top-k 4 |
|---|---|---|
| this artifact (adaptive) | 95.0% | **60.0%** |
| uniform ROCMFPX | 100% | 100% |

The failures are not degraded paraphrases -- they are `" 0 0 0 0 0 ..."`, repeated markdown
fragments and empty strings, on prompts as trivial as "repeat this line". Deterministic, and
reproducible at two context sizes and with fused decode on or off.

Some published example commands (including in this project's own `server/docs/DS4.md` until
today) included `--ds4-expert-top-k 4`. Omit it, or pass `0`, to use the model default.