Permission request: publish a target-attached DFlash2 fine-tune for an EXL3 3-bit GLM-5.3
Title: Permission request: publish a target-attached DFlash2 fine-tune for an EXL3 3-bit GLM-5.3
Hello, and thank you for DFlash2 — it is the backbone of our speculative decoding setup.
What we did. We serve full GLM-5.3 quantized to EXL3 3.0 bpw on four NVIDIA DGX Spark (GB10) nodes at TP=4. Third-party KLD
measurement of that checkpoint against BF16 shows mean KL 0.102 nats and 91.1% top-1 agreement, so roughly 9% of argmax decisions
differ from the unquantized model. A drafter trained against BF16 behaviour is therefore verified by a target that disagrees with it
on about one token in eleven, which caps acceptance regardless of drafter quality.
So we fine-tuned incoai/GLM-5.3-DFlash2 on-policy against our quantized target: captured auxiliary hidden states from the live
server, trained with block cross-entropy plus the candidate-selector loss (LR 1e-4, 3200 steps, ~1.1M tokens, 4.5 h on one GB10).
Architecture, tokenizer, block size and selector configuration are unchanged; only the drafter weights moved.
What we are asking. We would like to publish those fine-tuned weights on Hugging Face, non-commercially, so that people
reproducing our 4×DGX-Spark recipe get the same acceptance we measured. We understand the CC BY-NC-ND license does not permit
distributing modified weights, which is why we are asking rather than assuming — the repository is currently private and will stay
private unless you agree.
If you would prefer we not publish, that is completely fine. We will keep the weights private and publish only the training
procedure so others can reproduce the fine-tune from your original release.
If you do agree, the model card already: names incoai/GLM-5.3-DFlash2 as the base model, keeps your cc-by-nc-nd-4.0 tag
rather than relabelling it, describes exactly what was changed, restricts use to non-commercial, and offers immediate takedown on
request. We are happy to add any wording or restriction you would like.
For reference, the full recipe and measurements are public here:
https://github.com/drowzeys/keys-GLM-5.3-EXL3-3.0BPW-abliterated-vLLm-cuda-Exl3
Measured effect on our stack (200K context, DCP=1, k=5, block verification, single stream): prose 16.2 tok/s with the fine-tune.
We are benchmarking your stock drafter on the identical configuration now and will publish both numbers side by side either way, so
the value of the fine-tune is transparent.
Thank you for considering it, and thank you again for releasing DFlash2.
Withdrawing this request — thank you for your time.
We benchmarked our target-attached fine-tune against your stock DFlash2 drafter on the identical serving configuration, and the
stock drafter performed as well or better on most tasks (it won typical-prose and code; the fine-tune won prose and list; the
differences sit inside our measured run-to-run spread). There is no benefit worth asking you to make an exception for.
Our fine-tuned weights have been deleted, and our published recipe now uses your stock release as-is, with credit. Thank you for
DFlash2 — it is doing the work here unmodified.
Recipe, for reference: https://github.com/drowzeys/keys-GLM-5.3-EXL3-3.0BPW-abliterated-vLLm-cuda-Exl3