GLM-5.3-DFlash2-ft8

A fine-tune of incoai/GLM-5.3-DFlash2, the DFlash 2 draft model for zai-org/GLM-5.3. It is not a standalone language model: it runs inside a speculative-decoding server and drafts tokens for the target model to verify. Decoding stays lossless, as with the base drafter: the target accepts or rejects every drafted token.

What changed from the base: the weights only. The architecture and config.json are the base model's (6 layers, block size 8, selector rank 256). It was trained further "on policy": on hidden states and next-token labels captured from a running GLM-5.3 target over about 2,900 captured sequences (prose, code and long-context chats), so the drafter sees the same inputs in training that it sees when serving.

Which target it was trained against: a 2.75 bits-per-weight EXL3 quantisation of a modified variant of GLM-5.3, served across four NVIDIA DGX Sparks (tensor parallel 4). That is the only target it has been measured on.

Measured

Four DGX Sparks (GB10), tensor parallel 4, one stream, thinking on, the drafter alone (no MTP heads), draft depth 7, confidence 0.3. Measured 2026-10-07.

this drafter MTP heads only
prose, short / 32K / ~120K context (tokens per second) 39.1 / 34.2 / 31.2 37.1 / 32.6 / not measured
code, short / 32K / ~120K context (tokens per second) 44.8 / 42.9 / 39.4 41.1 / 35.2 / not measured
prompt processing, 32K / 128K (tokens per second) 1,196 / 1,061 1,188 / 1,039

Accepted tokens per decoding round on prose at 32K context, same protocol:

drafter tokens per round
incoai/GLM-5.3-DFlash2 (base) 2.029
an earlier fine-tune (ft7) 2.057
this one (ft8) 2.099

What is not known

  • Other targets. It has not been measured against full-precision GLM-5.3, an NVFP4 target or any other quantisation. A drafter tuned to one target's hidden states may accept more or fewer tokens on another; measure before relying on it, and compare with the base drafter.
  • Other servers. The file layout is the base model's (config.json, model.safetensors), so a server that loads incoai/GLM-5.3-DFlash2 should load this in its place. It has not been tested by us on SGLang or vLLM.
  • Noise. The prose figures are six runs a setting, with run-to-run noise of about 3 %. The gain over the base drafter (about 3.5 % more tokens per round on prose) is small.
  • Other hardware. All figures are from DGX Sparks.

Use

Point your server's draft-model path at this repository instead of the base drafter. With the base model card's SGLang example, that is --speculative-draft-model-path drowzeys/keys-GLM-5.3-DFlash2-ft8.

model.safetensors md5: a50f76216578ff2ce3479d6873da1952.

Licence and credit

Apache License 2.0, the licence of the base drafter (see LICENSE and NOTICE). The base drafter and the DFlash 2 method are the work of the authors of incoai/GLM-5.3-DFlash2 (blog, code). The target model is zai-org/GLM-5.3 and its licence applies to it. This repository changes the base drafter's weights by further training and is not affiliated with or endorsed by either.

Downloads last month
57
Safetensors
Model size
2B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for drowzeys/keys-GLM-5.3-DFlash2-ft8

Base model

zai-org/GLM-5.3
Finetuned
(1)
this model