README.md

metadata

license: other
library_name: torchspec
tags:
  - speculative-decoding
  - draft-model
  - dspark
  - glm-5.2
  - code

dspark-glm52-draft

DSpark (DFlash-family single-shot parallel drafting) draft models for GLM-5.2 speculative decoding, optimized for code / coding-chat traffic. This repo consolidates the lineage; each historical checkpoint is an immutable git tag (pick it from the revision dropdown or pass revision= to from_pretrained/hf_hub_download).

main tracks the current best checkpoint.

Revisions

tag	arch	checkpoint	trend official_accept_len	note
bs8-moe8-lr7xct-20260722-iter7651 (= main)	block-8 MoE-8 + DFlare	lr7x-ct iter_7651	4.704 / 8	current best; high-LR warm-restart continuation
bs8-moe8-dflare-20260715-iter10001	block-8 MoE-8 + DFlare	dflare iter_10001	4.59 / 8	predecessor run's pre-bump champion
bs8-moe8-dflare-20260715-iter5251	block-8 MoE-8 + DFlare	dflare iter_5251	~4.49	earlier dflare snapshot
bs8-moe8-dflare-20260714	block-8 MoE-8 + DFlare	dflare (earlier)	—	earliest block-8 export
bs16-moe-20260705	block-16 MoE	2026-07-05	—	distinct architecture (γ=16), research arm — NOT a version of the block-8 line

All trend numbers are teacher-forced metrics on a fixed 96-row set that overlaps the training view (no holdout) — NOT serving measurements and NOT SOTA claims. Keep/kill belongs to the offline 3-task screen (gsm8k + humaneval + mt-bench) and an sglang serving bench. Trend-eval noise is ±0.03–0.04.

Shared architecture (block-8 line)

• DSpark block size 8, 5 draft layers, hidden 6144, 64 heads (head_dim 64).
• Vanilla MoE-8 (8 routed top-2 + 1 shared per layer, moe_intermediate 3072); Jakiro OFF.
• DFlare per-layer context fusion, Markov/GRIFFIN correction head (rank 256) + confidence head (confidence_detach_input, griffin_fuse).
• Reads GLM-5.2 target hidden states from layers [8, 23, 39, 55, 70] of 78.
• 4.44 B total params (~2.7 B active/token), bf16, 8.3 GB.
• draft_vocab_size 32000 vs target 154880 → serve via sglang's DSpark loader with the d2t full-vocab scatter (d2t int64 [32000], t2d bool [154880] ship in the safetensors; do NOT bf16-cast them). Tokenizer: use GLM-5.2's (not bundled).

Target & data (block-8 line)

• Target (generation AND extraction): lukealonso/GLM-5.2-NVFP4 (modelopt_fp4, fp8-e4m3 KV), live target prefill — fully on-policy, no disk target-state cache.
• SFT view sha256:c17516837c… (1,379,922 production DB conversations + 522,176 on-policy OPB rows), seq 4096, GLM chat template.

See each tag's own README for its exact step, LR schedule, and provenance hashes.