meshllm/glm-4.7-flash-spd-sidecar

GLM 4.7 Flash SPD Sidecar

This repository contains a trained Speculative Pipeline Decoding (SPD) sidecar head for zai-org/GLM-4.7-Flash, produced for mesh-llm/Skippy PR #866.

The head was trained against a frozen GLM 4.7 Flash checkpoint using the reference SPD trainer from yuyijiong/speculative_pipeline_decoding, with mesh-llm patches for GLM 4.7 topology, reduced-vocab KL training, and usable-row filtering.

Intended Use

This is a research/validation sidecar for verified speculative decoding. The target GLM model remains authoritative and verifies every proposed token. Current Skippy/Rust code can validate the manifest and serving checkpoint shape; live Rust-side SPD execution is still the integration target.

Training Configuration

Field	Value
Base model	zai-org/GLM-4.7-Flash
Dataset	HuggingFaceH4/ultrachat_200k, split train_sft
Raw train rows	8192
Usable train rows after shifted-label filtering	7377
Stages	3
Stage layer boundaries	15,31,47
Spec layers	1
Draft vocab size	16000 GLM-tokenizer ids
Max train length	512
Epochs	1
Optimizer steps	923
Batch / grad accumulation	1 / 8
Learning rate	2e-5, 50 warmup steps
Device	Apple MPS

Verified Acceptance Eval

Evaluation used the donor verifier with target verification enabled, greedy decode (temperature=0.0), draft_top_k=1, and 4 prompts each from MT-Bench, HumanEval, and GSM8K.

Scope	Acceptance	Equivalent accept length	Theoretical gain	Accepted flags	New tokens	Decode steps
Overall	0.3699	1.1098	11.09%	109/768	768	2076
MT-Bench	0.3596	1.0787	7.88%	26/256	256	712
HumanEval	0.3689	1.1066	10.76%	35/256	256	694
GSM8K	0.3821	1.1463	14.64%	48/256	256	670

Files

• train/speculation_head_final.pt: original PyTorch reference checkpoint.
• train/spd-head.safetensors: Rust-readable serving checkpoint exported as F16.
• train/skippy-spd-head.json: Skippy SPD manifest with topology, draft vocabulary ids, and checksums.
• train/trainer_state.json: final trainer state and logged loss/accuracy curve.
• eval/summary/*.json: verifier acceptance summary.
• eval/raw/*.jsonl: per-sample verifier trace.
• data/draft_vocab_top_16000.json: generated GLM-tokenizer draft vocabulary.
• repro/train.sh: training/eval command used for this artifact.
• repro/export.sh: safetensors export command.
• checksums.json: SHA-256 and byte size for uploaded files.

Manifest Validation

The exported manifest was validated with:

SKIPPY_SPD_MANIFEST=/tmp/glm-4.7-flash-spd-sidecar-hub/train/skippy-spd-head.json   cargo test -p skippy-runtime --lib validates_external_manifest_when_skippy_spd_manifest_is_set