Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
ProCreations 
posted an update 2 days ago
Post
1,017
Nanbeige4.3-33B DFlash2 speculative drafter ⚡️

1.76× faster decoding on BF16, 1.65× on FP8, 1.51× on NVFP4. Agentic coding ~2.6×. Same outputs: exact rejection sampling. This is an early result, an improved version with higher acceptance is in training.

SGLang 0.5.21, drop-in:
ProCreations/Nanbeige4.3-33B-DFlash2

Could you qualify "same outputs" in the post? Your card reports greedy divergence on 18/25 prompts at 0-1 BF16-ULP logit ties, so token-for-token regression tests can differ even with exact stochastic rejection. Linking that caveat here would help readers expecting deterministic replay: https://huggingface.co/ProCreations/Nanbeige4.3-33B-DFlash2#greedy-numerical-behaviour

The card makes the batch-1 and decode/E2E distinction clear. For a serving decision, I would add a concurrency sweep with p95 time-to-first-token and inter-token latency, at a fixed prompt/output length mix. Reporting peak KV-cache memory alongside it would show whether the extra draft state changes how many requests fit before queueing. That would make the single-request speedup much easier to translate into capacity.