Same on my 5090:
Avg prefill: ~2300 tok/s .
Avg decode: ~65 tok/s.
Avg acceptance rate: 25%

Tested with configs using llama cpp (b10358):

[muse_glimmer]
model =./models/muse-glimmer-30B-kquant-dynamic.gguf
md=./models/dflash-kquant.gguf
ctx-size=131072
temp=1.0
top-p=0.95
top-k=64
device=CUDA0
spec-type=draft-dflash
spec-draft-device=CUDA0
spec-draft-n-max=15
spec-draft-n-min=2 
reasoning-preserve=1

Hey folks, pls check https://huggingface.co/meta-models/Muse-Glimmer-30B/discussions/21. This might help?

Hey folks, pls check https://huggingface.co/meta-models/Muse-Glimmer-30B/discussions/21. This might help?

Muse Glimmer 30B + DFlash Performance Report — RTX 5090 (Windows)

Date: 2026-08-12
Prepared for NVIDIA review

1. System Configuration

2. Models

3. Configurations Tested

Config A — Baseline, no DFlash, no flash attention

llama-server.exe
  --model  muse-glimmer-30B-kquant-17gb.gguf
  -ngl 99 -np 1 -c 8192
  --host 0.0.0.0 --port 8000

Config B — Baseline, no DFlash, flash attention ON

llama-server.exe
  --model  muse-glimmer-30B-kquant-17gb.gguf
  -ngl 99 -np 1 -c 8192
  --flash-attn on
  --host 0.0.0.0 --port 8000

Config C — DFlash ON, small context (Meta benchmark config)

llama-server.exe
  --model       muse-glimmer-30B-kquant-17gb.gguf
  --model-draft dflash-kquant.gguf
  -ngl 99 -ngld 99 -np 1 -c 8192
  --spec-type draft-dflash
  --spec-draft-n-max 15
  --flash-attn on
  --temp 0 --top-k 1 --jinja
  --host 0.0.0.0 --port 8000

Config D — DFlash ON, large context (agentic/Claude Code use)

llama-server.exe
  --model       muse-glimmer-30B-kquant-17gb.gguf
  --model-draft dflash-kquant.gguf
  --mmproj      mmproj-kquant.gguf
  -ngl 99 -ngld 99 -np 1 -c 131072
  --spec-type draft-dflash
  --spec-draft-n-max 15
  --flash-attn on
  --reasoning-preserve
  --host 0.0.0.0 --port 8000

4. Startup Log — DFlash Initialization

Observed on every DFlash run (confirms DFlash loaded correctly):

E llama_init_from_model: failed to initialize the context: dflash requires ctx_other
  to be set (this warning is normal during memory fitting)
W srv    load_model: [spec] failed to measure draft model memory: failed to create
  llama_context from model

I common_speculative_init_result: loading draft model 'dflash-kquant.gguf'
I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 131072, kv_unified = 'false'
I common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'
I common_speculative_impl_draft_dflash: - n_max=15, n_min=0, p_min=0.00
I common_speculative_impl_draft_dflash: - block_size=16, mask_token_id=201818, n_extract=5
I common_speculative_impl_draft_dflash: requested draft size (n_max=16, n_min=0) exceeds
  the trained block size 16 -- clamping to 15
I srv  llama_server: model loaded
I srv  llama_server: listening on http://0.0.0.0:8000

Note: The ctx_other error on line 1 is expected during memory fitting — it does NOT prevent DFlash from loading.

5. Measured Results

All generation numbers are measured via the OpenAI-compatible /v1/chat/completions API.
Test prompt: "Write a complete Python implementation of a binary search tree with insert, delete, search, and in-order traversal. Include type hints and docstrings."

5.1 Baseline decode speed

Finding: --flash-attn on reduces baseline throughput by ~28% on this hardware/build.
Flash attention is required for DFlash block-parallel; without it, DFlash cannot be enabled.

5.2 DFlash — small context benchmark (Meta's published config)

Config C: c=8192, greedy, draft-dflash

Finding: DFlash IS active (confirmed by draft acceptance lines in server log),
but 14% acceptance at mean length 3.09 provides minimal speedup over the FA baseline of 55.5 tok/s.
Net effect: 55.5 → 78 = 1.4× speedup.

Server log confirmation:

I slot print_timing: draft acceptance = 0.14262 (347 accepted / 2433 generated), mean len = 3.12
I slot print_timing: draft acceptance = 0.14250 (345 accepted / 2421 generated), mean len = 3.09

5.3 DFlash — large context, real sampling (agentic use case)

Config D: c=131072, temp=1.0, draft-dflash, with ~80K token prefill (Claude Code session)

Finding: Run 1 (fresh 80K-token context) achieves 168 tok/s — 3.0× over FA baseline.
This matches Meta's published 3.1× speedup claim.
Follow-up turns (runs 2–3) have lower acceptance, producing ~76–98 tok/s.

5.4 Prefill (prompt processing) rate

Observed across all configs:

6. Summary

7. Key Observations

1. DFlash is working. Block-parallel acceptance is confirmed via server logs. 3.0× speedup observed in the right conditions.

2. Acceptance rate drives everything. DFlash at 30% acceptance (mean_len 5.58) gives 168 tok/s. At 14% acceptance (mean_len 3.09) it gives only 78 tok/s.

3. Large context boosts acceptance. Fresh turns with ~80K token context achieve 30% acceptance. Short prompts or follow-up turns drop to 9–14%. DFlash appears to be optimized for long-context continuation — consistent with Meta's statement that "the DFlash head has been mostly optimized for agentic and coding flows."

4. Flash attention hurts raw throughput. Without DFlash, --flash-attn on costs ~22 tok/s (77→55). This is likely a Windows Clang build issue — Linux builds typically see FA as neutral or faster.

5. The gap to 233 tok/s. To close the gap from 168 to 233 tok/s, we need either higher acceptance rates (>30%) or the build optimizations in PR #26842 (referenced by Meta engineer ruanslv as unmerged but important). Another community user confirmed 238 tok/s after cherry-picking that PR.

6. kv_unified = 'false' — the server log shows this value on every startup. It is unclear whether setting this to true would improve DFlash KV-sharing and boost acceptance rate. No llama-server flag to control it was found.

8. Open Questions for NVIDIA / Meta

1. What kv_unified value does the internal benchmark use? The community build shows false; is true required for peak DFlash performance?

2. What is the expected DFlash acceptance rate for the benchmark prompt at c=8192? We see 14%; the benchmark appears to need >30%.

3. Is PR #26842 the primary path to closing the ~65 tok/s gap? If so, an official build with that patch would help the community reproduce the published numbers.

4. Are there Windows-specific issues with the DFlash CUDA kernels in the Clang build? Flash attention overhead on Windows is significantly higher than Linux reports suggest.