Lissa1-4B-DFlash

Experimental speculative-decoding drafter for Lissa1-4B.

This repository is a sidecar, not a standalone language model. It must be paired with the compatible Lissa1-4B target, tokenizer and serving engine. The drafter proposes tokens; the target verifies them.

Developer-to-developer rule: speculative decoding is a serving optimization, not a quality upgrade. A draft can raise acceptance and still reduce net tok/s if verification and draft overhead cost more than it saves. Always benchmark the exact target quant, engine, prompt domain and block size you intend to deploy.

Recommended current setup

For Apple Silicon, use the community draft as the current baseline:

python -m mlx_vlm generate \
  --model ./4bit \
  --draft-model z-lab/Qwen3.5-4B-DFlash \
  --draft-kind dflash \
  --draft-block-size 8 \
  --prompt "Explain this race condition and propose a minimal safe patch." \
  --max-tokens 256 \
  --temperature 0 \
  --no-verbose

For vLLM/SGLang, use the engine's installed DFlash speculative-decoding flags and this target/draft pair. The flag spelling changes across engine releases; inspect vllm serve --help or the SGLang launcher help before copying a command from another version.

For llama.cpp, the current experimental form is:

./llama-server \
  -m Lissa1-4B-Q4_K_M.gguf \
  -md Lissa1-4B-DFlash.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 8 \
  -ngl 99 -ngld all -fa on --jinja --port 8080

Measured results

MLX acceptance smoke

Drafter	Block	Accepted tok/round	Draft acceptance
community z-lab/Qwen3.5-4B-DFlash	8	2.78	55.4%
community z-lab/Qwen3.5-4B-DFlash	16	2.78	50.0%
this Lissa drafter	8	1.57	18.8%
this Lissa drafter	16	1.57	17.9%

The community draft is external and is not claimed as a Lissa-trained checkpoint. It is the current serving baseline while this drafter is retrained on larger Lissa-generated data.

GPU serving

The community draft paired with the corrected Lissa FP8 target measured:

• vLLM: 173.88 → 242.99 tok/s;
• SGLang: 171.86 → 249.97 tok/s.

The own draft is currently net-negative in the measured AWQ matrix:

• vLLM: 197.95 → 153.0 tok/s;
• SGLang: 204.12 → 138.8 tok/s.

llama.cpp

With Lissa Q4_K_M and the -fa on --spec-draft-n-max 16 path:

Target only: 143.70 tok/s
Community DFlash: 124.47 tok/s
Own DFlash: 69.49 tok/s

That is an exploratory ranking without a complete acceptance delta, not a final deployment verdict. The current target-specific training gate is also failed: 1.1014x measured versus a 1.6x promotion target.

Why this is experimental

The target was heavily fine-tuned after the base drafter distribution was learned. The current own drafter was trained on an older 2,500-row cache and does not follow the final Lissa distribution closely enough. The next run is being rebuilt as:

Lissa-generated prompts/completions
→ pure-Transformers hidden-state capture
→ shape/NaN/dtype validation
→ 1-step and 50-step training smoke
→ full DFlash training
→ HF/GGUF export
→ net tok/s + acceptance matrix

The draft will be promoted only if it beats target-only on the serving domain, not merely because its acceptance percentage looks good.

Sampling and evaluation

Use greedy decoding for acceptance measurements. For normal developer traffic:

• patches and tool calls: temperature 0–0.2, top_p 0.9–1.0;
• debugging: temperature 0.2–0.4, top_p 0.9;
• architecture exploration: temperature 0.4–0.6, top_p 0.9–0.95.

Record proposed tokens, accepted tokens, acceptance length, wall time and net tok/s. Test Spanish chat, code, tool calls and long context separately.

Long-context caution

The Ornith-1.5-9B ia-lab evidence is useful for designing the Lissa1 tests: Q4/Q4 KV, Flash Attention, slots, continuous batching and explicit context ladders. It is not a Lissa1 result. The historical report found DFlash useful in single-slot short-context work but less attractive at 512K; the Lissa1 decision must be based on its own net tok/s and domain prompts.

Also, the Ornith vLLM MTP result must not be copied here. Lissa1-4B has zero MTP tensors after the merge. This repository is for the DFlash sidecar path only.

Links

• Lissa1 target model
• GGUF target
• FP8 target
• AWQ target
• MLX target
• External community draft
• Lissa1-DFlash collection

Future QLoRA adapters are separate from this drafter: database, frontend, backend, devops/cicd, agents/tools, security and platform. The drafter will be retrained against the final base/model distribution rather than silently reused across those adapters.