Arete-Dflash2

DFlash2 draft model for speculative decoding, compatible with Arete-27B, Arete-27B-Uncensored, Qwen3.8-27B, and Qwen3.8-27B-Uncensored β€” meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev on the GB10 (2x GB10 cluster, vLLM 0.28).

What this is

A DFlash2 drafter: a small (1.92B) draft model that proposes tokens for a larger verifier (target) model. The verifier scores the drafts in one forward pass and accepts a prefix β€” lossless speedups, identical output.

In z-lab export format (DFlash2DraftModel + dflash_config), drop-in for vLLM --speculative-config.

repo base training
SargeDev/Arete-Dflash2 incoai/Qwen3.8-27B-DFlash2 (Z-Lab DFlash2, 1.92B) 7 epochs finetune on Arete-27B-Uncensored regen data

Targets

Both drafters are drop-in compatible with all three (greedy losslessness 5/5 on every combo β€” verified):

target repo
Arete-27B-Uncensored (our uncensored build, bf16) SargeDev/Arete-27B-Uncensored-V2
Arete-27B (censored, original) NousResearch/Arete-27B
Qwen3.8-27B Qwen/Qwen3.8-27B

Benchmarks

Accept length = mean accepted tokens per draft block (7-token blocks, greedy). Greedy losslessness: 5/5 prompts identical to the un-drafted target output.

Bench matrix (vLLM 0.28, GB10, single target serve)

target accept_len draft rate
Arete-27B-Uncensored 3.00 28.6%
Arete-27B (censored) 2.97 28.2%
Qwen3.8-27B 3.80 40.0%

Live testing β€” Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload)

Real-world serving with structured security content (pen-testing prompts): accept length up to 4.71 at 53.0% draft rate on structured windows, 2.6–3.9 accept length / 23–41% on general windows. The drafter's accept rate tracks how close the prompt is to its training style β€” structured technical content drafts markedly better than free-form prose.

Usage (vLLM)

vllm serve <target-model> \
  --speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}'

Training data

6,903 rows generated on-policy from Arete-27B-Uncensored (loop rows quarantined), hidden states extracted from layers 6/20/34/48/62/64 of the target. The exact training rows are published: SargeDev/arete-dflash2-trainset (97.6 MB, jsonl: id, primary_id, source, input_ids, loss_mask, text, metadata). Corpus lineage: SargeDev/jev-distill-corpus-v3.

Provenance

License

Apache-2.0 (base drafter). Arete-27B is cc-by-nc-4.0; the uncensored SargeDev build follows the same license as its base.

Downloads last month
44
Safetensors
Model size
2B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for SargeDev/Arete-Dflash2

Base model

Qwen/Qwen3.8-27B
Finetuned
(4)
this model