Whittle 35B-A3B DFlash2

A focused draft for Whittle.

526M parameters  ·  F16 GGUF  ·  v1.2.2  ·  Apache-2.0

Download GGUF · Training data · Release notes


Experimental community release. This drafter is intended for research and evaluation of speculative decoding with the Whittle Q5_K_M target.

Latest training checkpoint — November 16, 2026. v1.2.1 packages the best saved checkpoint from the latest KL-distillation run: step 52, selected by validation correct-prefix length, then token accuracy and selector loss. Training was stopped at step 660; the selected best weights are preserved.

A DFlash2 drafting adapter for speculative decoding with Whittle-Qwen-3.8-35B-A3B, adapted to its Q5_K_M GGUF. Run it alongside the separate Whittle target in a compatible DFlash2 runtime. The target stays frozen; the latest stage updates the feature projection, normalization, last two draft layers and candidate selector.

At a glance

Specification
Release v1.2.2, documentation update for the step 52 checkpoint
Draft 526,251,520 parameters · 6 layers · 16-token architecture block
Initialization v1.2.0 followed by selector adaptation; original ancestry: Ornith 1.5 DFlash2
Target Whittle Q5_K_M GGUF
Download F16 GGUF, about 0.991 GiB · Safetensors
License Apache-2.0, with inherited notices in NOTICE

What's improved

Recorded offline metrics for v1.0.0 and the checkpoint first published in v1.2.1.

Metric v1.0.0 v1.2.1, step 52 v1.0.0 vs Last
Validation loss ↓ 4.2199 2.9266 −30.65%
Draft-token accuracy ↑ 46.90% 59.64% +12.73 percentage points
Correct-prefix proxy ↑ 2.3965 tokens 1.6641 tokens −30.56%
Gold-token recall in top 16 ↑ 71.23% 83.59% +12.36 percentage points

The latest run processed 2,640 training contexts across 42 completed shards, reaching step 660. The selected checkpoint at step 52 had processed 208 contexts. Selection prioritizes the self-conditioned correct prefix, then accuracy and lower selector loss; it does not simply choose the last step or lowest loss. Full numeric history: EVALUATION.json and training/validation-history.jsonl.

These are offline draft-training metrics.

Training data and recipe

This training stage uses DFlash DeepSeek TraceMix, with a fresh selection of RU/EN project-code, text, reasoning and agent/tool-use contexts. The planned pool contains 16,384 contexts, balanced equally between Russian and English; 2,640 reached the optimizer before the stop. Earlier adaptation used DFlash ChatGPT TraceMix Lite and prompts selected from Tulu 3 SFT Mixture.

Whittle Q5 supplied the training completions, hidden features and full-vocabulary probability targets. DeepSeek Flash acted as a training mentor, reviewing aggregate metrics and advising continuation and learning-rate changes.

The latest stage uses forward KL distillation over all 248,320 vocabulary entries, plus selector cross-entropy. 282,898,944 of 526,251,520 draft parameters are updated; the first four draft layers remain frozen. Training uses AdamW, batch 4, a 384-token feature context, eight anchors per context, FP32 master weights and FP16 autocast. The selected step used LR 2e-6 with a 32-step warmup. See TRAINING.md, DATA_SOURCES.md and PROVENANCE.json.

Next training stage

The latest training run is stopped, with the best export and full local resume state retained. Further training can continue from the saved state when requested; no additional stage has been started for this release.

Files and use

  • Whittle-35B-A3B-DFlash2-v1.2.1-F16.gguf — inference draft, with author, version, license and checkpoint step embedded in GGUF metadata.
  • tensors/best-model.safetensors — the same 96 trained tensors with their original F16/F32 dtypes, in PyTorch module naming, plus tensor inventory. No optimizer state or target weights.
  • training/v1.2.1/ — portable KL trainer, AMP retry helper and recipe; training/validation-history.jsonl records the training metrics.

Use the compatible Linux/NVIDIA DFlash2 runtime and a separate Whittle Q5 target. USAGE.md lists the launch parameters and download verification. File and weight checksums are listed in SHA256SUMS.

Previous release measurements and limitations

The v1.2.0 first-stage release selected continuation step 9,074 with the following offline metrics: validation loss 3.8691, draft-token accuracy 50.39%, correct-prefix proxy 2.6211 and top-16 recall 73.83%. Stage one covered 36,296 usable traces across 142 of 176 first-pass shards; the full epoch was not completed. Archived evaluation.

Anonymization of publication text does not imply unlearning of earlier training data; see PRIVACY.md.

main contains v1.2.2. The previous release card is available on the v1.2.1 branch. The v1.2.0 branch, v1.1.0 branch and v1.0.0 branch remain available.


Adaptation by Egor Prokhorov · Published by biscet · License & notices

Downloads last month
191
GGUF
Model size
0.5B params
Architecture
dflash
Hardware compatibility
Log In to add your hardware

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for biscet/Whittle-35B-A3B-DFlash2

Datasets used to train biscet/Whittle-35B-A3B-DFlash2

Collection including biscet/Whittle-35B-A3B-DFlash2