Rahim/Ternary-Bonsai-27B-dspark

Ternary-Bonsai-27B DSpark drafter (safetensors, bf16)

The DSpark speculative-decoding drafter for prism-ml/Ternary-Bonsai-27B-mlx-2bit, converted to a standard safetensors checkpoint in the DeepSpec standalone layout.

PrismML publishes this drafter GGUF-only (Ternary-Bonsai-27B-dspark-bf16.gguf — no safetensors export exists upstream). This repo is a 1:1, lossless repack of that file: tensor names remapped from llama.cpp/GGUF names back to the original HF Qwen3DSparkModel export layout (the exact inverse of PrismML's own conversion/dspark.py), all weights kept at their source precision (BF16), plus a synthesized DeepSpec-style config.json from the GGUF metadata. No weights were modified, quantized, fine-tuned, or otherwise altered.

What it is

A 3.6B-parameter EAGLE-style block-diffusion drafter (DeepSpec DSpark lineage), trained by PrismML against the ternary Bonsai-27B target:

• 6 Qwen3-style decoder layers (hidden 5120, 40 Q / 4 KV heads, head_dim 128, q/k-norms)
• block_size 4, mask_token_id 248319, target layer taps [1, 16, 31, 46, 61] (of 64)
• rank-256 Markov head + confidence head
• GIDD log-SNR conditioning (min −9 / max +9) — an addition over DeepSpec's original DSpark

Drafters are target-specific: this one only accelerates the ternary Bonsai-27B. The 1-bit variant has its own matched drafter.

Use it (mlx-dspark)

Speculative decoding for Bonsai-27B on Apple Silicon — output is lossless (byte-identical to plain greedy decoding of the target; temperature sampling is an exact sample from the target):

pip install mlx-dspark   # needs mlx >= 0.32.0
mlx-dspark generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
  --max-draft auto --prompt "Implement binary search in Python."

mlx-dspark resolves this drafter automatically for the Bonsai targets; --max-draft auto calibrates the speculation depth to your machine and content (measured on an M4 Pro: ≈1.1–1.2× on code/structured output; it parks itself on content where a 2-bit target's fast plain step can't be beaten). The drafter is quantized to 4-bit at load (≈2 GB resident; drafter precision does not affect acceptance).

Attribution & license

All weights were trained and released by PrismML under Apache 2.0 as part of the Bonsai 27B family (ternary/1-bit rebuilds of Qwen3.6-27B). This repo only changes the container format. Conversion: gguf_convert.py.