paper/paper.md

---
title: "Speculative Decoding on a 16 GB Consumer GPU: Acceptance, Modern Drafters, and Quantization in Gemma 4 12B and Qwen3-8B"
author:
  - Martin Ulises Millan Guerrero
date: 2026-08-08
biblio-style: IEEEtranN
natbiboptions: numbers
keywords: [speculative decoding, consumer GPU, EAGLE-3, DFlash, DSpark, quantization, break-even]
abstract: |
  Speculative decoding (SD) is a mature family of lossless inference
  acceleration techniques, yet the evidence for its modern drafters, EAGLE-3,
  DFlash, and DSpark, comes almost entirely from datacenter GPUs serving
  batched workloads. The community that runs quantized GGUF models on a single
  16 GB consumer GPU has no empirical guidance on which drafter to use, at
  which draft length, or with which quantization. We present, to our knowledge, the first
  systematic evaluation of modern drafters on a 16 GB consumer NVIDIA GPU
  (RTX 4060 Ti) with official checkpoints in llama.cpp: 26 target $\times$
  drafter configurations over 1,474 greedy-decoded prompts across nine
  benchmarks in three cognitive domains, per-position acceptance curves for 20
  configurations, and a draft-length sweep ($k \in \{5,7,10\}$, 21 runs).
  DSpark is the fastest Qwen3-8B drafter ($1.64\times$ at Q4, $1.93\times$ at
  Q8_0); EAGLE-3 is robust ($1.39\times$–$1.63\times$, acceptance nearly constant under
  quantization); a vanilla 1.7B drafter is surprisingly competitive
  ($1.41\times$); DFlash fails on Qwen ($0.63\times$, $\alpha \approx 0.009$)
  yet reaches $2.09\times$–$2.30\times$ on Gemma, where the MTP head reaches
  $2.39\times$–$2.71\times$. Confidence-scheduled gating raises acceptance
  ($0.616 \to 0.714$) but lowers throughput ($87.6 \to 80.7$ tok/s) in the
  single-user regime. Target quantization barely affects acceptance, and Q4 draft
  quantization is favorable in this tested setup (about 1 GiB of VRAM saved).
  Break-even acceptance
  $\alpha_{be}$ is 6.5–39.9% in the matched-context analysis.
  The upper end slightly overlaps the low end of the $k=2$ reference reported
  on Apple Silicon (38.0–52.8%), while all tested values are below its $k=4$
  range; the comparison is directional, and the different draft-length ranges
  and serving stacks prevent attributing the gap to hardware alone.
---

# Introduction

Large language models (LLMs) are increasingly deployed outside datacenters:
researchers, students, and hobbyists run 8–12B parameter models on a single
consumer GPU, where autoregressive generation is memory-bound and often the
bottleneck. Speculative decoding (SD)
[@leviathan2023fastinferencetransformersspeculative; @chen2023acceleratinglargelanguagemodel]
attacks this bottleneck losslessly: a cheaper draft model proposes several
future tokens, and the target verifies them in a single batched forward pass,
accepting the longest prefix that matches its own distribution. The technique
has matured rapidly: dependent drafters such as the EAGLE family
[@li2025eaglespeculativesamplingrequires; @li2024eagle2fasterinferencelanguage; @li2025eagle3scalinginferenceacceleration]
and the block-parallel diffusion drafters DFlash
[@chen2026dflashblockdiffusionflash] and DSpark
[@cheng2026dsparkconfidencescheduledspeculativedecoding] report $3\times$–$5\times$
speedups on datacenter accelerators, and official checkpoints are now served
natively by llama.cpp. Yet this evidence is generated in a world very
different from the one the local community lives in.

Most published numbers come from H100/A100-class GPUs serving batched
multi-user workloads with full-precision weights. Local practice is
different: GGUF-quantized models on a 16 GB GPU, one request at a time (batch
size 1), through llama.cpp. For this setup there is essentially no systematic
evidence; guidance circulates through blog posts and forum reports that rarely
use modern drafters and sometimes contradict each other (e.g. $1.85\times$ on
a 14B target vs. $0.95\times$ on a 7B target). SpecExec
[@svirschevski2024specexecmassivelyparallelspeculative] studied SD on consumer
devices under aggressive memory offloading, and Fonal
[@fonal2026crossfamilyspeculativedecodingpolish] measured break-even
acceptance on Apple Silicon, but no study covers the combination that defines
current local practice: a consumer NVIDIA GPU, GGUF quantization of target and
draft, modern feature-level and block-parallel drafters with official
checkpoints, and per-domain analysis of where drafting pays off.

This paper fills that gap. On an RTX 4060 Ti 16 GB we benchmark 26
target $\times$ drafter configurations over 1,474 greedy-decoded prompts from
nine benchmarks in three cognitive domains (math, code, chat), measure
per-position acceptance curves for 20 configurations, and sweep draft length
$k \in \{5, 7, 10\}$ for seven configurations to build hardware break-even
maps. Everything runs in llama.cpp at a pinned commit, deterministically
(temperature 0, fixed seed and prompt order), one request at a time, the
exact regime of a local user. We ask:

**Research questions:**

- **RQ1** — How does token acceptance $\alpha$ evolve across domains
  (math/code/chat) and draft positions?
- **RQ2** — Do modern drafters (EAGLE-3/DFlash/DSpark) transfer their
  datacenter speedups to a consumer GPU?
- **RQ2b** — Is DSpark's confidence-scheduled verification calibrated per
  domain?
- **RQ3** — How does target/draft quantization (Q4/Q5/Q8) interact with
  acceptance and speedup?
- **RQ4** — Where is the hardware break-even point $\alpha_{be}(k)$ on a
  RTX 4060 Ti, and how does it compare with Apple Silicon results?

**Contributions.** This paper contributes:

- The first systematic empirical evaluation of modern drafters (EAGLE-3,
  DFlash, DSpark with official checkpoints, plus a vanilla 1.7B drafter and an
  MTP head) on a 16 GB consumer NVIDIA GPU: 26 configurations, 1,474 prompts,
  three domains, nine benchmarks, fully deterministic decoding.
- **Acceptance dynamics (RQ1).** Suffix decay is universal and
  drafter-specific; chat is generally the lowest-acceptance domain, especially
  on Gemma, while the relative ordering of math and code varies by drafter.
  The near-zero Qwen DFlash and confidence-gated DSpark configurations are
  exceptions. Relative to the small-target study of Mahmoud
  [@mahmoud2026acceptancedynamicscognitivedomains], the successful drafters
  show a different domain pattern; Section Discussion analyzes this difference.
- **Drafter ranking on consumer NVIDIA (RQ2).** DSpark is the fastest
  Qwen3-8B drafter ($1.64\times$ at Q4, $1.93\times$ at Q8_0); EAGLE-3 is
  robust ($1.39\times$–$1.63\times$ with acceptance nearly constant across
  quantization);
  a vanilla 1.7B drafter is competitive ($1.41\times$, the
  highest acceptance among Qwen configurations, $\alpha = 0.725$); on
  Gemma-4-12B the MTP head ($2.39\times$–$2.71\times$) and DFlash
  ($2.09\times$–$2.30\times$) dominate.
- **A clear negative result (RQ2).** DFlash on Qwen3-8B fails, with acceptance
  $\approx 0.009$ and a speedup of only $0.63\times$, showing that a drafter
  whose drafts are never accepted is worse than no drafter at all.
- **Confidence gating is not free in single-user mode (RQ2b).** Raising
  DSpark's $p_\mathrm{min}$ from 0.0 to 0.6 improves acceptance
  ($0.616 \to 0.714$) but *reduces* throughput ($87.6 \to 80.7$ tok/s,
  $1.64\times \to 1.51\times$); the knob is a batched-serving feature, not a
  local speedup.
- **Quantization is SD-friendly (RQ3).** Acceptance is robust to target
  quantization (EAGLE-3: $\alpha \approx 0.43$–0.44 across Q4/Q5/Q8), while
  throughput comparisons remain conditional on the serving context. Q4 draft
  quantization is favorable in the tested Gemma setup ($\Delta\alpha = 0.010$,
  about 1 GiB of VRAM saved).
- **Hardware break-even maps (RQ4).** $\alpha_{be}$ ranges 6.5–39.9% across
  $k \in \{5,7,10\}$ in the present analysis, below the Apple-Silicon $k=4$
  range and overlapping only its low $k=2$ range
  [@fonal2026crossfamilyspeculativedecodingpolish] in a directional
  comparison. The OLS fits are near-deterministic ($R^2 \ge 0.99$
  for most configurations), but the resulting maps remain hardware-, engine-,
  context- and drafter-specific diagnostics rather than universal constants.

**Reproducibility statement.** All recorded runs are deterministic and the
artifact bundle is sufficient to reproduce the reported analysis: we pin llama.cpp
master at commit
\texttt{07132750\allowbreak{}825a4f2d\allowbreak{}27a547cd\allowbreak{}9cdde1c6\allowbreak{}f6001885}
(version 22), record SHA-256
hashes of every model in `model-hashes.json`, fix the seed and prompt order
(seed 42), and store `config.json`, `metrics.json` and raw `results.jsonl` per
run. Section Reproducibility gives three-step instructions to regenerate every
number and figure.

# Background and Related Work

## Speculative decoding

SD losslessly accelerates autoregressive decoding by drafting multiple tokens
with a cheaper model and verifying them against the target in parallel
[@leviathan2023fastinferencetransformersspeculative; @chen2023acceleratinglargelanguagemodel].
The expected speedup is $(1 + \mathbb{E}[\tau]) / (1 + c_d/c_v)$, where
$\tau$ is the number of accepted tokens and $c_d/c_v$ the draft/verify cost
ratio.

## Drafter taxonomy

We classify drafters by how the draft is produced, following the survey
taxonomy of Xia et al. [@xia2024unlockingefficiencylargelanguage] and the
2025–2026 state of the art.

**Model-free and training-free methods** derive drafts from the target's own
past without a separate model: Jacobi and n-gram lookahead
[@lookahead2024], suffix-tree decoding for repetitive agentic workloads
[@oliaro2025suffixdecodingextremespeculativedecoding], and
multi-token-prediction (MTP) heads trained into the target itself (the
Gemma-4-12B MTP head evaluated here). They add no extra weights and are the
cheapest to deploy.

**Independent draft models** share the target's tokenizer but are trained
separately: the vanilla small-LM drafter of the original SD papers
[@leviathan2023fastinferencetransformersspeculative; @chen2023acceleratinglargelanguagemodel],
distilled recurrent drafters (ReDrafter
[@cheng2024recurrentdrafterfastspeculative], Clover-2
[@xiao2024clover2accurateinferenceregressive]), and cross-attention drafters
such as Beagle [@zhong2026crossattentionspeculativedecoding]. Because the
draft is generic, acceptance is usually lower, but the drafter can be reused
across targets.

**Target-dependent (feature-level) drafters** condition on the target's
hidden states and are the current industrial standard. Medusa attaches
multiple decoding heads [@cai2024medusasimplellminference]; the EAGLE family
drafts at feature level [@li2025eaglespeculativesamplingrequires], adds
dynamic draft trees in EAGLE-2 [@li2024eagle2fasterinferencelanguage], and in
EAGLE-3 predicts tokens directly with multi-layer fusion and training-time
test (TTT), reaching up to $4.79\times$ on a 70B target
[@li2025eagle3scalinginferenceacceleration]; ParallelSpec trains parallel
drafters [@xiao2024parallelspecparalleldrafterefficient]. Open-source training
frameworks such as SpecForge
[@li2026specforgeflexibleefficientopensource] have made this family the
default in practice.

**Block-parallel and diffusion drafters** (2025–2026) emit an entire draft
block in a single forward pass. DFlash [@chen2026dflashblockdiffusionflash]
uses a block-diffusion adapter and reports more than $2.5\times$ the speedup
of EAGLE-3 on Qwen3-8B; DSpark
[@cheng2026dsparkconfidencescheduledspeculativedecoding] adds a
semi-autoregressive serial head that suppresses suffix decay and a
confidence-scheduled verification policy for batched serving, reporting +30%
accepted tokens over EAGLE-3 and 60–85% faster per-user generation on
DeepSeek-V4 [@deepseekai2026deepseekv4highlyefficientmilliontoken]. Follow-ups
explore diffusion draft trees
[@ringel2026acceleratingspeculativedecodingblock], dual diffusion drafters
[@zhang2026d2sdacceleratingspeculativedecoding], and diffusion language models
as drafters [@li2025diffuspecunlockingdiffusionlanguage].

**Self-speculative drafters** reuse the target itself through early exit
(Draft&Verify [@draftverify2024], LayerSkip
[@elhoushi2024layerskipenablingearlyexit]), trading a separate drafter for
skipped layers. **Serving stacks** add a further axis: tree-based verification
(SpecInfer [@miao2024specinferacceleratinggenerativelarge]) and
speculative-speculative decoding, which overlaps drafting with verification
(SSD/Saguaro [@kumar2026speculativespeculativedecoding]), target batched
serving and are out of scope for the single-user regime studied here.

**Cross-vocabulary and cross-family drafting** removes the shared-tokenizer
constraint: UAG (SLEM string-round-trip and TLI vocabulary-intersection)
provides lossless drafting for arbitrary draft/target pairs
[@timor2025acceleratingllminferencelossless], on-device adaptive drafters such
as OmniDraft [@ramakrishnan2025omnidraftcrossvocabularyonlineadaptive] target
mobile hardware, and Fonal [@fonal2026crossfamilyspeculativedecodingpolish]
recently evaluated UAG cross-family drafting on Apple Silicon, introducing the
hardware break-even methodology we build on.

Finally, **quantization interacts with all families**: QSpec
[@zhao2025qspecspeculativedecodingcomplementary] combines a quantized target
with an FP16 draft, ML-SpecQD
[@georganas2025mlspecqdmultilevelspeculativedecoding] quantizes the draft
itself, and systematic compatibility evaluations show that 4-bit targets
barely degrade acceptance while the draft/target cost ratio, not acceptance,
ultimately bounds the speedup [@zhang2025speculativedecodingmeetsquantization].

## Prior empirical studies

Four empirical strands are directly related to ours. **Acceptance dynamics.**
Mahmoud [@mahmoud2026acceptancedynamicscognitivedomains] is the study closest
to RQ1: a TinyLlama-1.1B draft verifies against Llama-2-7B-Chat-GPTQ with
tree sampling over code, math, logical reasoning, and chat (99,768 nodes). He
finds that task type dominates tree depth, that the entropy–acceptance
correlation is negative but weak ($\rho \in [-0.20, -0.15]$), and, perhaps
counterintuitively, that chat shows the *highest* acceptance (0.565 vs. 0.518
for math), which he attributes to the lexical predictability of the
RLHF-aligned conversational register. Our RQ1 generally shows the opposite
pattern on successful drafters (higher math/code and lower chat acceptance)
with 8–12B GGUF targets and same-family block drafters, although the failed
Qwen DFlash and gated DSpark cases are exceptions; we analyze this difference
in the Discussion.

**Drafter choice.** Decoding
Speculative Decoding [@decoding-sd2025-naacl] runs 350+ experiments on
datacenter hardware and decomposes the speedup into a throughput–accuracy
trade-off, giving principled guidance on when a given draft model pays off,
but its targets and hardware are far from the 16 GB GGUF regime; recent work
also steers pretrained drafters at inference time to raise acceptance without
additional training [@steering-drafters2026-aaai]. **Small
models and software engineering.** An EACL study shows that drafting overhead
dominates on 1–2B targets, where unit speedup requires 77–94% acceptance
[@empirical-sd-small-lm2026-eacl]; Li et al.
[@li2026empiricalstudyspeculativedecoding] evaluate SD on software-engineering
tasks (SWE-bench, Aider, Polyglot) with Llama-3.1/Qwen3-32B targets and find
that $k=5$ helps most on code ($1.37\times$ to $1.58\times$) and that
chat-trained drafters suffer distribution shift on SE workloads.

**Hardware break-even.** Fonal
[@fonal2026crossfamilyspeculativedecodingpolish] evaluated UAG cross-family
drafting with Bielik-11B on an Apple Silicon M2 Pro and introduced the
break-even acceptance methodology we adopt: an OLS fit $TPS = a + \beta\cdot
\alpha$ and the break-even point $\alpha_{be} = (TPS_{base} - a)/\beta$. His
values (38.0–52.8% at $k=2$; 77.7–90.1% at $k=4$) are the reference we compare
against in RQ4; he explicitly notes that extrapolating to other hardware
requires re-estimating $\beta$ empirically on that platform, exactly what we
do here. **Quantization.** Zhang et al.
[@zhang2025speculativedecodingmeetsquantization] evaluate SD$\times$
quantization compatibility on Llama-3-8B/70B with EAGLE-2 and find that
W4A16/W8A8 barely degrade $\tau$, with the speedup limited by the growing
draft/target cost ratio rather than by acceptance; Zhao et al.
[@zhao2025qspecspeculativedecodingcomplementary] show that a quantized target
with an FP16 draft retains high acceptance; ML-SpecQD
[@georganas2025mlspecqdmultilevelspeculativedecoding] quantizes the draft
itself. These studies use full-precision or GPTQ/W4A16 targets on datacenter
hardware; none combines GGUF quantization with the modern drafters we test.

Finally, on **consumer hardware** SpecExec
[@svirschevski2024specexecmassivelyparallelspeculative] demonstrated SD on
consumer devices with massive offloading (up to 20 tokens per cycle),
establishing that the technique survives outside the datacenter; our work
complements it by measuring the mainstream single-GPU, fully offloaded
llama.cpp setup with official modern checkpoints.

**Positioning.** To our knowledge, no study combines the elements that define
current local practice: a consumer NVIDIA GPU (16 GB), GGUF quantization of
target and draft, modern dependent and block-parallel drafters with official
checkpoints (EAGLE-3, DFlash, DSpark), multi-domain evaluation with
per-position acceptance, and hardware break-even maps. Existing work covers
domains [@mahmoud2026acceptancedynamicscognitivedomains], drafter choice
[@decoding-sd2025-naacl], small models
[@empirical-sd-small-lm2026-eacl], software-engineering tasks
[@li2026empiricalstudyspeculativedecoding], a single Apple platform
[@fonal2026crossfamilyspeculativedecodingpolish], and quantization in
isolation [@zhang2025speculativedecodingmeetsquantization; @zhao2025qspecspeculativedecodingcomplementary],
but the combination, and the resulting practical guidance for a 16 GB GPU,
remains unmeasured.

# Experimental Setup

We benchmark speculative decoding on a consumer NVIDIA GPU with two GGUF
targets, seven drafter families and 26 target $\times$ drafter configurations.
Every run is deterministic (greedy decoding, fixed seed, fixed prompt order),
so all reported differences are attributable to the drafter and quantization
choices rather than sampling noise.

## Hardware and software

All experiments ran on a desktop with an NVIDIA RTX 4060 Ti 16 GB GPU, an
Intel Core i5-12400 CPU and 31 GB of system RAM under Linux. We used llama.cpp
at master commit `07132750825a4f2d27a547cd9cdde1c6f6001885` (version 22) built
with CUDA 12.8, and served every model through a persistent `llama-server`
process with full GPU offload (`-ngl 99`, CPU threads 8). Requests were issued
one at a time (single-user, batch size 1). Each run directory stores a
`config.json` that records the commit, the exact GGUF files, their SHA-256
hashes, and the sampling parameters, plus a `metrics.json` and raw
`results.jsonl`, in `experiments/runs/<id>/`. Time-to-first-token, peak VRAM
and peak power draw were sampled with a 1 Hz `nvidia-smi` sampler; the largest
peak VRAM footprint observed (Gemma 4 12B Q8_0 + DFlash-F16) is 15.5 GiB, i.e.
within the 16 GB budget.

## Models and drafters

Targets are Gemma 4 12B (instruct) and Qwen3-8B, each quantized to GGUF
Q4_K_M, Q5_K_M and Q8_0. Drafters, all converted to GGUF, are: the EAGLE-3
speculator for Qwen3-8B (F16) [@li2025eagle3scalinginferenceacceleration]; the
DFlash block-7 drafter [@chen2026dflashblockdiffusionflash] for Qwen3-8B (F16)
and for Gemma 4 12B (F16, Q4_K_M, Q8_0); the DSpark block-7 drafter
[@cheng2026dsparkconfidencescheduledspeculativedecoding] for Qwen3-8B (F16)
with confidence threshold (p-min) $\in \{0.0, 0.2, 0.4, 0.6\}$; a vanilla
Qwen3-1.7B (Q4_K_M) as the classic small-drafter baseline; and the Gemma 4 12B
multi-token-prediction (MTP) head (Q8_0).

Two cells do not exist because the drafter is unavailable for one target in
the pinned llama.cpp build, and we report them as absent rather than omitting
them by choice: DSpark for Gemma 4 12B, whose public Gemma DSpark GGUFs use the
draft format of pull request \#25549, which master does not expose (no
`LLM_ARCH_DSPARK`); and EAGLE-3 for Gemma 4 12B, for which no GGUF conversion exists.
Conversely, DFlash and MTP are only available for Gemma. Table
[@tbl:configs] lists the 26 final configurations (15 Qwen3-8B, 11 Gemma 4
12B), including the six target-only baselines used to compute speedups.

| Family | Target quant | Drafter | Draft format | p-min | k | ctx | n |
| :--- | :--- | :--- | :--- | :---: | :---: | :---: | :---: |
| Qwen3-8B | Q4_K_M | target only | — | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | Vanilla-1.7B | Q4_K_M | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | EAGLE-3 | F16 | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | DFlash | F16 (block 7) | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | DSpark | F16 (block 7) | 0.0 | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | DSpark | F16 (block 7) | 0.2 | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | DSpark | F16 (block 7) | 0.4 | 3 | 8192 | 1474 |
| Qwen3-8B | Q4_K_M | DSpark | F16 (block 7) | 0.6 | 3 | 8192 | 1474 |
| Qwen3-8B | Q5_K_M | target only | — | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q5_K_M | EAGLE-3 | F16 | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q5_K_M | DFlash | F16 (block 7) | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q5_K_M | DSpark | F16 (block 7) | 0.0 | 3 | 8192 | 1474 |
| Qwen3-8B | Q8_0 | target only | — | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q8_0 | EAGLE-3 | F16 | — | 3 | 8192 | 1474 |
| Qwen3-8B | Q8_0 | DSpark | F16 (block 7) | 0.0 | 3 | 8192 | 1474 |
| Gemma 4 12B | Q4_K_M | target only | — | — | 3 | 16384 | 1455 |
| Gemma 4 12B | Q4_K_M | DFlash | F16 | — | 3 | 16384 | 1455 |
| Gemma 4 12B | Q4_K_M | DFlash | Q4_K_M | — | 3 | 16384 | 1455 |
| Gemma 4 12B | Q4_K_M | DFlash | Q8_0 | — | 3 | 16384 | 1455 |
| Gemma 4 12B | Q4_K_M | MTP | Q8_0 | — | 3 | 16384 | 1455 |
| Gemma 4 12B | Q5_K_M | target only | — | — | 3 | 16384 | 1449 |
| Gemma 4 12B | Q5_K_M | DFlash | F16 | — | 3 | 16384 | 1449 |
| Gemma 4 12B | Q5_K_M | MTP | Q8_0 | — | 3 | 16384 | 1449 |
| Gemma 4 12B | Q8_0 | target only | — | — | 3 | 8192 | 1452 |
| Gemma 4 12B | Q8_0 | DFlash | F16 | — | 3 | 8192 | 1452 |
| Gemma 4 12B | Q8_0 | MTP | Q8_0 | — | 3 | 8192 | 1452 |

: The 26 final configurations (15 Qwen3-8B, 11 Gemma 4 12B, including six target-only baselines). Draft length $k = 3$ in every run; p-min is DSpark's confidence threshold; ctx is the context window (8,192 for all Qwen3-8B and Gemma-Q8_0, 16,384 for Gemma-Q4/Q5); $n$ is the number of records kept after the exclusions described below. {#tbl:configs}

## Benchmarks and prompts

Nine benchmarks grouped into three cognitive domains: math (GSM8K, MATH-500,
AIME25), code (HumanEval, MBPP, LiveCodeBench) and chat (MT-Bench, Alpaca,
Arena-Hard-v2). We subsampled each benchmark with fixed seed 42 into a main
evaluation set of 1,474 fully templated prompts (math 430: GSM8K 200, MATH-500
200, AIME25 30; code 564: HumanEval 164, MBPP 200, LiveCodeBench 200; chat
480: MT-Bench 80, Alpaca 200, Arena-Hard-v2 200), and a stratified acceptance
sample of 180 prompts (60 per domain) used for the acceptance curves and the
draft-length sweep. No thinking mode is used (`--reasoning off`); the models
generate in their default instruct style.

## Decoding protocol

All generations are greedy: sampling temperature 0.0, top-$k$ 40, top-$p$
0.95, seed 42, maximum 256 output tokens, with up to two retries per prompt
and incremental JSONL resume. The main comparison uses draft length $k = 3$
with context 8,192 for all Qwen3-8B runs and for Gemma-Q8_0, and 16,384 for
Gemma-Q4/Q5 (the Gemma 4 12B instruct template is longer). The acceptance
curves and the draft-length sweep (ksweep) run the 180-prompt sample at
context 2,048 with $k = 3$ and $k \in \{5, 7, 10\}$ respectively. Because
decoding is deterministic, the curves measured on the 180-prompt subset are
identical, for those prompts, to the final runs. For the final runs, speedups
are computed against a target-only baseline of the *same quantization* over the
*same prompt set*.

## Metrics

Per-prompt throughput (tok/s) is llama-server's `predicted_per_second`. The
headline speedup is the ratio of the mean tok/s of a draft configuration to
the mean tok/s of the matching baseline (ratio of means); we also compute the
mean and median of the per-prompt speedup ratios for robustness; they agree
with the ratio of means within a few percent (largest divergence, 7%, occurs
for Gemma DFlash, whose per-prompt distribution is mildly skewed). Acceptance
$\alpha$ is the fraction of drafted tokens accepted by the target, averaged
over prompts; $\tau$ is the mean number of accepted draft tokens per prompt.
Per-position acceptance is read from the server's `acc per pos` logs of the
curves runs. We additionally record time-to-first-token (TTFT, `prompt_ms`),
peak VRAM and peak power (1 Hz `nvidia-smi` sampler). Representative TTFT
figures: Qwen3-8B mean 65–84 ms and Gemma 4 12B mean 149–212 ms across
configurations; peak power draw ranges 138–167 W.

## Data exclusions and integrity

llama-server reports spurious timing records (`tok_per_s = 1,000,000`,
`predicted_ms = 0`) on 271 records of Gemma runs; we excluded every record
with `tok_per_s >= 1e5` or `predicted_ms <= 0`; in the observed records, these
timing sentinels also had `alpha/tau/draft_n = None`. No such sentinels occur in
Qwen runs. Deterministic failures further reduce the Gemma sample: the
Gemma-Q4 runs (5 configurations) lost `humaneval-00050` to a reproducible
HTTP 500 error, leaving $n = 1473$ raw and $n = 1455$ clean records per run;
the Gemma-Q8 runs (3 configurations) lost `arena-hard-v2-00233` (over
context) and `arena-hard-v2-00239` (HTTP 500), leaving $n = 1472$ raw and
$n = 1452$ clean; the Gemma-Q5 runs kept all 1,474 raw records and lost 25
sentinels per run ($n = 1449$). In the curves/ksweep runs, two Arena-Hard-v2
prompts exceed the 2,048-token context (`-00089`, `-00164`) and were dropped,
leaving 178 of 180 prompts for Qwen3-8B (chat $n = 58$); Gemma curves/ksweep
lose three additional sentinel records per run, leaving 175 (math 60, code 60,
chat 55). Finally, two power cuts occurred during measurement; both were
repaired by resuming from the incremental JSONL (union of `results` and
`errors` records) with no data loss.

# Results

## Acceptance dynamics by domain (RQ1)

[@fig:acceptance] (Qwen3-8B-Q4, panels a–g) and [@fig:acceptance-gemma] (Gemma 4 12B-Q4, panels a–d) show per-position acceptance at draft length $k = 3$ for all 11 drafter configurations, split by domain. Three regularities are visible. First, acceptance
decays monotonically with draft position (the well-documented suffix decay
[@oliaro2025suffixdecodingextremespeculativedecoding]): e.g., EAGLE-3 on
Qwen math drops from 0.659 at position 1 to 0.438 at position 2 and 0.238 at
position 3. The decay rate is drafter-specific: Gemma MTP stays near 0.74 at
the third position on math ($0.905 \rightarrow 0.813 \rightarrow 0.737$), while
EAGLE-3 loses most of its value by position 3. Second, chat is generally the
hardest domain: after the first position, its curve lies below math and code in
every configuration; the near-zero Qwen DFlash panels are the exception at
position 1. Third, the domain gap is much larger on Gemma than on
Qwen, both absolutely and per position: for Gemma DFlash-F16 the first
position accepts 0.888 of math drafts but only 0.652 of chat drafts, and by
position 3 chat acceptance falls to 0.322 (vs. 0.635 for math); on Qwen the
same drafter family shows a smaller spread (e.g., DSpark p=0.0: 0.835 vs.
0.692 at position 1).

Table [@tbl:alpha-tau] aggregates acceptance over the full 256-token
generations of the final runs. Chat is generally the lowest-acceptance domain,
especially on Gemma; near-zero Qwen DFlash and gated DSpark are exceptions,
while the relative ordering of math and code is configuration-dependent.
Gemma shows
the sharpest penalty: chat $\alpha$ ranges 0.463–0.608 while math $\alpha$
ranges 0.770–0.837, a gap of roughly 0.2–0.3. On Qwen the gap is smaller
(EAGLE-3: 0.428 vs. 0.445; DSpark p=0.0: 0.549 vs. 0.706) and vanilla-1.7B
keeps the highest chat acceptance of the non-gated Qwen configurations (0.684;
only DSpark at $p_\mathrm{min}=0.6$ reaches 0.723). This generally reversed
"chat paradox" (for viable drafters, the workload that dominates real-world
use of local models benefits the least from drafting) differs from the domain
ordering reported by Mahmoud
[@mahmoud2026acceptancedynamicscognitivedomains], who found chat to be the
*highest*-acceptance domain with a small independent draft on a 7B target
(0.565 vs. 0.518 for math); the Discussion analyzes this difference. For the
viable drafters, measured speedups on chat are generally the smallest of the
three domains (see RQ2); the near-zero DFlash case is an unhelpful exception.
Thus, gains reported on math-heavy or code-heavy suites can overestimate what a
typical chat user will see.

| Family | Config | Math $\alpha$ ($\tau$) | Code $\alpha$ ($\tau$) | Chat $\alpha$ ($\tau$) | All $\alpha$ ($\tau$) |
| :--- | :------------- | :---: | :---: | :---: | :---: |
| Qwen3-8B | q4-vanilla17b | 0.784 (178) | 0.716 (173) | 0.684 (169) | 0.725 (173) |
| Qwen3-8B | q4-eagle3 | 0.445 (144) | 0.447 (144) | 0.428 (133) | 0.440 (141) |
| Qwen3-8B | q4-dflash | 0.009 (6) | 0.009 (7) | 0.010 (7) | 0.009 (7) |
| Qwen3-8B | q4-dspark-p0 | 0.706 (171) | 0.606 (163) | 0.549 (154) | 0.616 (162) |
| Qwen3-8B | q4-dspark-p2 | 0.707 (171) | 0.606 (163) | 0.561 (154) | 0.621 (162) |
| Qwen3-8B | q4-dspark-p4 | 0.712 (171) | 0.622 (163) | 0.616 (150) | 0.646 (161) |
| Qwen3-8B | q4-dspark-p6 | 0.743 (169) | 0.684 (157) | 0.723 (138) | 0.714 (154) |
| Qwen3-8B | q5-eagle3 | 0.446 (144) | 0.450 (144) | 0.421 (132) | 0.439 (140) |
| Qwen3-8B | q5-dflash | 0.009 (7) | 0.009 (6) | 0.009 (7) | 0.009 (7) |
| Qwen3-8B | q5-dspark-p0 | 0.699 (171) | 0.609 (163) | 0.542 (153) | 0.613 (162) |
| Qwen3-8B | q8-eagle3 | 0.441 (144) | 0.442 (144) | 0.414 (132) | 0.432 (140) |
| Qwen3-8B | q8-dspark-p0 | 0.703 (171) | 0.603 (163) | 0.538 (153) | 0.611 (162) |
| Gemma 4 12B | q4-dflash-f16 | 0.776 (175) | 0.718 (167) | 0.475 (126) | 0.657 (156) |
| Gemma 4 12B | q4-dflash-q4 | 0.770 (174) | 0.708 (166) | 0.463 (124) | 0.647 (155) |
| Gemma 4 12B | q4-dflash-q8 | 0.775 (175) | 0.718 (167) | 0.474 (126) | 0.656 (156) |
| Gemma 4 12B | q4-mtp | 0.830 (179) | 0.783 (172) | 0.603 (140) | 0.739 (164) |
| Gemma 4 12B | q5-dflash-f16 | 0.782 (176) | 0.724 (167) | 0.469 (126) | 0.659 (157) |
| Gemma 4 12B | q5-mtp | 0.837 (180) | 0.788 (172) | 0.608 (142) | 0.745 (165) |
| Gemma 4 12B | q8-dflash-f16 | 0.780 (175) | 0.727 (167) | 0.464 (124) | 0.658 (156) |
| Gemma 4 12B | q8-mtp | 0.837 (179) | 0.790 (172) | 0.607 (140) | 0.745 (164) |

: Acceptance $\alpha$ (fraction of drafted tokens accepted) and $\tau$ (mean accepted draft tokens per prompt), shown as $\alpha$ ($\tau$), by configuration and domain (final runs, $k = 3$). {#tbl:alpha-tau}

![Acceptance by draft position for the seven Qwen3-8B-Q4 drafter configurations (a)–(g), $k = 3$, ctx 2,048, 180-prompt sample. Lines per domain (math circles, code squares, chat triangles); shaded bands are 95% CIs. Suffix decay is universal; chat is generally below math/code after the first position; DFlash on Qwen starts near zero and remains a flat loss.](figures/F1a_acceptance_qwen.png){#fig:acceptance}

![Acceptance by draft position for the four Gemma 4 12B-Q4 drafter configurations (a)–(d), same protocol. The chat gap is largest for Gemma: chat falls below math/code after the first position for every drafter, and the near-zero Qwen DFlash panels are the exception to the general ordering only at position 1.](figures/F1b_acceptance_gemma.png){#fig:acceptance-gemma}

**Takeaway.** Acceptance falls off quickly the further the draft looks ahead, and
aggregate acceptance is lowest on chat, the domain users actually run on local
GPUs. Any speedup reported on math/code suites overstates what chat workloads
will deliver.

## Modern drafters on consumer hardware (RQ2)

[@tbl:speedup] reports per-domain and overall speedups (ratio of means vs.
same-quantization target-only, same prompt set) for the 20 drafter
configurations, and [@fig:speedup] plots overall speedup against overall
$\alpha$ per configuration and domain. The transfer of datacenter gains to
this 16 GB GPU is real but drafter-dependent.

**DSpark is the fastest Qwen drafter.** With $p_\mathrm{min} = 0.0$ on
Qwen3-8B-Q4 it reaches 87.6 tok/s, $1.64\times$ overall (math $1.79\times$,
code $1.63\times$, chat $1.53\times$), and $1.93\times$ overall at Q8_0 (up to
$2.11\times$ on math) at 62.3 tok/s, the largest speedups of any Qwen
configuration. Its acceptance is moderate ($\alpha \approx 0.61$–0.62) and
stable across quantizations.

**EAGLE-3 is solid and quantization-robust.** It delivers $1.39\times$,
$1.46\times$ and $1.63\times$ at Q4/Q5/Q8 (74.4, 68.3 and 52.8 tok/s) while
$\alpha$ stays essentially constant (0.440/0.439/0.432). The speedup grows as
the target gets slower because the drafter cost is paid in draft tokens, not
in acceptance quality.

**The vanilla 1.7B drafter is surprisingly competitive on Q4.** It reaches
$1.41\times$ (75.5 tok/s) with the highest acceptance among Qwen
configurations ($\alpha = 0.725$; only Gemma's MTP head is higher, at
0.739–0.745), only 14% below DSpark at a fraction of the complexity.
Its per-position decay is the gentlest observed (math: $0.873 \rightarrow 0.686$
across positions 1–3), confirming that for an 8B target a small same-family
drafter can already carry most of the value.

**DFlash on Qwen fails, an important negative result.** Despite a block-7
drafter of comparable size to EAGLE-3's, the DFlash drafts are accepted at
$\alpha \approx 0.009$, i.e. essentially never (per-position acceptance stays
0.02–0.00). The system then degenerates into running both models per step and
slows down to $0.63\times$ (33.4 tok/s vs. 53.4 baseline), at every
quantization (0.63–$0.66\times$). This is drafter- and target-specific, not a
general DFlash failure: on Gemma 4 12B the same architecture is the strongest
drafter, reaching 2.09–$2.30\times$ (DFlash) and 2.39–$2.71\times$ (MTP). We
speculate the Qwen DFlash GGUF draft is distributionally incompatible with the
target's sampling (e.g. vocabulary or positional mismatch), which future work
should audit; for practitioners, the takeaway is that a same-size diffusion
drafter on a fast target can cost more than it saves.

**Gemma's block drafters are the best absolute performers.** MTP and DFlash
reach 2.39–$2.71\times$ and 2.09–$2.30\times$ respectively (82.7 and 78.4
tok/s at Q4) because the Gemma Q4/Q5/Q8 targets are far slower (34.7/29.9/19.9
tok/s solo) and the drafts are cheap: the draft runs at a fraction of the
target cost while acceptance stays high ($\alpha$ 0.65–0.75).

[@fig:speedup] organizes these results into four clusters: Gemma block drafters
in the upper region ($\alpha$ 0.65–0.75, speedup 2.1–$2.7\times$); DSpark and
EAGLE-3 in the middle band ($\alpha$ 0.43–0.62, speedup 1.39–$1.93\times$);
vanilla-1.7B at high $\alpha$ (0.73) with moderate speedup ($1.41\times$); and
DFlash-on-Qwen isolated at $\alpha \approx 0.009$ inside the shaded loss
region. Within a drafter, quantization moves points along a line of roughly
constant $\alpha$ (EAGLE-3) or slightly decreasing $\alpha$ (DSpark), while
the domain markers spread each configuration vertically: chat is generally
the slowest, with near-zero DFlash approximately tied across domains.

| Family | Config | $\alpha$ | $\tau$ | Math x | Code x | Chat x | All x | All tok/s |
| :------------- | :------------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Qwen3-8B | q4-solo | — | — | — | — | — | — | 53.4 |
| Qwen3-8B | q4-vanilla17b | 0.725 | 173 | 1.50x | 1.40x | 1.36x | 1.41x | 75.5 |
| Qwen3-8B | q4-eagle3 | 0.440 | 141 | 1.40x | 1.41x | 1.37x | 1.39x | 74.4 |
| Qwen3-8B | q4-dflash | 0.009 | 7 | 0.63x | 0.63x | 0.63x | 0.63x | 33.4 |
| Qwen3-8B | q4-dspark-p0 | 0.616 | 162 | 1.79x | 1.63x | 1.53x | 1.64x | 87.6 |
| Qwen3-8B | q4-dspark-p2 | 0.621 | 162 | 1.78x | 1.61x | 1.51x | 1.62x | 86.7 |
| Qwen3-8B | q4-dspark-p4 | 0.646 | 161 | 1.77x | 1.59x | 1.46x | 1.60x | 85.4 |
| Qwen3-8B | q4-dspark-p6 | 0.714 | 154 | 1.73x | 1.50x | 1.33x | 1.51x | 80.7 |
| Qwen3-8B | q5-solo | — | — | — | — | — | — | 46.8 |
| Qwen3-8B | q5-eagle3 | 0.439 | 140 | 1.47x | 1.48x | 1.43x | 1.46x | 68.3 |
| Qwen3-8B | q5-dflash | 0.009 | 7 | 0.66x | 0.66x | 0.66x | 0.66x | 30.8 |
| Qwen3-8B | q5-dspark-p0 | 0.613 | 162 | 1.87x | 1.72x | 1.60x | 1.72x | 80.6 |
| Qwen3-8B | q8-solo | — | — | — | — | — | — | 32.3 |
| Qwen3-8B | q8-eagle3 | 0.432 | 140 | 1.65x | 1.65x | 1.59x | 1.63x | 52.8 |
| Qwen3-8B | q8-dspark-p0 | 0.611 | 162 | 2.11x | 1.91x | 1.78x | 1.93x | 62.3 |
| Gemma 4 12B | q4-solo | — | — | — | — | — | — | 34.7 |
| Gemma 4 12B | q4-dflash-f16 | 0.657 | 156 | 2.33x | 2.22x | 1.71x | 2.09x | 72.2 |
| Gemma 4 12B | q4-dflash-q4 | 0.647 | 155 | 2.54x | 2.41x | 1.85x | 2.26x | 78.4 |
| Gemma 4 12B | q4-dflash-q8 | 0.656 | 156 | 2.47x | 2.35x | 1.81x | 2.21x | 76.6 |
| Gemma 4 12B | q4-mtp | 0.739 | 164 | 2.59x | 2.49x | 2.09x | 2.39x | 82.7 |
| Gemma 4 12B | q5-solo | — | — | — | — | — | — | 29.9 |
| Gemma 4 12B | q5-dflash-f16 | 0.659 | 157 | 2.42x | 2.30x | 1.75x | 2.16x | 64.5 |
| Gemma 4 12B | q5-mtp | 0.745 | 165 | 2.75x | 2.64x | 2.21x | 2.53x | 75.6 |
| Gemma 4 12B | q8-solo | — | — | — | — | — | — | 19.9 |
| Gemma 4 12B | q8-dflash-f16 | 0.658 | 156 | 2.58x | 2.46x | 1.85x | 2.30x | 45.7 |
| Gemma 4 12B | q8-mtp | 0.745 | 164 | 2.94x | 2.82x | 2.36x | 2.71x | 54.0 |

: Speedup by configuration and domain (ratio of means vs. same-quantization target-only over the same prompt set) with overall acceptance $\alpha$, $\tau$ and throughput (tok/s). Baselines (solo) shown for reference. {#tbl:speedup}

![Overall speedup vs. overall acceptance per configuration and domain for (a) Qwen3-8B and (b) Gemma 4 12B (markers: math circles, code squares, chat triangles; colors: drafter family, DSpark p-min variants merged). Shaded region: speedup below 1.0 (loss). DFlash-on-Qwen sits at $\alpha \approx 0.009$ in the loss region; Gemma block drafters (DFlash, MTP) dominate the upper right; EAGLE-3 and DSpark form a mid band; vanilla-1.7B shows high $\alpha$ but moderate speedup.](figures/F2_speedup.png){#fig:speedup}

**Takeaway.** Modern drafters do transfer to a 16 GB GPU, but the winner is
the one whose draft cost is genuinely small relative to the target: DSpark on
Qwen ($1.64\times$), DFlash/MTP on Gemma (up to $2.71\times$), and even a plain
1.7B drafter ($1.41\times$). A drafter whose drafts are never accepted
(DFlash-on-Qwen, $0.63\times$) is worse than no drafter at all.

## DSpark confidence gating (RQ2b)

DSpark's confidence-scheduled verification accepts a draft block only when the
drafter's confidence exceeds a threshold $p_\mathrm{min}$; otherwise the
target generates alone. On Qwen3-8B-Q4, raising $p_\mathrm{min}$ from 0.0 to
0.6 behaves exactly as the paper describes: acceptance improves
monotonically, from $\alpha = 0.616$ (p=0.0) to 0.621 (p=0.2), 0.646 (p=0.4)
and 0.714 (p=0.6), with the largest gain in chat ($0.549 \rightarrow 0.723$).
However, throughput *decreases*: 87.6 $\rightarrow$ 86.7 $\rightarrow$ 85.4
$\rightarrow$ 80.7 tok/s, i.e. speedup falls from $1.64\times$ to $1.51\times$
(chat drops hardest: $1.53\times \rightarrow 1.33\times$).

The likely mechanism is a batch-size effect. Tighter gating turns more decode steps
into target-only steps; in batched multi-user serving those target passes are
amortized across many requests, which is the regime DSpark is designed for
[@cheng2026dsparkconfidencescheduledspeculativedecoding]. In our single-user
batch-1 setting the extra target work is paid per request, and the higher
acceptance no longer compensates: each rejected block costs a full target
forward pass that produces nothing. The net effect is a monotone tok/s loss as
$p_\mathrm{min}$ grows, even where $\alpha$ gains are largest.

**Takeaway.** On single-user consumer hardware, confidence-scheduled
verification buys acceptance but not speed, because the extra target work is not
amortized. DSpark's gating knob is a serving-time feature, not a free
single-user speedup; this contrasts directly with the multi-user results in
the DSpark paper.

## Quantization (RQ3)

[@tbl:quantization] crosses target quantization (Q4/Q5/Q8) with drafter for
both families. Two effects stand out.

**Acceptance is robust to target quantization.** EAGLE-3 keeps
$\alpha \approx 0.43$–0.44 across Q4/Q5/Q8 (0.440/0.439/0.432), DSpark stays
at 0.61–0.62 (0.616/0.613/0.611), and Gemma MTP at 0.739–0.745. Quantizing
the target degrades its own generation slightly but does not change how well
it agrees with the drafter, which agrees with the compatibility studies of
Zhang et al. [@zhang2025speculativedecodingmeetsquantization] and Zhao et al.
[@zhao2025qspecspeculativedecodingcomplementary].

**Throughput differences are context-dependent.** For Qwen, where all final
target quantizations use `ctx=8192`, the slower Q8_0 target gives the same
acceptance more time to amortize drafting: EAGLE-3 goes
$1.39\times \rightarrow 1.46\times \rightarrow 1.63\times$ and DSpark
$1.64\times \rightarrow 1.72\times \rightarrow 1.93\times$ across
Q4/Q5/Q8. Gemma Q4/Q5 use `ctx=16384`, whereas Gemma Q8 uses `ctx=8192`, so
its raw throughput and speedup differences cannot be attributed to target
quantization alone. We therefore treat those cross-quantization values as
conditional observations rather than an isolated quantization effect.

**Quantizing the draft is favorable in the tested Gemma setup.** On Gemma-Q4 with DFlash drafts
[@tbl:draft-quant], the Q4_K_M draft is the fastest (78.4 tok/s, $2.26\times$,
$\alpha$ 0.647) and the F16 draft the slowest (72.2 tok/s, $2.09\times$,
$\alpha$ 0.657), yet $\alpha$ differs by only 0.010 across draft
quantizations, so the draft's predictive distribution is preserved. The Q4
draft also saves ~1.0 GiB of VRAM (9,730 vs. 10,766 MiB peak vs. F16) and
reduces the on-disk draft from 1.47 GB (F16) to 0.44 GB (Q4_K_M) or 0.79 GB
(Q8_0). This is a favorable, configuration-specific trade-off whose
applicability beyond this setup is untested.

| Family | Target quant | Drafter | n | tok/s | x vs solo | $\alpha$ | $\tau$ |
| :--- | :---: | :--- | :---: | :---: | :---: | :---: | :---: |
| Qwen3-8B | q4 | Vanilla-1.7B | 1474 | 75.5 | 1.41x | 0.725 | 173 |
| Qwen3-8B | q4 | EAGLE-3 | 1474 | 74.4 | 1.39x | 0.440 | 141 |
| Qwen3-8B | q4 | DFlash-F16 | 1474 | 33.4 | 0.63x | 0.009 | 7 |
| Qwen3-8B | q4 | DSpark p=0.0 | 1474 | 87.6 | 1.64x | 0.616 | 162 |
| Qwen3-8B | q4 | DSpark p=0.2 | 1474 | 86.7 | 1.62x | 0.621 | 162 |
| Qwen3-8B | q4 | DSpark p=0.4 | 1474 | 85.4 | 1.60x | 0.646 | 161 |
| Qwen3-8B | q4 | DSpark p=0.6 | 1474 | 80.7 | 1.51x | 0.714 | 154 |
| Qwen3-8B | q5 | EAGLE-3 | 1474 | 68.3 | 1.46x | 0.439 | 140 |
| Qwen3-8B | q5 | DFlash-F16 | 1474 | 30.8 | 0.66x | 0.009 | 7 |
| Qwen3-8B | q5 | DSpark p=0.0 | 1474 | 80.6 | 1.72x | 0.613 | 162 |
| Qwen3-8B | q8 | EAGLE-3 | 1474 | 52.8 | 1.63x | 0.432 | 140 |
| Qwen3-8B | q8 | DSpark p=0.0 | 1474 | 62.3 | 1.93x | 0.611 | 162 |
| Gemma 4 12B | q4 | DFlash-F16 | 1455 | 72.2 | 2.09x | 0.657 | 156 |
| Gemma 4 12B | q4 | DFlash-Q4 | 1455 | 78.4 | 2.26x | 0.647 | 155 |
| Gemma 4 12B | q4 | DFlash-Q8 | 1455 | 76.6 | 2.21x | 0.656 | 156 |
| Gemma 4 12B | q4 | MTP | 1455 | 82.7 | 2.39x | 0.739 | 164 |
| Gemma 4 12B | q5 | DFlash-F16 | 1449 | 64.5 | 2.16x | 0.659 | 157 |
| Gemma 4 12B | q5 | MTP | 1449 | 75.6 | 2.53x | 0.745 | 165 |
| Gemma 4 12B | q8 | DFlash-F16 | 1452 | 45.7 | 2.30x | 0.658 | 156 |
| Gemma 4 12B | q8 | MTP | 1452 | 54.0 | 2.71x | 0.745 | 164 |

: Target quantization (Q4_K_M/Q5_K_M/Q8_0) crossed with drafter, per family: throughput (tok/s), speedup vs. same-quant solo, acceptance $\alpha$ and $\tau$ (final runs, $k = 3$). {#tbl:quantization}

| Drafter (draft quant) | n | tok/s | x vs solo | $\alpha$ | $\tau$ | max VRAM (MiB) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| DFlash-F16 | 1455 | 72.2 | 2.09x | 0.657 | 156 | 10766 |
| DFlash-Q4 | 1455 | 78.4 | 2.26x | 0.647 | 155 | 9730 |
| DFlash-Q8 | 1455 | 76.6 | 2.21x | 0.656 | 156 | 10043 |

: Effect of draft quantization on Gemma 4 12B-Q4 (DFlash drafts, final runs). F16 = 1.47 GB draft, Q4_K_M = 0.44 GB, Q8_0 = 0.79 GB (model hashes). {#tbl:draft-quant}

**Takeaway.** In the tested configurations, target quantization changes
acceptance only modestly, and a Q4 draft is both faster and ~1 GiB cheaper than
an F16 draft on Gemma-Q4. Throughput comparisons across target quantizations
must retain their context-window conditions. On this 16 GB card, Q4 target plus
Q4 draft is a practical operating point, not a universal rule.

## Break-even maps (RQ4)

Following the OLS methodology used for Apple Silicon by Fonal
[@fonal2026crossfamilyspeculativedecodingpolish] (adapted from Bielik et
al.), we fit, per configuration and draft length $k$, a least-squares line
$TPS = a + \beta \cdot \alpha$ over the per-prompt $(\alpha, \mathrm{tok/s})$
pairs of the ksweep runs, and define the break-even acceptance
$\alpha_{be} = (TPS_{base} - a)/\beta$, where $TPS_{base}$ is the mean tok/s
of a target-only baseline matched by target, context, prompt-set content, and
sampling protocol; the estimate uses only prompt IDs shared by both runs.
95% CIs come from the delta method on the OLS covariance. [@tbl:breakeven] and
[@fig:breakeven] report the pooled (all-domain) results for $k \in \{5, 7,
10\}$.

**On the RTX 4060 Ti, several tested configurations break even at low
acceptance.** The pooled $\alpha_{be}$ ranges are 10.7–39.6% at $k=5$,
11.0–39.9% at $k=7$ and 6.5–32.8% at $k=10$. For the best drafters the map
is striking: Gemma DFlash-F16 needs only $\alpha_{be} = 0.065$ at $k=10$
(0.118 at $k=5$), and Qwen EAGLE-3 ranges from 0.185 to 0.232 across the
three draft lengths. The upper end of the full RTX range overlaps only the
low end of the Apple-Silicon reference of 38.0–52.8% at $k=2$; the best
block drafters remain below that band, and all tested RTX values are below
its 77.7–90.1% range at $k=4$ [@fonal2026crossfamilyspeculativedecodingpolish].
The comparison is directional: the platforms, engines, target/drafter
combinations, and draft-length ranges differ. Within this matched ctx=2,048
protocol, the target-only baselines are slow enough that low acceptance
clears the measured drafting cost for the best configurations.

**The slope $\beta$ generally grows with $k$.** Longer drafts usually win more
tok/s per unit of acceptance (e.g. Gemma DFlash-F16: $\beta = 96.8 \rightarrow
110.3 \rightarrow 185.7$ for $k = 5/7/10$), which often lowers $\alpha_{be}$
with $k$. Qwen Q4 DFlash-F16 is the exception: its slope peaks at $k=7$ and
its pooled map stays nearly flat at 0.157–0.160; Qwen Q4 DSpark varies only
from 0.182 to 0.187.

[@fig:ksweep] shows the per-position acceptance curves behind these maps for
two Qwen3-8B-Q4 configurations: extending the draft from $k=5$ to $k=10$
adds positions with progressively lower acceptance (suffix decay), yet the
break-even gains at larger $k$ come from the slope $\beta$, not from higher
late-position acceptance.

![Per-position token acceptance for draft lengths $k \in \{5, 7, 10\}$ (ksweep, ctx 2,048, pooled over domains) for (a) EAGLE-3 and (b) DSpark p=0.0 on Qwen3-8B-Q4. Longer drafts add trailing positions with progressively lower acceptance; the break-even gain at larger $k$ comes from the slope $\beta$ rather than from higher late-position acceptance.](figures/F4_alpha_position_ksweep.png){#fig:ksweep}

The fits are near-deterministic: $R^2$ ranges 0.86–1.00 across all pooled
configurations (mostly above 0.99), so $\alpha_{be}$ is stable within this
measurement protocol rather than an artifact of noisy data. It remains a
diagnostic of the tested hardware, engine, context, and drafter combination,
not a universal hardware constant.

**Slow targets and slow drafters bracket the map.** Under the matched
ctx=2,048 baselines, Qwen Q8_0 runs at 31.8 tok/s versus 51.5 tok/s for Q4,
and EAGLE-3 has $\alpha_{be}=0.110$–$0.116$ across the sweep; a slower
target is cheaper to pay with acceptance. Conversely the vanilla 1.7B drafter
has the highest break-even of all (0.328–0.399): its flatter slope
($\beta = 89$–117) means each accepted token buys less, so it needs
substantially more acceptance to justify itself. Domain-level maps track the
pooled values within a few points (e.g. Qwen Q4 EAGLE-3 $k=7$: 0.220–0.247 across
math/code/chat), i.e. the break-even picture is domain-insensitive.

| Config | k | baseline (tok/s; n/match) | beta (slope) | alpha_be | CI95 | R2 |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| Gemma Q4 / DFlash-F16 | 10 | 33.5; 175/175 | 185.65 | 0.065 | 0.008 | 0.997 |
| Gemma Q4 / DFlash-F16 | 5 | 33.5; 175/175 | 96.78 | 0.118 | 0.012 | 0.993 |
| Gemma Q4 / DFlash-F16 | 7 | 33.5; 175/175 | 110.32 | 0.144 | 0.009 | 0.996 |
| Gemma Q4 / MTP | 10 | 33.5; 175/175 | 162.16 | 0.084 | 0.010 | 0.996 |
| Gemma Q4 / MTP | 5 | 33.5; 175/175 | 97.16 | 0.107 | 0.014 | 0.990 |
| Gemma Q4 / MTP | 7 | 33.5; 175/175 | 106.38 | 0.147 | 0.012 | 0.993 |
| Qwen Q4 / DFlash-F16 | 10 | 51.5; 178/178 | 155.12 | 0.160 | 0.060 | 0.858 |
| Qwen Q4 / DFlash-F16 | 5 | 51.5; 178/178 | 142.47 | 0.159 | 0.029 | 0.963 |
| Qwen Q4 / DFlash-F16 | 7 | 51.5; 178/178 | 157.21 | 0.157 | 0.024 | 0.974 |
| Qwen Q4 / DSpark p=0 | 10 | 51.5; 178/178 | 152.84 | 0.183 | 0.004 | 0.999 |
| Qwen Q4 / DSpark p=0 | 5 | 51.5; 178/178 | 126.65 | 0.187 | 0.012 | 0.992 |
| Qwen Q4 / DSpark p=0 | 7 | 51.5; 178/178 | 150.92 | 0.182 | 0.009 | 0.996 |
| Qwen Q4 / EAGLE-3 | 10 | 51.5; 178/178 | 179.97 | 0.185 | 0.004 | 0.999 |
| Qwen Q4 / EAGLE-3 | 5 | 51.5; 178/178 | 124.01 | 0.212 | 0.003 | 0.999 |
| Qwen Q4 / EAGLE-3 | 7 | 51.5; 178/178 | 136.39 | 0.232 | 0.002 | 1.000 |
| Qwen Q4 / Vanilla-1.7B | 10 | 51.5; 178/178 | 116.94 | 0.328 | 0.008 | 0.996 |
| Qwen Q4 / Vanilla-1.7B | 5 | 51.5; 178/178 | 89.13 | 0.396 | 0.017 | 0.978 |
| Qwen Q4 / Vanilla-1.7B | 7 | 51.5; 178/178 | 93.02 | 0.399 | 0.007 | 0.997 |
| Qwen Q8 / EAGLE-3 | 10 | 31.8; 178/178 | 149.50 | 0.111 | 0.004 | 0.999 |
| Qwen Q8 / EAGLE-3 | 5 | 31.8; 178/178 | 99.71 | 0.116 | 0.004 | 0.999 |
| Qwen Q8 / EAGLE-3 | 7 | 31.8; 178/178 | 124.64 | 0.110 | 0.004 | 0.999 |

: Break-even acceptance $\alpha_{be}$ per configuration and draft length $k$ (ksweep runs, ctx 2,048, pooled over domains). All baselines use ctx=2,048; the baseline is matched by target, prompt-set content, and sampling protocol. `n/match` reports the baseline records retained and shared with the ksweep. OLS fit $TPS = a + \beta \alpha$; CI95 by delta method. $\alpha_{be} > 1.00$ means no achievable break-even. Reference on Apple Silicon: $k=2$: 38.0–52.8%, $k=4$: 77.7–90.1% [@fonal2026crossfamilyspeculativedecodingpolish]. {#tbl:breakeven}

![Break-even acceptance $\alpha_{be}$ vs. draft length $k$ on RTX 4060 Ti 16 GB (ksweep, pooled over domains; error bars: 95% CI). All RTX baselines use ctx=2,048 and a matched prompt set/protocol. Gold band: M2 Pro compact reference range 40–77% (k=2–4) of Fonal [@fonal2026crossfamilyspeculativedecodingpolish]; dashed line: $\alpha_{be} = 1.0$ (no break-even). The RTX curves sit just below the lower edge of this compact reference band; the upper end slightly overlaps the exact $k=2$ reference range, and the cross-platform comparison is directional.](figures/F3_breakeven.png){#fig:breakeven}

**Takeaway.** On a 16 GB consumer GPU, the tested configurations break even at
acceptance rates as low as 6.5%, with a pooled range of 6.5–39.9% across the
sweep. The upper end of this range slightly overlaps the low end of the
38–90% reported on Apple Silicon, while all tested values are below its
$k=4$ range; that cross-platform comparison is directional rather than
controlled. The slower
the target and the longer the draft, the cheaper it is to pay with acceptance;
only a slow drafter like vanilla-1.7B demands substantial acceptance
(33–40%) to justify itself.

# Discussion

## Why datacenter numbers do not transfer

The gap between published datacenter speedups and what we measure has a
structural cause, visible in the speedup formula
$(1 + \mathbb{E}[\tau]) / (1 + c_d/c_v)$: every result is a race between the
tokens a drafter gets accepted ($\mathbb{E}[\tau]$) and the relative cost of
drafting ($c_d/c_v$). Datacenter serving changes the second term, not the
first. DSpark is the clearest case. Its confidence-scheduled verification is
designed for batched multi-user serving, where the target-only steps
introduced by rejection are amortized across many concurrent requests; that is
the regime in which it reports 60–85% faster per-user generation on
DeepSeek-V4 [@cheng2026dsparkconfidencescheduledspeculativedecoding]. In our
batch-1 setting every rejected block costs a full target forward pass paid by
a single request, so raising $p_\mathrm{min}$ raises acceptance ($0.616 \to
0.714$) while *lowering* throughput ($87.6 \to 80.7$ tok/s). The knob is
correctly calibrated, doing exactly what the paper says, but its benefit
only materializes when target work can be shared across users.

DFlash on Qwen is the second case, and it isolates the cost term. The
block-diffusion drafter emits a whole block per forward pass; that cost is
recovered only if the block is accepted. On Gemma-4-12B, where acceptance is
0.65–0.75, DFlash is among the best drafters we measured
($2.09\times$–$2.30\times$); on Qwen3-8B, where acceptance collapses to
$\alpha \approx 0.009$, the same architecture degenerates into running both
models every step and loses to the baseline ($0.63\times$). The DFlash paper
reports more than
$2.5\times$ the speedup of EAGLE-3 on Qwen3-8B; our measurement shows that
gain is not intrinsic to the architecture: it depends on the draft being
distributionally compatible with the target as served (with our GGUF
conversion and greedy sampler it is not), and on the target being slow enough
to absorb the draft cost. This is a concrete instance of the limitation the
DSpark authors themselves flag: the draft-side cost of a full block is
unrecoverable on intrinsically low-acceptance queries.

A third, subtler consequence of the regime difference concerns the domain
ordering itself. Mahmoud [@mahmoud2026acceptancedynamicscognitivedomains]
found chat to be the highest-acceptance domain with a TinyLlama-1.1B draft on
a 7B target, attributing the effect to the lexical predictability of the
RLHF-aligned register. We instead generally find lower chat acceptance on
these 8–12B GGUF targets, especially for Gemma and viable drafters; near-zero
Qwen DFlash and gated DSpark are exceptions, while the relative ordering of
math and code changes slightly across configurations. Two non-exclusive
explanations are
plausible, but this experiment does not identify their separate causal
contributions. First, our drafters are target-dependent: conditioning on the
target's hidden states may favor structured, low-entropy continuations of math
and code, while open-ended chat remains hard to predict. Second, our targets
are larger, quantized, and decoded greedily, which may change the relative
advantage of a same-family draft. These are hypotheses rather than causal
tests. The two studies are therefore best treated as complementary: observed
acceptance is a property of the domain together with the drafter, target, and
serving regime, which is why per-family measurements matter.

## Lessons for the local community

Within the limits of a single GPU and engine, our results translate into
concrete recommendations for a 16 GB NVIDIA card:

- **Qwen3-8B at Q4**: use DSpark with $p_\mathrm{min} = 0.0$ ($1.64\times$,
  87.6 tok/s) or, if a simpler setup is preferred, the vanilla 1.7B drafter
  ($1.41\times$, 75.5 tok/s, highest Qwen acceptance). EAGLE-3 ($1.39\times$)
  is a solid alternative with a smaller drafter. Avoid DFlash on Qwen:
  $0.63\times$ is a measured loss in this setup.
- **Gemma-4-12B**: use the MTP head ($2.39\times$–$2.71\times$) or DFlash with
  a Q4_K_M draft ($2.26\times$, 78.4 tok/s) (the quantized draft is both faster
  and about 1 GiB cheaper in VRAM than F16).
- **Draft length**: longer drafts pay on slow targets. $\beta$ generally grows
  with $k$, so $\alpha_{be}$ often falls from $k=5$ to $k=10$. Qwen Q4
  DFlash-F16 is the exception, while Qwen Q4 DSpark changes only slightly.
  On Gemma, $k=10$ requires the least acceptance (0.065–0.084); on Qwen the
  difference is modest.
- **Quantization**: target Q4_K_M plus a Q4 draft is a practical operating point
  in this setup on 16 GB; acceptance barely moves with target quantization, so
  the throughput and memory trade-off is the reason to prefer Q8_0 only when its extra target
  precision is needed; this study does not measure task accuracy.
- **Domains**: chat is generally the least profitable domain, especially on
  Gemma; the near-zero Qwen DFlash case is approximately tied across domains.
  Users whose workloads are math- or code-heavy will generally see the
  headline speedups, while chat-heavy users will see the smallest ones on
  viable drafters.

The general lesson is to measure acceptance before trusting a drafter. The
break-even maps provide the yardstick: if a drafter's $\alpha$ on your
workload is above the $\alpha_{be}$ for your target and $k$, drafting pays.

## What the break-even maps mean

The most consequential result is how generous this hardware is for several
tested configurations. Pooled $\alpha_{be}$ ranges 6.5–39.9% across
$k \in \{5,7,10\}$, and the best drafters break even at single-digit
acceptance (Gemma DFlash-F16: 0.065 at $k=10$; MTP: 0.084). The Apple-Silicon
reference (38.0–52.8% at $k=2$ and 77.7–90.1% at $k=4$
[@fonal2026crossfamilyspeculativedecodingpolish]) is higher at $k=4$ and
overlaps the upper end of our range only at $k=2$. The comparison is
directional: its sweep starts at $k=2$–4, ours
at $k=5$, and the platforms, engines, and model families differ. The mechanism
is visible in the OLS parameters. In the matched ctx=2,048 analysis, the
target baselines are slow and memory-bound on this card, 33.5 tok/s
(Gemma-Q4), 51.5 tok/s (Qwen-Q4), and 31.8 tok/s (Qwen-Q8), while the slope
$\beta$ (tok/s gained per unit of acceptance) is high, 89–186 across
configurations, because a draft forward pass costs a small fraction of the
target's. With $TPS_{base}$ small and $\beta$ large, even modest acceptance
clears the intercept. On the M2 Pro with MLX-LM the
drafters are comparatively more expensive relative to a faster target, so
$\alpha_{be}$ rises. Two consequences follow. First, $\alpha_{be}$ is nearly
domain-insensitive: within this setup, the maps provide a
hardware/engine/context diagnostic, while the achieved acceptance, rather
than the fitted threshold, varies by domain. Second, the variance across
drafters is driven by draft cost: the vanilla 1.7B drafter, with a flatter
slope ($\beta \approx 89$–117), needs 0.328–0.399 acceptance to justify
itself, several times above the best block drafters, because each accepted
token buys less. Draft cost, not raw acceptance, is the lever a practitioner
should optimize.

## Limits of interpretation

Three caveats bound these conclusions. First, the headline speedup is the
ratio of means; the mean and median of the per-prompt ratios agree with it
within a few percent, with the largest divergence (7%) on Gemma DFlash, whose
per-prompt distribution is mildly skewed; point estimates are therefore stable, but
single-prompt speedups can vary widely. Second, the acceptance numbers rest on
records we had to clean: 271 Gemma records with spurious timing sentinels were
excluded, and a handful of deterministic failures reduced the Gemma sample
(see Section Limitations); acceptance at draft position comes from the
2,048-context curves runs, which are deterministic and therefore identical for
those prompts to the final runs, but the absolute tok/s values of the
curves/ksweep runs are not directly comparable with the 8,192/16,384-context
final runs. Third, the whole study is one GPU, one engine, one seed: the
break-even methodology transfers, the specific numbers require re-estimation
on other hardware, exactly as Fonal
[@fonal2026crossfamilyspeculativedecodingpolish] predicted for his own.

# Limitations

- **Single GPU model.** All measurements come from one RTX 4060 Ti 16 GB. The
  break-even maps are hardware-, engine- and context-specific diagnostics for
  this card and serving setup; as Fonal
  [@fonal2026crossfamilyspeculativedecodingpolish] notes, extrapolating to
  other GPUs requires re-estimating $\beta$ empirically on that platform.
- **Subsampled datasets.** We evaluate subsets of nine benchmarks (1,474
  main / 180 acceptance-curve prompts) with a fixed seed 42, not the full
  sets; domain-level aggregates are therefore indicative rather than
  benchmark-complete.
- **One engine, one decoding regime.** Everything runs through llama.cpp's
  `llama-server` with non-thinking mode (`--reasoning off`) and greedy
  decoding; results may differ under sampling, thinking mode, or other
  serving stacks (e.g. vLLM).
- **GGUF-specific quantization.** We quantize with Q4_K_M/Q5_K_M/Q8_0 in
  GGUF; other schemes (GPTQ, AWQ, W4A16) and full precision may interact with
  the drafters differently.
- **Unavailable drafters.** Two planned cells do not exist and are
  reported as absent, not omitted: DSpark for Gemma-4-12B, whose public Gemma
  DSpark GGUFs use the draft format of pull request \#25549, which llama.cpp
  master does not expose (no `LLM_ARCH_DSPARK`); and EAGLE-3 for Gemma-4-12B,
  for which no GGUF conversion exists.
- **DFlash-on-Qwen cause not isolated.** The negative result ($\alpha \approx
  0.009$ with the official block-7 checkpoint converted to GGUF) is measured
  but not explained: we did not determine whether the failure is a conversion
  artifact, a vocabulary or positional mismatch, or a genuine model defect.
  Auditing it against the original PyTorch checkpoint is future work.
- **Context-window differences.** Final runs use 8,192 tokens (all Qwen3-8B
  and Gemma-Q8_0) or 16,384 (Gemma-Q4/Q5); the curves and ksweep runs use
  2,048. Because decoding is deterministic, acceptance is identical for the
  overlapping prompts, but absolute throughput values are not comparable
  across contexts, and the per-position curves do not probe long-context
  behavior.
- **Data exclusions.** For the record: the only over-context discards in the
  curves/ksweep runs are two Arena-Hard-v2 prompts
  (`arena-hard-v2-00089`, `-00164`, both exceeding the 2,048-token context),
  leaving 178 of 180 prompts for Qwen and 175 for Gemma. In the final runs,
  deterministic failures reduced the Gemma sample: `humaneval-00050` (a
  reproducible HTTP 500) left $n=1{,}473$ raw and $n=1{,}455$ clean records
  for Gemma-Q4; `arena-hard-v2-00233` (over context) and `-00239` (HTTP 500)
  left $n=1{,}472$ raw and $n=1{,}452$ clean for Gemma-Q8; and 271 Gemma
  records with spurious timing sentinels (`tok_per_s \ge 1e5` or
  `predicted_ms \le 0`) were excluded from all computations (Gemma-Q5: 1,474
  raw $\to$ 1,449 clean). The matched Gemma-Q4 ctx=2,048 target-only baseline
  used for the break-even fits retained 175 of 178 records after excluding
  three additional timing sentinels; the two over-context prompts were stored
  in its error log and were not retried. No Qwen records required exclusion.

# Conclusion

This paper reports, to our knowledge, the first systematic evaluation of modern speculative-decoding
drafters on a 16 GB consumer NVIDIA GPU. Over 26 configurations and 1,474
prompts across three domains, DSpark is the strongest Qwen3-8B drafter
($1.64\times$ at Q4, $1.93\times$ at Q8_0), EAGLE-3 transfers robustly
($1.39\times$–$1.63\times$ with acceptance nearly constant across
quantization), and even
a plain 1.7B drafter is competitive ($1.41\times$); on Gemma-4-12B the MTP
head ($2.39\times$–$2.71\times$) and DFlash ($2.09\times$–$2.30\times$)
dominate. We document two negative results with direct practical value: DFlash
on Qwen fails outright ($0.63\times$, $\alpha \approx 0.009$), and DSpark's
confidence gating, although it raises acceptance, reduces single-user
throughput, so it is a serving feature rather than a free local speedup.
Quantizing the target barely moves acceptance, and Q4 draft quantization is
favorable in the tested setup. The matched-context break-even acceptance is
6.5–39.9% in the tested configurations. The upper end slightly overlaps the
low end of the reported Apple-Silicon $k=2$ reference, while all tested values
are below its $k=4$ range; that cross-platform comparison is directional rather
than a controlled estimate of a hardware effect.

Future work proceeds along several lines: cross-family drafting with UAG on
NVIDIA (deferred to a separate experiment); DSpark
difficulty-aware early exit, the direction its authors suggest for making
block drafts profitable on low-acceptance queries; domain fine-tuning of
drafters, explicitly out of scope for this study;
thinking-mode targets, which may change acceptance dynamics substantially;
additional GPUs and platforms to re-estimate the break-even maps; a systematic
study of quantized drafts across drafter families; and larger target models as
VRAM budgets grow.

# Reproducibility

The companion artifact bundle contains everything needed to regenerate
every table, figure, and number in this paper. Public release of that bundle
is planned together with the publication of the manuscript.

- **Software.** llama.cpp at master commit
  `07132750825a4f2d27a547cd9cdde1c6f6001885` (version 22), built with CUDA
  12.8; Python tooling managed with `uv` (`pyproject.toml`/`uv.lock`).
- **Models.** `experiments/runs/model-hashes.json` records the SHA-256 hash of
  every GGUF used (targets, drafts, and quantizations). Models are the
  official DeepSpec checkpoints converted to GGUF and are re-downloadable;
  `scripts/download_f3.sh` lists the exact files.
- **Prompts.** `experiments/prompts/f1-sample.jsonl` (1,474 fully templated
  prompts, seed 42) for the final runs and a 180-prompt stratified
  acceptance sample (60 per domain) for the curves and ksweep runs.
- **Scripts.** `scripts/bench_accept.py` is the llama-server runner (greedy
  T=0, fixed seed, per-prompt JSONL, incremental resume, retries, and
  $\alpha$/$\tau$ extraction); `scripts/analyze_final.py` regenerates all
  tables and figures;
  `scripts/run_final_chain.sh`, `scripts/run_curves_chain.sh`, and
  `scripts/run_ksweep_chain.sh` orchestrate the three measurement campaigns.
- **Raw data.** Each run lives in `experiments/runs/<id>/` and stores
  `config.json` (llama.cpp commit, GGUFs, SHA-256 hashes, sampling
  parameters), `metrics.json`, the raw `results.jsonl` (per-prompt tok/s,
  acceptance, timings), `vram.json`, and the `server*.log` files (including
  the per-position `acc per pos` lines used for the curves).

Reproduction, in three steps:

1. **Build and verify.** Set up the toolchain (`uv sync`), build llama.cpp at
   the pinned commit, download the GGUFs listed in `model-hashes.json`, and
   verify each file's SHA-256.
2. **Measure.** Run `scripts/run_final_chain.sh` (26 configurations),
   `scripts/run_curves_chain.sh` (20 configurations, per-position acceptance),
   and `scripts/run_ksweep_chain.sh` (21 runs, $k \in \{5,7,10\}$). Each
   script is idempotent and resumable.
3. **Analyze.** Run `uv run python scripts/analyze_final.py` to regenerate
   the analysis tables and the figures in `manuscript/figures/`.

# References