paper/paper.md

metadata

title: >-
  Speculative Decoding on a 16 GB Consumer GPU: Acceptance, Modern Drafters, and
  Quantization in Gemma 4 12B and Qwen3-8B
author:
  - Martin Ulises Millan Guerrero
date: 2026-08-08T00:00:00.000Z
biblio-style: IEEEtranN
natbiboptions: numbers
keywords:
  - speculative decoding
  - consumer GPU
  - EAGLE-3
  - DFlash
  - DSpark
  - quantization
  - break-even
abstract: >
  Speculative decoding (SD) is a mature family of lossless inference

  acceleration techniques, yet the evidence for its modern drafters, EAGLE-3,

  DFlash, and DSpark, comes almost entirely from datacenter GPUs serving

  batched workloads. The community that runs quantized GGUF models on a single

  16 GB consumer GPU has no empirical guidance on which drafter to use, at

  which draft length, or with which quantization. We present, to our knowledge,
  the first

  systematic evaluation of modern drafters on a 16 GB consumer NVIDIA GPU

  (RTX 4060 Ti) with official checkpoints in llama.cpp: 26 target $\times$

  drafter configurations over 1,474 greedy-decoded prompts across nine

  benchmarks in three cognitive domains, per-position acceptance curves for 20

  configurations, and a draft-length sweep ($k \in \{5,7,10\}$, 21 runs).

  DSpark is the fastest Qwen3-8B drafter ($1.64\times$ at Q4, $1.93\times$ at

  Q8_0); EAGLE-3 is robust ($1.39\times$–$1.63\times$, acceptance nearly
  constant under

  quantization); a vanilla 1.7B drafter is surprisingly competitive

  ($1.41\times$); DFlash fails on Qwen ($0.63\times$, $\alpha \approx 0.009$)

  yet reaches $2.09\times$–$2.30\times$ on Gemma, where the MTP head reaches

  $2.39\times$–$2.71\times$. Confidence-scheduled gating raises acceptance

  ($0.616 \to 0.714$) but lowers throughput ($87.6 \to 80.7$ tok/s) in the

  single-user regime. Target quantization barely affects acceptance, and Q4
  draft

  quantization is favorable in this tested setup (about 1 GiB of VRAM saved).

  Break-even acceptance

  $\alpha_{be}$ is 6.5–39.9% in the matched-context analysis.

  The upper end slightly overlaps the low end of the $k=2$ reference reported

  on Apple Silicon (38.0–52.8%), while all tested values are below its $k=4$

  range; the comparison is directional, and the different draft-length ranges

  and serving stacks prevent attributing the gap to hardware alone.

Introduction

Large language models (LLMs) are increasingly deployed outside datacenters: researchers, students, and hobbyists run 8–12B parameter models on a single consumer GPU, where autoregressive generation is memory-bound and often the bottleneck. Speculative decoding (SD) [@leviathan2023fastinferencetransformersspeculative; @chen2023acceleratinglargelanguagemodel] attacks this bottleneck losslessly: a cheaper draft model proposes several future tokens, and the target verifies them in a single batched forward pass, accepting the longest prefix that matches its own distribution. The technique has matured rapidly: dependent drafters such as the EAGLE family [@li2025eaglespeculativesamplingrequires; @li2024eagle2fasterinferencelanguage; @li2025eagle3scalinginferenceacceleration] and the block-parallel diffusion drafters DFlash [@chen2026dflashblockdiffusionflash] and DSpark [@cheng2026dsparkconfidencescheduledspeculativedecoding] report $3\times$–$5\times$ speedups on datacenter accelerators, and official checkpoints are now served natively by llama.cpp. Yet this evidence is generated in a world very different from the one the local community lives in.

Most published numbers come from H100/A100-class GPUs serving batched multi-user workloads with full-precision weights. Local practice is different: GGUF-quantized models on a 16 GB GPU, one request at a time (batch size 1), through llama.cpp. For this setup there is essentially no systematic evidence; guidance circulates through blog posts and forum reports that rarely use modern drafters and sometimes contradict each other (e.g. $1.85\times$ on a 14B target vs. $0.95\times$ on a 7B target). SpecExec [@svirschevski2024specexecmassivelyparallelspeculative] studied SD on consumer devices under aggressive memory offloading, and Fonal [@fonal2026crossfamilyspeculativedecodingpolish] measured break-even acceptance on Apple Silicon, but no study covers the combination that defines current local practice: a consumer NVIDIA GPU, GGUF quantization of target and draft, modern feature-level and block-parallel drafters with official checkpoints, and per-domain analysis of where drafting pays off.

This paper fills that gap. On an RTX 4060 Ti 16 GB we benchmark 26 target $\times$ drafter configurations over 1,474 greedy-decoded prompts from nine benchmarks in three cognitive domains (math, code, chat), measure per-position acceptance curves for 20 configurations, and sweep draft length $k \in {5, 7, 10}$ for seven configurations to build hardware break-even maps. Everything runs in llama.cpp at a pinned commit, deterministically (temperature 0, fixed seed and prompt order), one request at a time, the exact regime of a local user. We ask:

Research questions:

• RQ1 — How does token acceptance $\alpha$ evolve across domains (math/code/chat) and draft positions?
• RQ2 — Do modern drafters (EAGLE-3/DFlash/DSpark) transfer their datacenter speedups to a consumer GPU?
• RQ2b — Is DSpark's confidence-scheduled verification calibrated per domain?
• RQ3 — How does target/draft quantization (Q4/Q5/Q8) interact with acceptance and speedup?
• RQ4 — Where is the hardware break-even point $\alpha_{be}(k)$ on a RTX 4060 Ti, and how does it compare with Apple Silicon results?

Contributions. This paper contributes:

• The first systematic empirical evaluation of modern drafters (EAGLE-3, DFlash, DSpark with official checkpoints, plus a vanilla 1.7B drafter and an MTP head) on a 16 GB consumer NVIDIA GPU: 26 configurations, 1,474 prompts, three domains, nine benchmarks, fully deterministic decoding.
• Acceptance dynamics (RQ1). Suffix decay is universal and drafter-specific; chat is generally the lowest-acceptance domain, especially on Gemma, while the relative ordering of math and code varies by drafter. The near-zero Qwen DFlash and confidence-gated DSpark configurations are exceptions. Relative to the small-target study of Mahmoud [@mahmoud2026acceptancedynamicscognitivedomains], the successful drafters show a different domain pattern; Section Discussion analyzes this difference.
• Drafter ranking on consumer NVIDIA (RQ2). DSpark is the fastest Qwen3-8B drafter ($1.64\times$ at Q4, $1.93\times$ at Q8_0); EAGLE-3 is robust ($1.39\times$–$1.63\times$ with acceptance nearly constant across quantization); a vanilla 1.7B drafter is competitive ($1.41\times$, the highest acceptance among Qwen configurations, $\alpha = 0.725$); on Gemma-4-12B the MTP head ($2.39\times$–$2.71\times$) and DFlash ($2.09\times$–$2.30\times$) dominate.
• A clear negative result (RQ2). DFlash on Qwen3-8B fails, with acceptance $\approx 0.009$ and a speedup of only $0.63\times$, showing that a drafter whose drafts are never accepted is worse than no drafter at all.
• Confidence gating is not free in single-user mode (RQ2b). Raising DSpark's $p_\mathrm{min}$ from 0.0 to 0.6 improves acceptance ($0.616 \to 0.714$) but reduces throughput ($87.6 \to 80.7$ tok/s, $1.64\times \to 1.51\times$); the knob is a batched-serving feature, not a local speedup.
• Quantization is SD-friendly (RQ3). Acceptance is robust to target quantization (EAGLE-3: $\alpha \approx 0.43$–0.44 across Q4/Q5/Q8), while throughput comparisons remain conditional on the serving context. Q4 draft quantization is favorable in the tested Gemma setup ($\Delta\alpha = 0.010$, about 1 GiB of VRAM saved).
• Hardware break-even maps (RQ4). $\alpha_{be}$ ranges 6.5–39.9% across $k \in {5,7,10}$ in the present analysis, below the Apple-Silicon $k=4$ range and overlapping only its low $k=2$ range [@fonal2026crossfamilyspeculativedecodingpolish] in a directional comparison. The OLS fits are near-deterministic ($R^2 \ge 0.99$ for most configurations), but the resulting maps remain hardware-, engine-, context- and drafter-specific diagnostics rather than universal constants.

Reproducibility statement. All recorded runs are deterministic and the artifact bundle is sufficient to reproduce the reported analysis: we pin llama.cpp master at commit \texttt{07132750\allowbreak{}825a4f2d\allowbreak{}27a547cd\allowbreak{}9cdde1c6\allowbreak{}f6001885} (version 22), record SHA-256 hashes of every model in model-hashes.json, fix the seed and prompt order (seed 42), and store config.json, metrics.json and raw results.jsonl per run. Section Reproducibility gives three-step instructions to regenerate every number and figure.

Background and Related Work

Speculative decoding

SD losslessly accelerates autoregressive decoding by drafting multiple tokens with a cheaper model and verifying them against the target in parallel [@leviathan2023fastinferencetransformersspeculative; @chen2023acceleratinglargelanguagemodel]. The expected speedup is $(1 + \mathbb{E}[\tau]) / (1 + c_d/c_v)$, where $\tau$ is the number of accepted tokens and $c_d/c_v$ the draft/verify cost ratio.

Drafter taxonomy

We classify drafters by how the draft is produced, following the survey taxonomy of Xia et al. [@xia2024unlockingefficiencylargelanguage] and the 2025–2026 state of the art.

Model-free and training-free methods derive drafts from the target's own past without a separate model: Jacobi and n-gram lookahead [@lookahead2024], suffix-tree decoding for repetitive agentic workloads [@oliaro2025suffixdecodingextremespeculativedecoding], and multi-token-prediction (MTP) heads trained into the target itself (the Gemma-4-12B MTP head evaluated here). They add no extra weights and are the cheapest to deploy.

Independent draft models share the target's tokenizer but are trained separately: the vanilla small-LM drafter of the original SD papers [@leviathan2023fastinferencetransformersspeculative; @chen2023acceleratinglargelanguagemodel], distilled recurrent drafters (ReDrafter [@cheng2024recurrentdrafterfastspeculative], Clover-2 [@xiao2024clover2accurateinferenceregressive]), and cross-attention drafters such as Beagle [@zhong2026crossattentionspeculativedecoding]. Because the draft is generic, acceptance is usually lower, but the drafter can be reused across targets.

Target-dependent (feature-level) drafters condition on the target's hidden states and are the current industrial standard. Medusa attaches multiple decoding heads [@cai2024medusasimplellminference]; the EAGLE family drafts at feature level [@li2025eaglespeculativesamplingrequires], adds dynamic draft trees in EAGLE-2 [@li2024eagle2fasterinferencelanguage], and in EAGLE-3 predicts tokens directly with multi-layer fusion and training-time test (TTT), reaching up to $4.79\times$ on a 70B target [@li2025eagle3scalinginferenceacceleration]; ParallelSpec trains parallel drafters [@xiao2024parallelspecparalleldrafterefficient]. Open-source training frameworks such as SpecForge [@li2026specforgeflexibleefficientopensource] have made this family the default in practice.

Block-parallel and diffusion drafters (2025–2026) emit an entire draft block in a single forward pass. DFlash [@chen2026dflashblockdiffusionflash] uses a block-diffusion adapter and reports more than $2.5\times$ the speedup of EAGLE-3 on Qwen3-8B; DSpark [@cheng2026dsparkconfidencescheduledspeculativedecoding] adds a semi-autoregressive serial head that suppresses suffix decay and a confidence-scheduled verification policy for batched serving, reporting +30% accepted tokens over EAGLE-3 and 60–85% faster per-user generation on DeepSeek-V4 [@deepseekai2026deepseekv4highlyefficientmilliontoken]. Follow-ups explore diffusion draft trees [@ringel2026acceleratingspeculativedecodingblock], dual diffusion drafters [@zhang2026d2sdacceleratingspeculativedecoding], and diffusion language models as drafters [@li2025diffuspecunlockingdiffusionlanguage].

Self-speculative drafters reuse the target itself through early exit (Draft&Verify [@draftverify2024], LayerSkip [@elhoushi2024layerskipenablingearlyexit]), trading a separate drafter for skipped layers. Serving stacks add a further axis: tree-based verification (SpecInfer [@miao2024specinferacceleratinggenerativelarge]) and speculative-speculative decoding, which overlaps drafting with verification (SSD/Saguaro [@kumar2026speculativespeculativedecoding]), target batched serving and are out of scope for the single-user regime studied here.

Cross-vocabulary and cross-family drafting removes the shared-tokenizer constraint: UAG (SLEM string-round-trip and TLI vocabulary-intersection) provides lossless drafting for arbitrary draft/target pairs [@timor2025acceleratingllminferencelossless], on-device adaptive drafters such as OmniDraft [@ramakrishnan2025omnidraftcrossvocabularyonlineadaptive] target mobile hardware, and Fonal [@fonal2026crossfamilyspeculativedecodingpolish] recently evaluated UAG cross-family drafting on Apple Silicon, introducing the hardware break-even methodology we build on.

Finally, quantization interacts with all families: QSpec [@zhao2025qspecspeculativedecodingcomplementary] combines a quantized target with an FP16 draft, ML-SpecQD [@georganas2025mlspecqdmultilevelspeculativedecoding] quantizes the draft itself, and systematic compatibility evaluations show that 4-bit targets barely degrade acceptance while the draft/target cost ratio, not acceptance, ultimately bounds the speedup [@zhang2025speculativedecodingmeetsquantization].

Prior empirical studies

Four empirical strands are directly related to ours. Acceptance dynamics. Mahmoud [@mahmoud2026acceptancedynamicscognitivedomains] is the study closest to RQ1: a TinyLlama-1.1B draft verifies against Llama-2-7B-Chat-GPTQ with tree sampling over code, math, logical reasoning, and chat (99,768 nodes). He finds that task type dominates tree depth, that the entropy–acceptance correlation is negative but weak ($\rho \in [-0.20, -0.15]$), and, perhaps counterintuitively, that chat shows the highest acceptance (0.565 vs. 0.518 for math), which he attributes to the lexical predictability of the RLHF-aligned conversational register. Our RQ1 generally shows the opposite pattern on successful drafters (higher math/code and lower chat acceptance) with 8–12B GGUF targets and same-family block drafters, although the failed Qwen DFlash and gated DSpark cases are exceptions; we analyze this difference in the Discussion.

Drafter choice. Decoding Speculative Decoding [@decoding-sd2025-naacl] runs 350+ experiments on datacenter hardware and decomposes the speedup into a throughput–accuracy trade-off, giving principled guidance on when a given draft model pays off, but its targets and hardware are far from the 16 GB GGUF regime; recent work also steers pretrained drafters at inference time to raise acceptance without additional training [@steering-drafters2026-aaai]. Small models and software engineering. An EACL study shows that drafting overhead dominates on 1–2B targets, where unit speedup requires 77–94% acceptance [@empirical-sd-small-lm2026-eacl]; Li et al. [@li2026empiricalstudyspeculativedecoding] evaluate SD on software-engineering tasks (SWE-bench, Aider, Polyglot) with Llama-3.1/Qwen3-32B targets and find that $k=5$ helps most on code ($1.37\times$ to $1.58\times$) and that chat-trained drafters suffer distribution shift on SE workloads.

Hardware break-even. Fonal [@fonal2026crossfamilyspeculativedecodingpolish] evaluated UAG cross-family drafting with Bielik-11B on an Apple Silicon M2 Pro and introduced the break-even acceptance methodology we adopt: an OLS fit $TPS = a + \beta\cdot \alpha$ and the break-even point $\alpha_{be} = (TPS_{base} - a)/\beta$. His values (38.0–52.8% at $k=2$; 77.7–90.1% at $k=4$) are the reference we compare against in RQ4; he explicitly notes that extrapolating to other hardware requires re-estimating $\beta$ empirically on that platform, exactly what we do here. Quantization. Zhang et al. [@zhang2025speculativedecodingmeetsquantization] evaluate SD$\times$ quantization compatibility on Llama-3-8B/70B with EAGLE-2 and find that W4A16/W8A8 barely degrade $\tau$, with the speedup limited by the growing draft/target cost ratio rather than by acceptance; Zhao et al. [@zhao2025qspecspeculativedecodingcomplementary] show that a quantized target with an FP16 draft retains high acceptance; ML-SpecQD [@georganas2025mlspecqdmultilevelspeculativedecoding] quantizes the draft itself. These studies use full-precision or GPTQ/W4A16 targets on datacenter hardware; none combines GGUF quantization with the modern drafters we test.

Finally, on consumer hardware SpecExec [@svirschevski2024specexecmassivelyparallelspeculative] demonstrated SD on consumer devices with massive offloading (up to 20 tokens per cycle), establishing that the technique survives outside the datacenter; our work complements it by measuring the mainstream single-GPU, fully offloaded llama.cpp setup with official modern checkpoints.

Positioning. To our knowledge, no study combines the elements that define current local practice: a consumer NVIDIA GPU (16 GB), GGUF quantization of target and draft, modern dependent and block-parallel drafters with official checkpoints (EAGLE-3, DFlash, DSpark), multi-domain evaluation with per-position acceptance, and hardware break-even maps. Existing work covers domains [@mahmoud2026acceptancedynamicscognitivedomains], drafter choice [@decoding-sd2025-naacl], small models [@empirical-sd-small-lm2026-eacl], software-engineering tasks [@li2026empiricalstudyspeculativedecoding], a single Apple platform [@fonal2026crossfamilyspeculativedecodingpolish], and quantization in isolation [@zhang2025speculativedecodingmeetsquantization; @zhao2025qspecspeculativedecodingcomplementary], but the combination, and the resulting practical guidance for a 16 GB GPU, remains unmeasured.

Experimental Setup

We benchmark speculative decoding on a consumer NVIDIA GPU with two GGUF targets, seven drafter families and 26 target $\times$ drafter configurations. Every run is deterministic (greedy decoding, fixed seed, fixed prompt order), so all reported differences are attributable to the drafter and quantization choices rather than sampling noise.

Hardware and software

All experiments ran on a desktop with an NVIDIA RTX 4060 Ti 16 GB GPU, an Intel Core i5-12400 CPU and 31 GB of system RAM under Linux. We used llama.cpp at master commit 07132750825a4f2d27a547cd9cdde1c6f6001885 (version 22) built with CUDA 12.8, and served every model through a persistent llama-server process with full GPU offload (-ngl 99, CPU threads 8). Requests were issued one at a time (single-user, batch size 1). Each run directory stores a config.json that records the commit, the exact GGUF files, their SHA-256 hashes, and the sampling parameters, plus a metrics.json and raw results.jsonl, in experiments/runs/<id>/. Time-to-first-token, peak VRAM and peak power draw were sampled with a 1 Hz nvidia-smi sampler; the largest peak VRAM footprint observed (Gemma 4 12B Q8_0 + DFlash-F16) is 15.5 GiB, i.e. within the 16 GB budget.

Models and drafters

Targets are Gemma 4 12B (instruct) and Qwen3-8B, each quantized to GGUF Q4_K_M, Q5_K_M and Q8_0. Drafters, all converted to GGUF, are: the EAGLE-3 speculator for Qwen3-8B (F16) [@li2025eagle3scalinginferenceacceleration]; the DFlash block-7 drafter [@chen2026dflashblockdiffusionflash] for Qwen3-8B (F16) and for Gemma 4 12B (F16, Q4_K_M, Q8_0); the DSpark block-7 drafter [@cheng2026dsparkconfidencescheduledspeculativedecoding] for Qwen3-8B (F16) with confidence threshold (p-min) $\in {0.0, 0.2, 0.4, 0.6}$; a vanilla Qwen3-1.7B (Q4_K_M) as the classic small-drafter baseline; and the Gemma 4 12B multi-token-prediction (MTP) head (Q8_0).

Two cells do not exist because the drafter is unavailable for one target in the pinned llama.cpp build, and we report them as absent rather than omitting them by choice: DSpark for Gemma 4 12B, whose public Gemma DSpark GGUFs use the draft format of pull request #25549, which master does not expose (no LLM_ARCH_DSPARK); and EAGLE-3 for Gemma 4 12B, for which no GGUF conversion exists. Conversely, DFlash and MTP are only available for Gemma. Table [@tbl:configs] lists the 26 final configurations (15 Qwen3-8B, 11 Gemma 4 12B), including the six target-only baselines used to compute speedups.

Family	Target quant	Drafter	Draft format	p-min	k	ctx	n
Qwen3-8B	Q4_K_M	target only	—	—	3	8192	1474
Qwen3-8B	Q4_K_M	Vanilla-1.7B	Q4_K_M	—	3	8192	1474
Qwen3-8B	Q4_K_M	EAGLE-3	F16	—	3	8192	1474
Qwen3-8B	Q4_K_M	DFlash	F16 (block 7)	—	3	8192	1474
Qwen3-8B	Q4_K_M	DSpark	F16 (block 7)	0.0	3	8192	1474
Qwen3-8B	Q4_K_M	DSpark	F16 (block 7)	0.2	3	8192	1474
Qwen3-8B	Q4_K_M	DSpark	F16 (block 7)	0.4	3	8192	1474
Qwen3-8B	Q4_K_M	DSpark	F16 (block 7)	0.6	3	8192	1474
Qwen3-8B	Q5_K_M	target only	—	—	3	8192	1474
Qwen3-8B	Q5_K_M	EAGLE-3	F16	—	3	8192	1474
Qwen3-8B	Q5_K_M	DFlash	F16 (block 7)	—	3	8192	1474
Qwen3-8B	Q5_K_M	DSpark	F16 (block 7)	0.0	3	8192	1474
Qwen3-8B	Q8_0	target only	—	—	3	8192	1474
Qwen3-8B	Q8_0	EAGLE-3	F16	—	3	8192	1474
Qwen3-8B	Q8_0	DSpark	F16 (block 7)	0.0	3	8192	1474
Gemma 4 12B	Q4_K_M	target only	—	—	3	16384	1455
Gemma 4 12B	Q4_K_M	DFlash	F16	—	3	16384	1455
Gemma 4 12B	Q4_K_M	DFlash	Q4_K_M	—	3	16384	1455
Gemma 4 12B	Q4_K_M	DFlash	Q8_0	—	3	16384	1455
Gemma 4 12B	Q4_K_M	MTP	Q8_0	—	3	16384	1455
Gemma 4 12B	Q5_K_M	target only	—	—	3	16384	1449
Gemma 4 12B	Q5_K_M	DFlash	F16	—	3	16384	1449
Gemma 4 12B	Q5_K_M	MTP	Q8_0	—	3	16384	1449
Gemma 4 12B	Q8_0	target only	—	—	3	8192	1452
Gemma 4 12B	Q8_0	DFlash	F16	—	3	8192	1452
Gemma 4 12B	Q8_0	MTP	Q8_0	—	3	8192	1452

: The 26 final configurations (15 Qwen3-8B, 11 Gemma 4 12B, including six target-only baselines). Draft length $k = 3$ in every run; p-min is DSpark's confidence threshold; ctx is the context window (8,192 for all Qwen3-8B and Gemma-Q8_0, 16,384 for Gemma-Q4/Q5); $n$ is the number of records kept after the exclusions described below. {#tbl:configs}

Benchmarks and prompts

Nine benchmarks grouped into three cognitive domains: math (GSM8K, MATH-500, AIME25), code (HumanEval, MBPP, LiveCodeBench) and chat (MT-Bench, Alpaca, Arena-Hard-v2). We subsampled each benchmark with fixed seed 42 into a main evaluation set of 1,474 fully templated prompts (math 430: GSM8K 200, MATH-500 200, AIME25 30; code 564: HumanEval 164, MBPP 200, LiveCodeBench 200; chat 480: MT-Bench 80, Alpaca 200, Arena-Hard-v2 200), and a stratified acceptance sample of 180 prompts (60 per domain) used for the acceptance curves and the draft-length sweep. No thinking mode is used (--reasoning off); the models generate in their default instruct style.

Decoding protocol

All generations are greedy: sampling temperature 0.0, top-$k$ 40, top-$p$ 0.95, seed 42, maximum 256 output tokens, with up to two retries per prompt and incremental JSONL resume. The main comparison uses draft length $k = 3$ with context 8,192 for all Qwen3-8B runs and for Gemma-Q8_0, and 16,384 for Gemma-Q4/Q5 (the Gemma 4 12B instruct template is longer). The acceptance curves and the draft-length sweep (ksweep) run the 180-prompt sample at context 2,048 with $k = 3$ and $k \in {5, 7, 10}$ respectively. Because decoding is deterministic, the curves measured on the 180-prompt subset are identical, for those prompts, to the final runs. For the final runs, speedups are computed against a target-only baseline of the same quantization over the same prompt set.

Metrics

Per-prompt throughput (tok/s) is llama-server's predicted_per_second. The headline speedup is the ratio of the mean tok/s of a draft configuration to the mean tok/s of the matching baseline (ratio of means); we also compute the mean and median of the per-prompt speedup ratios for robustness; they agree with the ratio of means within a few percent (largest divergence, 7%, occurs for Gemma DFlash, whose per-prompt distribution is mildly skewed). Acceptance $\alpha$ is the fraction of drafted tokens accepted by the target, averaged over prompts; $\tau$ is the mean number of accepted draft tokens per prompt. Per-position acceptance is read from the server's acc per pos logs of the curves runs. We additionally record time-to-first-token (TTFT, prompt_ms), peak VRAM and peak power (1 Hz nvidia-smi sampler). Representative TTFT figures: Qwen3-8B mean 65–84 ms and Gemma 4 12B mean 149–212 ms across configurations; peak power draw ranges 138–167 W.

Data exclusions and integrity

llama-server reports spurious timing records (tok_per_s = 1,000,000, predicted_ms = 0) on 271 records of Gemma runs; we excluded every record with tok_per_s >= 1e5 or predicted_ms <= 0; in the observed records, these timing sentinels also had alpha/tau/draft_n = None. No such sentinels occur in Qwen runs. Deterministic failures further reduce the Gemma sample: the Gemma-Q4 runs (5 configurations) lost humaneval-00050 to a reproducible HTTP 500 error, leaving $n = 1473$ raw and $n = 1455$ clean records per run; the Gemma-Q8 runs (3 configurations) lost arena-hard-v2-00233 (over context) and arena-hard-v2-00239 (HTTP 500), leaving $n = 1472$ raw and $n = 1452$ clean; the Gemma-Q5 runs kept all 1,474 raw records and lost 25 sentinels per run ($n = 1449$). In the curves/ksweep runs, two Arena-Hard-v2 prompts exceed the 2,048-token context (-00089, -00164) and were dropped, leaving 178 of 180 prompts for Qwen3-8B (chat $n = 58$); Gemma curves/ksweep lose three additional sentinel records per run, leaving 175 (math 60, code 60, chat 55). Finally, two power cuts occurred during measurement; both were repaired by resuming from the incremental JSONL (union of results and errors records) with no data loss.

Results

Acceptance dynamics by domain (RQ1)

[@fig:acceptance] (Qwen3-8B-Q4, panels a–g) and [@fig:acceptance-gemma] (Gemma 4 12B-Q4, panels a–d) show per-position acceptance at draft length $k = 3$ for all 11 drafter configurations, split by domain. Three regularities are visible. First, acceptance decays monotonically with draft position (the well-documented suffix decay [@oliaro2025suffixdecodingextremespeculativedecoding]): e.g., EAGLE-3 on Qwen math drops from 0.659 at position 1 to 0.438 at position 2 and 0.238 at position 3. The decay rate is drafter-specific: Gemma MTP stays near 0.74 at the third position on math ($0.905 \rightarrow 0.813 \rightarrow 0.737$), while EAGLE-3 loses most of its value by position 3. Second, chat is generally the hardest domain: after the first position, its curve lies below math and code in every configuration; the near-zero Qwen DFlash panels are the exception at position 1. Third, the domain gap is much larger on Gemma than on Qwen, both absolutely and per position: for Gemma DFlash-F16 the first position accepts 0.888 of math drafts but only 0.652 of chat drafts, and by position 3 chat acceptance falls to 0.322 (vs. 0.635 for math); on Qwen the same drafter family shows a smaller spread (e.g., DSpark p=0.0: 0.835 vs. 0.692 at position 1).

Table [@tbl:alpha-tau] aggregates acceptance over the full 256-token generations of the final runs. Chat is generally the lowest-acceptance domain, especially on Gemma; near-zero Qwen DFlash and gated DSpark are exceptions, while the relative ordering of math and code is configuration-dependent. Gemma shows the sharpest penalty: chat $\alpha$ ranges 0.463–0.608 while math $\alpha$ ranges 0.770–0.837, a gap of roughly 0.2–0.3. On Qwen the gap is smaller (EAGLE-3: 0.428 vs. 0.445; DSpark p=0.0: 0.549 vs. 0.706) and vanilla-1.7B keeps the highest chat acceptance of the non-gated Qwen configurations (0.684; only DSpark at $p_\mathrm{min}=0.6$ reaches 0.723). This generally reversed "chat paradox" (for viable drafters, the workload that dominates real-world use of local models benefits the least from drafting) differs from the domain ordering reported by Mahmoud [@mahmoud2026acceptancedynamicscognitivedomains], who found chat to be the highest-acceptance domain with a small independent draft on a 7B target (0.565 vs. 0.518 for math); the Discussion analyzes this difference. For the viable drafters, measured speedups on chat are generally the smallest of the three domains (see RQ2); the near-zero DFlash case is an unhelpful exception. Thus, gains reported on math-heavy or code-heavy suites can overestimate what a typical chat user will see.

Family	Config	Math $\alpha$ ($\tau$)	Code $\alpha$ ($\tau$)	Chat $\alpha$ ($\tau$)	All $\alpha$ ($\tau$)
Qwen3-8B	q4-vanilla17b	0.784 (178)	0.716 (173)	0.684 (169)	0.725 (173)
Qwen3-8B	q4-eagle3	0.445 (144)	0.447 (144)	0.428 (133)	0.440 (141)
Qwen3-8B	q4-dflash	0.009 (6)	0.009 (7)	0.010 (7)	0.009 (7)
Qwen3-8B	q4-dspark-p0	0.706 (171)	0.606 (163)	0.549 (154)	0.616 (162)
Qwen3-8B	q4-dspark-p2	0.707 (171)	0.606 (163)	0.561 (154)	0.621 (162)
Qwen3-8B	q4-dspark-p4	0.712 (171)	0.622 (163)	0.616 (150)	0.646 (161)
Qwen3-8B	q4-dspark-p6	0.743 (169)	0.684 (157)	0.723 (138)	0.714 (154)
Qwen3-8B	q5-eagle3	0.446 (144)	0.450 (144)	0.421 (132)	0.439 (140)
Qwen3-8B	q5-dflash	0.009 (7)	0.009 (6)	0.009 (7)	0.009 (7)
Qwen3-8B	q5-dspark-p0	0.699 (171)	0.609 (163)	0.542 (153)	0.613 (162)
Qwen3-8B	q8-eagle3	0.441 (144)	0.442 (144)	0.414 (132)	0.432 (140)
Qwen3-8B	q8-dspark-p0	0.703 (171)	0.603 (163)	0.538 (153)	0.611 (162)
Gemma 4 12B	q4-dflash-f16	0.776 (175)	0.718 (167)	0.475 (126)	0.657 (156)
Gemma 4 12B	q4-dflash-q4	0.770 (174)	0.708 (166)	0.463 (124)	0.647 (155)
Gemma 4 12B	q4-dflash-q8	0.775 (175)	0.718 (167)	0.474 (126)	0.656 (156)
Gemma 4 12B	q4-mtp	0.830 (179)	0.783 (172)	0.603 (140)	0.739 (164)
Gemma 4 12B	q5-dflash-f16	0.782 (176)	0.724 (167)	0.469 (126)	0.659 (157)
Gemma 4 12B	q5-mtp	0.837 (180)	0.788 (172)	0.608 (142)	0.745 (165)
Gemma 4 12B	q8-dflash-f16	0.780 (175)	0.727 (167)	0.464 (124)	0.658 (156)
Gemma 4 12B	q8-mtp	0.837 (179)	0.790 (172)	0.607 (140)	0.745 (164)

: Acceptance $\alpha$ (fraction of drafted tokens accepted) and $\tau$ (mean accepted draft tokens per prompt), shown as $\alpha$ ($\tau$), by configuration and domain (final runs, $k = 3$). {#tbl:alpha-tau}

{#fig:acceptance}

{#fig:acceptance-gemma}

Takeaway. Acceptance falls off quickly the further the draft looks ahead, and aggregate acceptance is lowest on chat, the domain users actually run on local GPUs. Any speedup reported on math/code suites overstates what chat workloads will deliver.

Modern drafters on consumer hardware (RQ2)

[@tbl:speedup] reports per-domain and overall speedups (ratio of means vs. same-quantization target-only, same prompt set) for the 20 drafter configurations, and [@fig:speedup] plots overall speedup against overall $\alpha$ per configuration and domain. The transfer of datacenter gains to this 16 GB GPU is real but drafter-dependent.

DSpark is the fastest Qwen drafter. With $p_\mathrm{min} = 0.0$ on Qwen3-8B-Q4 it reaches 87.6 tok/s, $1.64\times$ overall (math $1.79\times$, code $1.63\times$, chat $1.53\times$), and $1.93\times$ overall at Q8_0 (up to $2.11\times$ on math) at 62.3 tok/s, the largest speedups of any Qwen configuration. Its acceptance is moderate ($\alpha \approx 0.61$–0.62) and stable across quantizations.

EAGLE-3 is solid and quantization-robust. It delivers $1.39\times$, $1.46\times$ and $1.63\times$ at Q4/Q5/Q8 (74.4, 68.3 and 52.8 tok/s) while $\alpha$ stays essentially constant (0.440/0.439/0.432). The speedup grows as the target gets slower because the drafter cost is paid in draft tokens, not in acceptance quality.

The vanilla 1.7B drafter is surprisingly competitive on Q4. It reaches $1.41\times$ (75.5 tok/s) with the highest acceptance among Qwen configurations ($\alpha = 0.725$; only Gemma's MTP head is higher, at 0.739–0.745), only 14% below DSpark at a fraction of the complexity. Its per-position decay is the gentlest observed (math: $0.873 \rightarrow 0.686$ across positions 1–3), confirming that for an 8B target a small same-family drafter can already carry most of the value.

DFlash on Qwen fails, an important negative result. Despite a block-7 drafter of comparable size to EAGLE-3's, the DFlash drafts are accepted at $\alpha \approx 0.009$, i.e. essentially never (per-position acceptance stays 0.02–0.00). The system then degenerates into running both models per step and slows down to $0.63\times$ (33.4 tok/s vs. 53.4 baseline), at every quantization (0.63–$0.66\times$). This is drafter- and target-specific, not a general DFlash failure: on Gemma 4 12B the same architecture is the strongest drafter, reaching 2.09–$2.30\times$ (DFlash) and 2.39–$2.71\times$ (MTP). We speculate the Qwen DFlash GGUF draft is distributionally incompatible with the target's sampling (e.g. vocabulary or positional mismatch), which future work should audit; for practitioners, the takeaway is that a same-size diffusion drafter on a fast target can cost more than it saves.

Gemma's block drafters are the best absolute performers. MTP and DFlash reach 2.39–$2.71\times$ and 2.09–$2.30\times$ respectively (82.7 and 78.4 tok/s at Q4) because the Gemma Q4/Q5/Q8 targets are far slower (34.7/29.9/19.9 tok/s solo) and the drafts are cheap: the draft runs at a fraction of the target cost while acceptance stays high ($\alpha$ 0.65–0.75).

[@fig:speedup] organizes these results into four clusters: Gemma block drafters in the upper region ($\alpha$ 0.65–0.75, speedup 2.1–$2.7\times$); DSpark and EAGLE-3 in the middle band ($\alpha$ 0.43–0.62, speedup 1.39–$1.93\times$); vanilla-1.7B at high $\alpha$ (0.73) with moderate speedup ($1.41\times$); and DFlash-on-Qwen isolated at $\alpha \approx 0.009$ inside the shaded loss region. Within a drafter, quantization moves points along a line of roughly constant $\alpha$ (EAGLE-3) or slightly decreasing $\alpha$ (DSpark), while the domain markers spread each configuration vertically: chat is generally the slowest, with near-zero DFlash approximately tied across domains.

Family	Config	$\alpha$	$\tau$	Math x	Code x	Chat x	All x	All tok/s
Qwen3-8B	q4-solo	—	—	—	—	—	—	53.4
Qwen3-8B	q4-vanilla17b	0.725	173	1.50x	1.40x	1.36x	1.41x	75.5
Qwen3-8B	q4-eagle3	0.440	141	1.40x	1.41x	1.37x	1.39x	74.4
Qwen3-8B	q4-dflash	0.009	7	0.63x	0.63x	0.63x	0.63x	33.4
Qwen3-8B	q4-dspark-p0	0.616	162	1.79x	1.63x	1.53x	1.64x	87.6
Qwen3-8B	q4-dspark-p2	0.621	162	1.78x	1.61x	1.51x	1.62x	86.7
Qwen3-8B	q4-dspark-p4	0.646	161	1.77x	1.59x	1.46x	1.60x	85.4
Qwen3-8B	q4-dspark-p6	0.714	154	1.73x	1.50x	1.33x	1.51x	80.7
Qwen3-8B	q5-solo	—	—	—	—	—	—	46.8
Qwen3-8B	q5-eagle3	0.439	140	1.47x	1.48x	1.43x	1.46x	68.3
Qwen3-8B	q5-dflash	0.009	7	0.66x	0.66x	0.66x	0.66x	30.8
Qwen3-8B	q5-dspark-p0	0.613	162	1.87x	1.72x	1.60x	1.72x	80.6
Qwen3-8B	q8-solo	—	—	—	—	—	—	32.3
Qwen3-8B	q8-eagle3	0.432	140	1.65x	1.65x	1.59x	1.63x	52.8
Qwen3-8B	q8-dspark-p0	0.611	162	2.11x	1.91x	1.78x	1.93x	62.3
Gemma 4 12B	q4-solo	—	—	—	—	—	—	34.7
Gemma 4 12B	q4-dflash-f16	0.657	156	2.33x	2.22x	1.71x	2.09x	72.2
Gemma 4 12B	q4-dflash-q4	0.647	155	2.54x	2.41x	1.85x	2.26x	78.4
Gemma 4 12B	q4-dflash-q8	0.656	156	2.47x	2.35x	1.81x	2.21x	76.6
Gemma 4 12B	q4-mtp	0.739	164	2.59x	2.49x	2.09x	2.39x	82.7
Gemma 4 12B	q5-solo	—	—	—	—	—	—	29.9
Gemma 4 12B	q5-dflash-f16	0.659	157	2.42x	2.30x	1.75x	2.16x	64.5
Gemma 4 12B	q5-mtp	0.745	165	2.75x	2.64x	2.21x	2.53x	75.6
Gemma 4 12B	q8-solo	—	—	—	—	—	—	19.9
Gemma 4 12B	q8-dflash-f16	0.658	156	2.58x	2.46x	1.85x	2.30x	45.7
Gemma 4 12B	q8-mtp	0.745	164	2.94x	2.82x	2.36x	2.71x	54.0

: Speedup by configuration and domain (ratio of means vs. same-quantization target-only over the same prompt set) with overall acceptance $\alpha$, $\tau$ and throughput (tok/s). Baselines (solo) shown for reference. {#tbl:speedup}

{#fig:speedup}

Takeaway. Modern drafters do transfer to a 16 GB GPU, but the winner is the one whose draft cost is genuinely small relative to the target: DSpark on Qwen ($1.64\times$), DFlash/MTP on Gemma (up to $2.71\times$), and even a plain 1.7B drafter ($1.41\times$). A drafter whose drafts are never accepted (DFlash-on-Qwen, $0.63\times$) is worse than no drafter at all.

DSpark confidence gating (RQ2b)

DSpark's confidence-scheduled verification accepts a draft block only when the drafter's confidence exceeds a threshold $p_\mathrm{min}$; otherwise the target generates alone. On Qwen3-8B-Q4, raising $p_\mathrm{min}$ from 0.0 to 0.6 behaves exactly as the paper describes: acceptance improves monotonically, from $\alpha = 0.616$ (p=0.0) to 0.621 (p=0.2), 0.646 (p=0.4) and 0.714 (p=0.6), with the largest gain in chat ($0.549 \rightarrow 0.723$). However, throughput decreases: 87.6 $\rightarrow$ 86.7 $\rightarrow$ 85.4 $\rightarrow$ 80.7 tok/s, i.e. speedup falls from $1.64\times$ to $1.51\times$ (chat drops hardest: $1.53\times \rightarrow 1.33\times$).

The likely mechanism is a batch-size effect. Tighter gating turns more decode steps into target-only steps; in batched multi-user serving those target passes are amortized across many requests, which is the regime DSpark is designed for [@cheng2026dsparkconfidencescheduledspeculativedecoding]. In our single-user batch-1 setting the extra target work is paid per request, and the higher acceptance no longer compensates: each rejected block costs a full target forward pass that produces nothing. The net effect is a monotone tok/s loss as $p_\mathrm{min}$ grows, even where $\alpha$ gains are largest.

Takeaway. On single-user consumer hardware, confidence-scheduled verification buys acceptance but not speed, because the extra target work is not amortized. DSpark's gating knob is a serving-time feature, not a free single-user speedup; this contrasts directly with the multi-user results in the DSpark paper.

Quantization (RQ3)

[@tbl:quantization] crosses target quantization (Q4/Q5/Q8) with drafter for both families. Two effects stand out.

Acceptance is robust to target quantization. EAGLE-3 keeps $\alpha \approx 0.43$–0.44 across Q4/Q5/Q8 (0.440/0.439/0.432), DSpark stays at 0.61–0.62 (0.616/0.613/0.611), and Gemma MTP at 0.739–0.745. Quantizing the target degrades its own generation slightly but does not change how well it agrees with the drafter, which agrees with the compatibility studies of Zhang et al. [@zhang2025speculativedecodingmeetsquantization] and Zhao et al. [@zhao2025qspecspeculativedecodingcomplementary].

Throughput differences are context-dependent. For Qwen, where all final target quantizations use ctx=8192, the slower Q8_0 target gives the same acceptance more time to amortize drafting: EAGLE-3 goes $1.39\times \rightarrow 1.46\times \rightarrow 1.63\times$ and DSpark $1.64\times \rightarrow 1.72\times \rightarrow 1.93\times$ across Q4/Q5/Q8. Gemma Q4/Q5 use ctx=16384, whereas Gemma Q8 uses ctx=8192, so its raw throughput and speedup differences cannot be attributed to target quantization alone. We therefore treat those cross-quantization values as conditional observations rather than an isolated quantization effect.

Quantizing the draft is favorable in the tested Gemma setup. On Gemma-Q4 with DFlash drafts [@tbl:draft-quant], the Q4_K_M draft is the fastest (78.4 tok/s, $2.26\times$, $\alpha$ 0.647) and the F16 draft the slowest (72.2 tok/s, $2.09\times$, $\alpha$ 0.657), yet $\alpha$ differs by only 0.010 across draft quantizations, so the draft's predictive distribution is preserved. The Q4 draft also saves ~1.0 GiB of VRAM (9,730 vs. 10,766 MiB peak vs. F16) and reduces the on-disk draft from 1.47 GB (F16) to 0.44 GB (Q4_K_M) or 0.79 GB (Q8_0). This is a favorable, configuration-specific trade-off whose applicability beyond this setup is untested.

Family	Target quant	Drafter	n	tok/s	x vs solo	$\alpha$	$\tau$
Qwen3-8B	q4	Vanilla-1.7B	1474	75.5	1.41x	0.725	173
Qwen3-8B	q4	EAGLE-3	1474	74.4	1.39x	0.440	141
Qwen3-8B	q4	DFlash-F16	1474	33.4	0.63x	0.009	7
Qwen3-8B	q4	DSpark p=0.0	1474	87.6	1.64x	0.616	162
Qwen3-8B	q4	DSpark p=0.2	1474	86.7	1.62x	0.621	162
Qwen3-8B	q4	DSpark p=0.4	1474	85.4	1.60x	0.646	161
Qwen3-8B	q4	DSpark p=0.6	1474	80.7	1.51x	0.714	154
Qwen3-8B	q5	EAGLE-3	1474	68.3	1.46x	0.439	140
Qwen3-8B	q5	DFlash-F16	1474	30.8	0.66x	0.009	7
Qwen3-8B	q5	DSpark p=0.0	1474	80.6	1.72x	0.613	162
Qwen3-8B	q8	EAGLE-3	1474	52.8	1.63x	0.432	140
Qwen3-8B	q8	DSpark p=0.0	1474	62.3	1.93x	0.611	162
Gemma 4 12B	q4	DFlash-F16	1455	72.2	2.09x	0.657	156
Gemma 4 12B	q4	DFlash-Q4	1455	78.4	2.26x	0.647	155
Gemma 4 12B	q4	DFlash-Q8	1455	76.6	2.21x	0.656	156
Gemma 4 12B	q4	MTP	1455	82.7	2.39x	0.739	164
Gemma 4 12B	q5	DFlash-F16	1449	64.5	2.16x	0.659	157
Gemma 4 12B	q5	MTP	1449	75.6	2.53x	0.745	165
Gemma 4 12B	q8	DFlash-F16	1452	45.7	2.30x	0.658	156
Gemma 4 12B	q8	MTP	1452	54.0	2.71x	0.745	164

: Target quantization (Q4_K_M/Q5_K_M/Q8_0) crossed with drafter, per family: throughput (tok/s), speedup vs. same-quant solo, acceptance $\alpha$ and $\tau$ (final runs, $k = 3$). {#tbl:quantization}

Drafter (draft quant)	n	tok/s	x vs solo	$\alpha$	$\tau$	max VRAM (MiB)
DFlash-F16	1455	72.2	2.09x	0.657	156	10766
DFlash-Q4	1455	78.4	2.26x	0.647	155	9730
DFlash-Q8	1455	76.6	2.21x	0.656	156	10043

: Effect of draft quantization on Gemma 4 12B-Q4 (DFlash drafts, final runs). F16 = 1.47 GB draft, Q4_K_M = 0.44 GB, Q8_0 = 0.79 GB (model hashes). {#tbl:draft-quant}

Takeaway. In the tested configurations, target quantization changes acceptance only modestly, and a Q4 draft is both faster and ~1 GiB cheaper than an F16 draft on Gemma-Q4. Throughput comparisons across target quantizations must retain their context-window conditions. On this 16 GB card, Q4 target plus Q4 draft is a practical operating point, not a universal rule.

Break-even maps (RQ4)

Following the OLS methodology used for Apple Silicon by Fonal [@fonal2026crossfamilyspeculativedecodingpolish] (adapted from Bielik et al.), we fit, per configuration and draft length $k$, a least-squares line $TPS = a + \beta \cdot \alpha$ over the per-prompt $(\alpha, \mathrm{tok/s})$ pairs of the ksweep runs, and define the break-even acceptance $\alpha_{be} = (TPS_{base} - a)/\beta$, where $TPS_{base}$ is the mean tok/s of a target-only baseline matched by target, context, prompt-set content, and sampling protocol; the estimate uses only prompt IDs shared by both runs. 95% CIs come from the delta method on the OLS covariance. [@tbl:breakeven] and [@fig:breakeven] report the pooled (all-domain) results for $k \in {5, 7, 10}$.

On the RTX 4060 Ti, several tested configurations break even at low acceptance. The pooled $\alpha_{be}$ ranges are 10.7–39.6% at $k=5$, 11.0–39.9% at $k=7$ and 6.5–32.8% at $k=10$. For the best drafters the map is striking: Gemma DFlash-F16 needs only $\alpha_{be} = 0.065$ at $k=10$ (0.118 at $k=5$), and Qwen EAGLE-3 ranges from 0.185 to 0.232 across the three draft lengths. The upper end of the full RTX range overlaps only the low end of the Apple-Silicon reference of 38.0–52.8% at $k=2$; the best block drafters remain below that band, and all tested RTX values are below its 77.7–90.1% range at $k=4$ [@fonal2026crossfamilyspeculativedecodingpolish]. The comparison is directional: the platforms, engines, target/drafter combinations, and draft-length ranges differ. Within this matched ctx=2,048 protocol, the target-only baselines are slow enough that low acceptance clears the measured drafting cost for the best configurations.

The slope $\beta$ generally grows with $k$. Longer drafts usually win more tok/s per unit of acceptance (e.g. Gemma DFlash-F16: $\beta = 96.8 \rightarrow 110.3 \rightarrow 185.7$ for $k = 5/7/10$), which often lowers $\alpha_{be}$ with $k$. Qwen Q4 DFlash-F16 is the exception: its slope peaks at $k=7$ and its pooled map stays nearly flat at 0.157–0.160; Qwen Q4 DSpark varies only from 0.182 to 0.187.

[@fig:ksweep] shows the per-position acceptance curves behind these maps for two Qwen3-8B-Q4 configurations: extending the draft from $k=5$ to $k=10$ adds positions with progressively lower acceptance (suffix decay), yet the break-even gains at larger $k$ come from the slope $\beta$, not from higher late-position acceptance.

{#fig:ksweep}

The fits are near-deterministic: $R^2$ ranges 0.86–1.00 across all pooled configurations (mostly above 0.99), so $\alpha_{be}$ is stable within this measurement protocol rather than an artifact of noisy data. It remains a diagnostic of the tested hardware, engine, context, and drafter combination, not a universal hardware constant.

Slow targets and slow drafters bracket the map. Under the matched ctx=2,048 baselines, Qwen Q8_0 runs at 31.8 tok/s versus 51.5 tok/s for Q4, and EAGLE-3 has $\alpha_{be}=0.110$–$0.116$ across the sweep; a slower target is cheaper to pay with acceptance. Conversely the vanilla 1.7B drafter has the highest break-even of all (0.328–0.399): its flatter slope ($\beta = 89$–117) means each accepted token buys less, so it needs substantially more acceptance to justify itself. Domain-level maps track the pooled values within a few points (e.g. Qwen Q4 EAGLE-3 $k=7$: 0.220–0.247 across math/code/chat), i.e. the break-even picture is domain-insensitive.

Config	k	baseline (tok/s; n/match)	beta (slope)	alpha_be	CI95	R2
Gemma Q4 / DFlash-F16	10	33.5; 175/175	185.65	0.065	0.008	0.997
Gemma Q4 / DFlash-F16	5	33.5; 175/175	96.78	0.118	0.012	0.993
Gemma Q4 / DFlash-F16	7	33.5; 175/175	110.32	0.144	0.009	0.996
Gemma Q4 / MTP	10	33.5; 175/175	162.16	0.084	0.010	0.996
Gemma Q4 / MTP	5	33.5; 175/175	97.16	0.107	0.014	0.990
Gemma Q4 / MTP	7	33.5; 175/175	106.38	0.147	0.012	0.993
Qwen Q4 / DFlash-F16	10	51.5; 178/178	155.12	0.160	0.060	0.858
Qwen Q4 / DFlash-F16	5	51.5; 178/178	142.47	0.159	0.029	0.963
Qwen Q4 / DFlash-F16	7	51.5; 178/178	157.21	0.157	0.024	0.974
Qwen Q4 / DSpark p=0	10	51.5; 178/178	152.84	0.183	0.004	0.999
Qwen Q4 / DSpark p=0	5	51.5; 178/178	126.65	0.187	0.012	0.992
Qwen Q4 / DSpark p=0	7	51.5; 178/178	150.92	0.182	0.009	0.996
Qwen Q4 / EAGLE-3	10	51.5; 178/178	179.97	0.185	0.004	0.999
Qwen Q4 / EAGLE-3	5	51.5; 178/178	124.01	0.212	0.003	0.999
Qwen Q4 / EAGLE-3	7	51.5; 178/178	136.39	0.232	0.002	1.000
Qwen Q4 / Vanilla-1.7B	10	51.5; 178/178	116.94	0.328	0.008	0.996
Qwen Q4 / Vanilla-1.7B	5	51.5; 178/178	89.13	0.396	0.017	0.978
Qwen Q4 / Vanilla-1.7B	7	51.5; 178/178	93.02	0.399	0.007	0.997
Qwen Q8 / EAGLE-3	10	31.8; 178/178	149.50	0.111	0.004	0.999
Qwen Q8 / EAGLE-3	5	31.8; 178/178	99.71	0.116	0.004	0.999
Qwen Q8 / EAGLE-3	7	31.8; 178/178	124.64	0.110	0.004	0.999

: Break-even acceptance $\alpha_{be}$ per configuration and draft length $k$ (ksweep runs, ctx 2,048, pooled over domains). All baselines use ctx=2,048; the baseline is matched by target, prompt-set content, and sampling protocol. n/match reports the baseline records retained and shared with the ksweep. OLS fit $TPS = a + \beta \alpha$; CI95 by delta method. $\alpha_{be} > 1.00$ means no achievable break-even. Reference on Apple Silicon: $k=2$: 38.0–52.8%, $k=4$: 77.7–90.1% [@fonal2026crossfamilyspeculativedecodingpolish]. {#tbl:breakeven}

{#fig:breakeven}

Takeaway. On a 16 GB consumer GPU, the tested configurations break even at acceptance rates as low as 6.5%, with a pooled range of 6.5–39.9% across the sweep. The upper end of this range slightly overlaps the low end of the 38–90% reported on Apple Silicon, while all tested values are below its $k=4$ range; that cross-platform comparison is directional rather than controlled. The slower the target and the longer the draft, the cheaper it is to pay with acceptance; only a slow drafter like vanilla-1.7B demands substantial acceptance (33–40%) to justify itself.

Discussion

Why datacenter numbers do not transfer

The gap between published datacenter speedups and what we measure has a structural cause, visible in the speedup formula $(1 + \mathbb{E}[\tau]) / (1 + c_d/c_v)$: every result is a race between the tokens a drafter gets accepted ($\mathbb{E}[\tau]$) and the relative cost of drafting ($c_d/c_v$). Datacenter serving changes the second term, not the first. DSpark is the clearest case. Its confidence-scheduled verification is designed for batched multi-user serving, where the target-only steps introduced by rejection are amortized across many concurrent requests; that is the regime in which it reports 60–85% faster per-user generation on DeepSeek-V4 [@cheng2026dsparkconfidencescheduledspeculativedecoding]. In our batch-1 setting every rejected block costs a full target forward pass paid by a single request, so raising $p_\mathrm{min}$ raises acceptance ($0.616 \to 0.714$) while lowering throughput ($87.6 \to 80.7$ tok/s). The knob is correctly calibrated, doing exactly what the paper says, but its benefit only materializes when target work can be shared across users.

DFlash on Qwen is the second case, and it isolates the cost term. The block-diffusion drafter emits a whole block per forward pass; that cost is recovered only if the block is accepted. On Gemma-4-12B, where acceptance is 0.65–0.75, DFlash is among the best drafters we measured ($2.09\times$–$2.30\times$); on Qwen3-8B, where acceptance collapses to $\alpha \approx 0.009$, the same architecture degenerates into running both models every step and loses to the baseline ($0.63\times$). The DFlash paper reports more than $2.5\times$ the speedup of EAGLE-3 on Qwen3-8B; our measurement shows that gain is not intrinsic to the architecture: it depends on the draft being distributionally compatible with the target as served (with our GGUF conversion and greedy sampler it is not), and on the target being slow enough to absorb the draft cost. This is a concrete instance of the limitation the DSpark authors themselves flag: the draft-side cost of a full block is unrecoverable on intrinsically low-acceptance queries.

A third, subtler consequence of the regime difference concerns the domain ordering itself. Mahmoud [@mahmoud2026acceptancedynamicscognitivedomains] found chat to be the highest-acceptance domain with a TinyLlama-1.1B draft on a 7B target, attributing the effect to the lexical predictability of the RLHF-aligned register. We instead generally find lower chat acceptance on these 8–12B GGUF targets, especially for Gemma and viable drafters; near-zero Qwen DFlash and gated DSpark are exceptions, while the relative ordering of math and code changes slightly across configurations. Two non-exclusive explanations are plausible, but this experiment does not identify their separate causal contributions. First, our drafters are target-dependent: conditioning on the target's hidden states may favor structured, low-entropy continuations of math and code, while open-ended chat remains hard to predict. Second, our targets are larger, quantized, and decoded greedily, which may change the relative advantage of a same-family draft. These are hypotheses rather than causal tests. The two studies are therefore best treated as complementary: observed acceptance is a property of the domain together with the drafter, target, and serving regime, which is why per-family measurements matter.

Lessons for the local community

Within the limits of a single GPU and engine, our results translate into concrete recommendations for a 16 GB NVIDIA card:

• Qwen3-8B at Q4: use DSpark with $p_\mathrm{min} = 0.0$ ($1.64\times$, 87.6 tok/s) or, if a simpler setup is preferred, the vanilla 1.7B drafter ($1.41\times$, 75.5 tok/s, highest Qwen acceptance). EAGLE-3 ($1.39\times$) is a solid alternative with a smaller drafter. Avoid DFlash on Qwen: $0.63\times$ is a measured loss in this setup.
• Gemma-4-12B: use the MTP head ($2.39\times$–$2.71\times$) or DFlash with a Q4_K_M draft ($2.26\times$, 78.4 tok/s) (the quantized draft is both faster and about 1 GiB cheaper in VRAM than F16).
• Draft length: longer drafts pay on slow targets. $\beta$ generally grows with $k$, so $\alpha_{be}$ often falls from $k=5$ to $k=10$. Qwen Q4 DFlash-F16 is the exception, while Qwen Q4 DSpark changes only slightly. On Gemma, $k=10$ requires the least acceptance (0.065–0.084); on Qwen the difference is modest.
• Quantization: target Q4_K_M plus a Q4 draft is a practical operating point in this setup on 16 GB; acceptance barely moves with target quantization, so the throughput and memory trade-off is the reason to prefer Q8_0 only when its extra target precision is needed; this study does not measure task accuracy.
• Domains: chat is generally the least profitable domain, especially on Gemma; the near-zero Qwen DFlash case is approximately tied across domains. Users whose workloads are math- or code-heavy will generally see the headline speedups, while chat-heavy users will see the smallest ones on viable drafters.

The general lesson is to measure acceptance before trusting a drafter. The break-even maps provide the yardstick: if a drafter's $\alpha$ on your workload is above the $\alpha_{be}$ for your target and $k$, drafting pays.

What the break-even maps mean

The most consequential result is how generous this hardware is for several tested configurations. Pooled $\alpha_{be}$ ranges 6.5–39.9% across $k \in {5,7,10}$, and the best drafters break even at single-digit acceptance (Gemma DFlash-F16: 0.065 at $k=10$; MTP: 0.084). The Apple-Silicon reference (38.0–52.8% at $k=2$ and 77.7–90.1% at $k=4$ [@fonal2026crossfamilyspeculativedecodingpolish]) is higher at $k=4$ and overlaps the upper end of our range only at $k=2$. The comparison is directional: its sweep starts at $k=2$–4, ours at $k=5$, and the platforms, engines, and model families differ. The mechanism is visible in the OLS parameters. In the matched ctx=2,048 analysis, the target baselines are slow and memory-bound on this card, 33.5 tok/s (Gemma-Q4), 51.5 tok/s (Qwen-Q4), and 31.8 tok/s (Qwen-Q8), while the slope $\beta$ (tok/s gained per unit of acceptance) is high, 89–186 across configurations, because a draft forward pass costs a small fraction of the target's. With $TPS_{base}$ small and $\beta$ large, even modest acceptance clears the intercept. On the M2 Pro with MLX-LM the drafters are comparatively more expensive relative to a faster target, so $\alpha_{be}$ rises. Two consequences follow. First, $\alpha_{be}$ is nearly domain-insensitive: within this setup, the maps provide a hardware/engine/context diagnostic, while the achieved acceptance, rather than the fitted threshold, varies by domain. Second, the variance across drafters is driven by draft cost: the vanilla 1.7B drafter, with a flatter slope ($\beta \approx 89$–117), needs 0.328–0.399 acceptance to justify itself, several times above the best block drafters, because each accepted token buys less. Draft cost, not raw acceptance, is the lever a practitioner should optimize.

Limits of interpretation

Three caveats bound these conclusions. First, the headline speedup is the ratio of means; the mean and median of the per-prompt ratios agree with it within a few percent, with the largest divergence (7%) on Gemma DFlash, whose per-prompt distribution is mildly skewed; point estimates are therefore stable, but single-prompt speedups can vary widely. Second, the acceptance numbers rest on records we had to clean: 271 Gemma records with spurious timing sentinels were excluded, and a handful of deterministic failures reduced the Gemma sample (see Section Limitations); acceptance at draft position comes from the 2,048-context curves runs, which are deterministic and therefore identical for those prompts to the final runs, but the absolute tok/s values of the curves/ksweep runs are not directly comparable with the 8,192/16,384-context final runs. Third, the whole study is one GPU, one engine, one seed: the break-even methodology transfers, the specific numbers require re-estimation on other hardware, exactly as Fonal [@fonal2026crossfamilyspeculativedecodingpolish] predicted for his own.

Limitations

• Single GPU model. All measurements come from one RTX 4060 Ti 16 GB. The break-even maps are hardware-, engine- and context-specific diagnostics for this card and serving setup; as Fonal [@fonal2026crossfamilyspeculativedecodingpolish] notes, extrapolating to other GPUs requires re-estimating $\beta$ empirically on that platform.
• Subsampled datasets. We evaluate subsets of nine benchmarks (1,474 main / 180 acceptance-curve prompts) with a fixed seed 42, not the full sets; domain-level aggregates are therefore indicative rather than benchmark-complete.
• One engine, one decoding regime. Everything runs through llama.cpp's llama-server with non-thinking mode (--reasoning off) and greedy decoding; results may differ under sampling, thinking mode, or other serving stacks (e.g. vLLM).
• GGUF-specific quantization. We quantize with Q4_K_M/Q5_K_M/Q8_0 in GGUF; other schemes (GPTQ, AWQ, W4A16) and full precision may interact with the drafters differently.
• Unavailable drafters. Two planned cells do not exist and are reported as absent, not omitted: DSpark for Gemma-4-12B, whose public Gemma DSpark GGUFs use the draft format of pull request #25549, which llama.cpp master does not expose (no LLM_ARCH_DSPARK); and EAGLE-3 for Gemma-4-12B, for which no GGUF conversion exists.
• DFlash-on-Qwen cause not isolated. The negative result ($\alpha \approx 0.009$ with the official block-7 checkpoint converted to GGUF) is measured but not explained: we did not determine whether the failure is a conversion artifact, a vocabulary or positional mismatch, or a genuine model defect. Auditing it against the original PyTorch checkpoint is future work.
• Context-window differences. Final runs use 8,192 tokens (all Qwen3-8B and Gemma-Q8_0) or 16,384 (Gemma-Q4/Q5); the curves and ksweep runs use 2,048. Because decoding is deterministic, acceptance is identical for the overlapping prompts, but absolute throughput values are not comparable across contexts, and the per-position curves do not probe long-context behavior.
• Data exclusions. For the record: the only over-context discards in the curves/ksweep runs are two Arena-Hard-v2 prompts (arena-hard-v2-00089, -00164, both exceeding the 2,048-token context), leaving 178 of 180 prompts for Qwen and 175 for Gemma. In the final runs, deterministic failures reduced the Gemma sample: humaneval-00050 (a reproducible HTTP 500) left $n=1{,}473$ raw and $n=1{,}455$ clean records for Gemma-Q4; arena-hard-v2-00233 (over context) and -00239 (HTTP 500) left $n=1{,}472$ raw and $n=1{,}452$ clean for Gemma-Q8; and 271 Gemma records with spurious timing sentinels (tok_per_s \ge 1e5 or predicted_ms \le 0) were excluded from all computations (Gemma-Q5: 1,474 raw $\to$ 1,449 clean). The matched Gemma-Q4 ctx=2,048 target-only baseline used for the break-even fits retained 175 of 178 records after excluding three additional timing sentinels; the two over-context prompts were stored in its error log and were not retried. No Qwen records required exclusion.

Conclusion

This paper reports, to our knowledge, the first systematic evaluation of modern speculative-decoding drafters on a 16 GB consumer NVIDIA GPU. Over 26 configurations and 1,474 prompts across three domains, DSpark is the strongest Qwen3-8B drafter ($1.64\times$ at Q4, $1.93\times$ at Q8_0), EAGLE-3 transfers robustly ($1.39\times$–$1.63\times$ with acceptance nearly constant across quantization), and even a plain 1.7B drafter is competitive ($1.41\times$); on Gemma-4-12B the MTP head ($2.39\times$–$2.71\times$) and DFlash ($2.09\times$–$2.30\times$) dominate. We document two negative results with direct practical value: DFlash on Qwen fails outright ($0.63\times$, $\alpha \approx 0.009$), and DSpark's confidence gating, although it raises acceptance, reduces single-user throughput, so it is a serving feature rather than a free local speedup. Quantizing the target barely moves acceptance, and Q4 draft quantization is favorable in the tested setup. The matched-context break-even acceptance is 6.5–39.9% in the tested configurations. The upper end slightly overlaps the low end of the reported Apple-Silicon $k=2$ reference, while all tested values are below its $k=4$ range; that cross-platform comparison is directional rather than a controlled estimate of a hardware effect.

Future work proceeds along several lines: cross-family drafting with UAG on NVIDIA (deferred to a separate experiment); DSpark difficulty-aware early exit, the direction its authors suggest for making block drafts profitable on low-acceptance queries; domain fine-tuning of drafters, explicitly out of scope for this study; thinking-mode targets, which may change acceptance dynamics substantially; additional GPUs and platforms to re-estimate the break-even maps; a systematic study of quantized drafts across drafter families; and larger target models as VRAM budgets grow.

Reproducibility

The companion artifact bundle contains everything needed to regenerate every table, figure, and number in this paper. Public release of that bundle is planned together with the publication of the manuscript.

• Software. llama.cpp at master commit 07132750825a4f2d27a547cd9cdde1c6f6001885 (version 22), built with CUDA 12.8; Python tooling managed with uv (pyproject.toml/uv.lock).
• Models. experiments/runs/model-hashes.json records the SHA-256 hash of every GGUF used (targets, drafts, and quantizations). Models are the official DeepSpec checkpoints converted to GGUF and are re-downloadable; scripts/download_f3.sh lists the exact files.
• Prompts. experiments/prompts/f1-sample.jsonl (1,474 fully templated prompts, seed 42) for the final runs and a 180-prompt stratified acceptance sample (60 per domain) for the curves and ksweep runs.
• Scripts. scripts/bench_accept.py is the llama-server runner (greedy T=0, fixed seed, per-prompt JSONL, incremental resume, retries, and $\alpha$/$\tau$ extraction); scripts/analyze_final.py regenerates all tables and figures; scripts/run_final_chain.sh, scripts/run_curves_chain.sh, and scripts/run_ksweep_chain.sh orchestrate the three measurement campaigns.
• Raw data. Each run lives in experiments/runs/<id>/ and stores config.json (llama.cpp commit, GGUFs, SHA-256 hashes, sampling parameters), metrics.json, the raw results.jsonl (per-prompt tok/s, acceptance, timings), vram.json, and the server*.log files (including the per-position acc per pos lines used for the curves).

Reproduction, in three steps:

1. Build and verify. Set up the toolchain (uv sync), build llama.cpp at the pinned commit, download the GGUFs listed in model-hashes.json, and verify each file's SHA-256.
2. Measure. Run scripts/run_final_chain.sh (26 configurations), scripts/run_curves_chain.sh (20 configurations, per-position acceptance), and scripts/run_ksweep_chain.sh (21 runs, $k \in {5,7,10}$). Each script is idempotent and resumable.
3. Analyze. Run uv run python scripts/analyze_final.py to regenerate the analysis tables and the figures in manuscript/figures/.

References