Hugging Face's logo

reyden009
/
speculative-decoding-lab

GGUF
speculative-decoding
llm-inference
benchmark
quantization
llama.cpp
Model card Files
xet
 Community
speculative-decoding-lab / experiments /analysis /tables /t6_breakeven.md
reyden009's picture
reyden009
Upload folder using huggingface_hub
d4c2896 verified
preview code
|
Raw
History Blame Contribute Delete
9.57 kB

Pooled (all domains)

Config k n baseline run baseline ctx baseline n/match baseline (tok/s) beta (slope) alpha_be CI95 R2
gemma-q4-dflash-f16 10 175 baseline-ctx2048-gemma-q4-solo 2048 175/175 33.5 185.65 0.065 0.008 0.997
gemma-q4-dflash-f16 5 175 baseline-ctx2048-gemma-q4-solo 2048 175/175 33.5 96.78 0.118 0.012 0.993
gemma-q4-dflash-f16 7 175 baseline-ctx2048-gemma-q4-solo 2048 175/175 33.5 110.32 0.144 0.009 0.996
gemma-q4-mtp 10 175 baseline-ctx2048-gemma-q4-solo 2048 175/175 33.5 162.16 0.084 0.010 0.996
gemma-q4-mtp 5 175 baseline-ctx2048-gemma-q4-solo 2048 175/175 33.5 97.16 0.107 0.014 0.990
gemma-q4-mtp 7 175 baseline-ctx2048-gemma-q4-solo 2048 175/175 33.5 106.38 0.147 0.012 0.993
qwen-q4-dflash-f16 10 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 155.12 0.160 0.060 0.858
qwen-q4-dflash-f16 5 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 142.47 0.159 0.029 0.963
qwen-q4-dflash-f16 7 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 157.21 0.157 0.024 0.974
qwen-q4-dspark-p0 10 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 152.84 0.183 0.004 0.999
qwen-q4-dspark-p0 5 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 126.65 0.187 0.012 0.992
qwen-q4-dspark-p0 7 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 150.92 0.182 0.009 0.996
qwen-q4-eagle3 10 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 179.97 0.185 0.004 0.999
qwen-q4-eagle3 5 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 124.01 0.212 0.003 1.000
qwen-q4-eagle3 7 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 136.39 0.232 0.002 1.000
qwen-q4-vanilla17b 10 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 116.94 0.328 0.008 0.996
qwen-q4-vanilla17b 5 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 89.13 0.396 0.017 0.978
qwen-q4-vanilla17b 7 178 baseline-ctx2048-qwen-q4-solo 2048 178/178 51.5 93.02 0.399 0.007 0.997
qwen-q8-eagle3 10 178 baseline-ctx2048-qwen-q8-solo 2048 178/178 31.8 149.50 0.111 0.004 0.999
qwen-q8-eagle3 5 178 baseline-ctx2048-qwen-q8-solo 2048 178/178 31.8 99.71 0.116 0.004 0.999
qwen-q8-eagle3 7 178 baseline-ctx2048-qwen-q8-solo 2048 178/178 31.8 124.64 0.110 0.004 0.999

Per domain

Config k domain n baseline ctx baseline n baseline beta alpha_be CI95 R2
gemma-q4-dflash-f16 10 math 60 2048 60 32.8 182.86 0.055 0.012 0.998
gemma-q4-dflash-f16 10 code 60 2048 60 32.5 188.60 0.065 0.010 0.999
gemma-q4-dflash-f16 10 chat 55 2048 55 35.3 183.88 0.074 0.026 0.990
gemma-q4-dflash-f16 5 math 60 2048 60 32.8 98.10 0.117 0.010 0.998
gemma-q4-dflash-f16 5 code 60 2048 60 32.5 99.30 0.122 0.011 0.998
gemma-q4-dflash-f16 5 chat 55 2048 55 35.3 95.17 0.133 0.037 0.980
gemma-q4-dflash-f16 7 math 60 2048 60 32.8 110.88 0.139 0.006 0.999
gemma-q4-dflash-f16 7 code 60 2048 60 32.5 111.80 0.141 0.008 0.999
gemma-q4-dflash-f16 7 chat 55 2048 55 35.3 109.09 0.159 0.029 0.987
gemma-q4-mtp 10 math 60 2048 60 32.8 160.93 0.073 0.012 0.998
gemma-q4-mtp 10 code 60 2048 60 32.5 162.28 0.079 0.019 0.994
gemma-q4-mtp 10 chat 55 2048 55 35.3 160.61 0.095 0.021 0.994
gemma-q4-mtp 5 math 60 2048 60 32.8 98.18 0.104 0.013 0.997
gemma-q4-mtp 5 code 60 2048 60 32.5 100.48 0.119 0.021 0.992
gemma-q4-mtp 5 chat 55 2048 55 35.3 95.44 0.120 0.034 0.982
gemma-q4-mtp 7 math 60 2048 60 32.8 107.52 0.143 0.008 0.999
gemma-q4-mtp 7 code 60 2048 60 32.5 107.20 0.142 0.015 0.996
gemma-q4-mtp 7 chat 55 2048 55 35.3 104.73 0.162 0.029 0.986
qwen-q4-dflash-f16 10 math 60 2048 60 53.5 144.15 0.187 0.158 0.726
qwen-q4-dflash-f16 10 code 60 2048 60 51.1 152.80 0.160 0.070 0.930
qwen-q4-dflash-f16 10 chat 58 2048 58 49.8 159.84 0.144 0.037 0.980
qwen-q4-dflash-f16 5 math 60 2048 60 53.5 140.21 0.176 0.018 0.995
qwen-q4-dflash-f16 5 code 60 2048 60 51.1 139.54 0.160 0.083 0.905
qwen-q4-dflash-f16 5 chat 58 2048 58 49.8 142.82 0.147 0.047 0.968
qwen-q4-dflash-f16 7 math 60 2048 60 53.5 155.25 0.172 0.015 0.997
qwen-q4-dflash-f16 7 code 60 2048 60 51.1 152.59 0.160 0.069 0.932
qwen-q4-dflash-f16 7 chat 58 2048 58 49.8 160.09 0.144 0.035 0.982
qwen-q4-dspark-p0 10 math 60 2048 60 53.5 151.19 0.192 0.007 0.999
qwen-q4-dspark-p0 10 code 60 2048 60 51.1 153.79 0.181 0.008 0.999
qwen-q4-dspark-p0 10 chat 58 2048 58 49.8 154.15 0.174 0.005 1.000
qwen-q4-dspark-p0 5 math 60 2048 60 53.5 125.79 0.202 0.024 0.988
qwen-q4-dspark-p0 5 code 60 2048 60 51.1 128.71 0.189 0.032 0.982
qwen-q4-dspark-p0 5 chat 58 2048 58 49.8 128.42 0.175 0.017 0.995
qwen-q4-dspark-p0 7 math 60 2048 60 53.5 149.53 0.196 0.023 0.990
qwen-q4-dspark-p0 7 code 60 2048 60 51.1 154.45 0.183 0.013 0.997
qwen-q4-dspark-p0 7 chat 58 2048 58 49.8 154.72 0.173 0.005 1.000
qwen-q4-eagle3 10 math 60 2048 60 53.5 180.15 0.196 0.011 0.998
qwen-q4-eagle3 10 code 60 2048 60 51.1 183.49 0.183 0.006 0.999
qwen-q4-eagle3 10 chat 58 2048 58 49.8 178.32 0.176 0.007 0.999
qwen-q4-eagle3 5 math 60 2048 60 53.5 123.40 0.228 0.006 0.999
qwen-q4-eagle3 5 code 60 2048 60 51.1 124.52 0.210 0.006 0.999
qwen-q4-eagle3 5 chat 58 2048 58 49.8 124.00 0.198 0.004 1.000
qwen-q4-eagle3 7 math 60 2048 60 53.5 136.00 0.247 0.007 0.999
qwen-q4-eagle3 7 code 60 2048 60 51.1 137.57 0.230 0.004 1.000
qwen-q4-eagle3 7 chat 58 2048 58 49.8 135.81 0.220 0.003 1.000
qwen-q4-vanilla17b 10 math 60 2048 60 53.5 117.82 0.343 0.006 0.999
qwen-q4-vanilla17b 10 code 60 2048 60 51.1 117.08 0.329 0.022 0.990
qwen-q4-vanilla17b 10 chat 58 2048 58 49.8 115.44 0.312 0.013 0.997
qwen-q4-vanilla17b 5 math 60 2048 60 53.5 85.97 0.398 0.010 0.997
qwen-q4-vanilla17b 5 code 60 2048 60 51.1 85.82 0.381 0.039 0.961
qwen-q4-vanilla17b 5 chat 58 2048 58 49.8 83.93 0.381 0.036 0.971
qwen-q4-vanilla17b 7 math 60 2048 60 53.5 93.31 0.418 0.005 0.999
qwen-q4-vanilla17b 7 code 60 2048 60 51.1 93.36 0.400 0.021 0.989
qwen-q4-vanilla17b 7 chat 58 2048 58 49.8 91.96 0.380 0.011 0.998
qwen-q8-eagle3 10 math 60 2048 60 30.8 148.79 0.105 0.011 0.998
qwen-q8-eagle3 10 code 60 2048 60 32.2 151.31 0.115 0.007 0.999
qwen-q8-eagle3 10 chat 58 2048 58 32.3 148.96 0.114 0.005 1.000
qwen-q8-eagle3 5 math 60 2048 60 30.8 99.08 0.105 0.009 0.999
qwen-q8-eagle3 5 code 60 2048 60 32.2 101.20 0.124 0.009 0.999
qwen-q8-eagle3 5 chat 58 2048 58 32.3 99.23 0.121 0.005 1.000
qwen-q8-eagle3 7 math 60 2048 60 30.8 123.39 0.102 0.012 0.998
qwen-q8-eagle3 7 code 60 2048 60 32.2 125.76 0.115 0.008 0.999
qwen-q8-eagle3 7 chat 58 2048 58 32.3 124.54 0.114 0.005 1.000

Comparison with M2 Pro (paper #32, Bielik et al., cross-family)

Paper #32 fits TPS = a + b*alpha by OLS and defines alpha_be = (TPS_base - a) / b (its b is our beta, the 'recovery rate'). Published values (Table 5, ranging across drafters and datasets) are k=2: 38.0-52.8% and k=4: 77.7-90.1%. The compact 40-77% range summarizes k=2..4. Our ksweep starts at k=5, so the comparison is directional: the k=10 RTX range is below the reported k=2 band, while the upper ends at k=5 and k=7 slightly overlap its lower edge.

Referencia alpha_be rango
M2 Pro k=2 38.0-52.8%
M2 Pro k=4 77.7-90.1%
Nuestro k=5 (n=7 configs) 10.7-39.6%
Nuestro k=7 (n=7 configs) 11.0-39.9%
Nuestro k=10 (n=7 configs) 6.5-32.8%

alpha_be > 1.00 = sin break-even alcanzable por OLS; CI95 por delta method sobre la covarianza OLS.