CHECKS - expected values (session 08) vs computed
Tolerance: >5% relative deviation -> warning. Speedups are compared against the mean and median of the per-prompt ratio; either one passing is enough.
| Check | Expected | Computed | Pass? | Note |
|---|---|---|---|---|
| qwen-q4 solo | 53.400 | 53.368 | OK | |
| vanilla17b | 75.500 | 75.453 | OK | |
| vanilla17b x | 1.410 | 1.41 (med 1.41) | OK | mean 0.3% off, median 0.3% off |
| vanilla17b alpha | 0.725 | 0.725 | OK | |
| eagle3 | 74.400 | 74.371 | OK | |
| eagle3 x | 1.390 | 1.39 (med 1.37) | OK | mean 0.3% off, median 1.2% off |
| eagle3 alpha | 0.440 | 0.440 | OK | |
| dspark-p0 | 87.700 | 87.617 | OK | |
| dspark-p0 x | 1.640 | 1.64 (med 1.63) | OK | mean 0.1% off, median 0.8% off |
| dspark-p0 alpha | 0.616 | 0.616 | OK | |
| dspark-p6 | 80.800 | 80.731 | OK | |
| dspark-p6 x | 1.510 | 1.51 (med 1.50) | OK | mean 0.2% off, median 0.8% off |
| dspark-p6 alpha | 0.714 | 0.714 | OK | |
| qwen-q5 solo | 46.800 | 46.800 | OK | |
| q5 eagle3 | 68.300 | 68.331 | OK | |
| q5 eagle3 x | 1.460 | 1.46 (med 1.44) | OK | mean 0.0% off, median 1.4% off |
| q5 dspark-p0 | 80.700 | 80.641 | OK | |
| q5 dspark-p0 x | 1.720 | 1.72 (med 1.72) | OK | mean 0.2% off, median 0.3% off |
| qwen-q8 solo | 32.400 | 32.345 | OK | |
| q8 eagle3 | 52.800 | 52.812 | OK | |
| q8 eagle3 x | 1.630 | 1.63 (med 1.61) | OK | mean 0.2% off, median 1.0% off |
| gemma-q4 solo | 34.900 | 34.661 | OK | |
| gemma-q4 dflash-f16 x | 2.240 | 2.09 (med 2.19) | OK | mean 6.8% off, median 2.4% off |
| gemma-q4 mtp x | 2.500 | 2.39 (med 2.47) | OK | mean 4.4% off, median 1.0% off |
24/24 checks pass.