I benchmark the GGUF quants I make on a 42-task agentic-coding suite, and I've put the results in one place:
https://huggingface.co/Myric/apex-quant-benchmarks
What it is. OpenCode drives each model through a real tool-use loop (bash/read/write/edit) against 42 tasks in four suites — 25 general, 5 hard, 9 frontier, 3 repair-in-place. The code is graded by a fresh pytest per task, partial credit. Timeouts and output-cap truncations are voided, not scored as failures (valid = 42 − void).
The table covers APEX tiers, stock K-quants, QAT and vendor GGUFs. Each row links to the actual model/quant card where one is public, and to the base/official card where the tested GGUF is local-only.
A few things that fell out:
- 42/42 is the entry bar, not a ranking. The suites saturate for competent coders. The signal is the suite where a model falls off, and output tokens at equal score.
- Wall-clock is not a proxy for parameter count. A 4B Spark-X2.5 BF16 run took 14.5 h (10 voids) while the 124B Ornith-1.5-122B finished in 1.75 h. Output tokens and timeouts drive the clock, not size.
- Voids change the story. Spark-X2.6 is 39/39 valid with 3 voids — not 39/42. Several others look worse than they are under the raw denominator.
- Aggressive APEX tiers do cost correctness at the edge. Mellum2.1 APEX 3.7 bpw matched 42/42; 3.1 bpw dropped to 40/42, both misses on the frontier.
Caveat: pass/fail is a gate; the harness does not measure code quality, and single-run rows are noisy. The repo includes the raw per-run JSON/CSV, the four plots, and a runbook for adding new results.
Repo, data and plots: https://huggingface.co/Myric/apex-quant-benchmarks