Seeded from llm-lab card-bench gemma4-12b-ggml-q4km-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-12b-ggml-q4km-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-26b-a4b-aeon-nvfp4-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-26b-a4b-aeon-nvfp4-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-26b-a4b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-26b-a4b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-31b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-31b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-31b-q8-0-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-e4b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench gemma4-e4b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench granite41-30b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench granite41-30b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench granite41-8b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench granite41-8b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench nex-n2-mini-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-aeon-text-nvfp4-mtp-xs-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-aeon-text-nvfp4-mtp-xs-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-dense-stock-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-dense-stock-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-pi-tune-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q5-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q5-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q6-k-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q6-k-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-27b-q8-0-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-35b-a3b-q4-k-m-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-35b-a3b-q4-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-35b-a3b-q5-k-m-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-35b-a3b-q6-k-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwen36-35b-a3b-q8-0-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwopus36-27b-q4km-1gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench qwopus36-27b-q4km-2gpu; one row per context; median of kept runs; production stack stopped for clean measurement.


vLLM concurrency bench 2026-06-22: stock NVFP4 ModelOpt verifier of Qwen3.6-35B-A3B served alone (no DFlash draft) on the 0.23.0-cu129 container, dual-GPU tensor-parallel + expert-parallel. Single-stream decode, 3-run median, client-stream timing. Same-session llama.cpp Q4_K_M baseline 130.69 tok/s decode at this context. Production stack stopped for clean measurement.


vLLM concurrency bench 2026-06-22 (CC-PROMPT-vllm-concurrency-bench). 4-way concurrent SUSTAINED decode: tok_s_decode is the aggregate over 4 streams at 256 tok/stream forced via ignore_eos (equal-length vs llama's natural 256); EP = tensor-parallel-size 2 + expert-parallel, 0.23.0-cu129 container. NOT natural-chat throughput; single measured volley (one-shot), not a 3-run median. tok_s_prefill and power_w are null - not separately measured at concurrency 4; vram_used_gb is the served dual-card footprint at 0.90 util. Same-session llama.cpp Q4_K_M 4-way aggregate 253.44 tok/s. Re-measure script: bench/vllm_concurrent_remeasure.sh. Production stopped for clean measurement.


vLLM concurrency bench 2026-06-22: llama.cpp production baseline, 4-way concurrent SUSTAINED decode: tok_s_decode is the aggregate over 4 streams at 256 tok/stream (natural, hit the 256 cap), dual-GPU layer-split, q8_0 KV. Single measured volley (one-shot), not a 3-run median. tok_s_prefill is null - not separately measured at concurrency 4; power_w is the real 4-way concurrent measurement; vram_used_gb is the concurrent footprint. Same-session vLLM NVFP4 EP 4-way aggregate 391.79 tok/s. Production stopped for clean measurement.


Seeded from llm-lab card-bench ornith-1.0-35b-q4-k-m-1gpu; reasoning-on serve; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench ornith-1.0-35b-q4-k-m-2gpu; reasoning-on serve; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench titus-cyber-35b-q4-k-m-1gpu; reasoning-off serve; one row per context; median of kept runs; production stack stopped for clean measurement.


Seeded from llm-lab card-bench titus-cyber-35b-q4-k-m-2gpu; reasoning-off serve; one row per context; median of kept runs; production stack stopped for clean measurement.