eval/fixed_generation/README.md

Optional post-listing fixed-generation measurement

This runner sends the same short coding prompt at concurrency one, performs one warm-up request, then retains five measured responses and their API token counts.

export OPENAI_BASE_URL=http://127.0.0.1:8001/v1
export OPENAI_API_KEY="$VLLM_API_KEY"
export SMOKE_MODEL=local-qwen3-coder-next
/usr/bin/python3 -I -S eval/fixed_generation/run_benchmark.py

The ignored local output is results/server_bench.json. Its reported rate is completion tokens divided by full request elapsed time, so it includes prefill, decoding, server overhead, and local HTTP overhead. It is not a decode-only throughput number or a coding-quality benchmark. Do not use it as a launch headline or concurrency claim.