eval/fixed_generation/README.md

# Optional post-listing fixed-generation measurement

This runner sends the same short coding prompt at concurrency one, performs one
warm-up request, then retains five measured responses and their API token counts.

```bash
export OPENAI_BASE_URL=http://127.0.0.1:8001/v1
export OPENAI_API_KEY="$VLLM_API_KEY"
export SMOKE_MODEL=local-qwen3-coder-next
/usr/bin/python3 -I -S eval/fixed_generation/run_benchmark.py
```

The ignored local output is `results/server_bench.json`. Its reported rate is completion tokens
divided by full request elapsed time, so it includes prefill, decoding, server
overhead, and local HTTP overhead. It is not a decode-only throughput number or
a coding-quality benchmark. Do not use it as a launch headline or concurrency claim.