"""Serving-economics panel for the Polygen x Qwen2-VL Space. A self-contained HTML fragment (inline CSS, no JS, no external assets) rendered into a Gradio gr.HTML block. The requests-per-GPU capacity bars play once whenever the fragment is (re)rendered; `app.py` re-renders it on each "Run comparison" click by passing a changing `data-run` nonce, so the animation replays on the user's action rather than looping on its own. The figures are measured: NVIDIA H200, Qwen2-VL-7B, vLLM continuous batching. Source: `tools/runpod_results/results_serving_throughput_7B.json` (throughput, T=2, cached visual tokens); storage from the SDK tier registry. This panel makes a throughput and storage claim only -- no per-request latency / wall-clock claim. """ # ruff: noqa: E501 (self-contained HTML fragment; inline CSS lines do not wrap) # `data-run="__NONCE__"` is replaced per render (not str.format, because the # inline CSS is full of literal braces). A different nonce makes the gr.HTML # value differ, so Gradio replaces the DOM and the play-once animation restarts. _PANEL = """

Same answer. More requests per GPU.

Polygen compresses Qwen2-VL's visual tokens before the language model reads them. At the T=2 operating point (accuracy +7.4pp above baseline, a statistical tie with the best tier), the shorter visual sequence is a smaller KV cache and less attention compute, so one GPU serves more concurrent requests.

Requests per GPU · T=2 · precomputed visual tokens

Baseline 1x

Polygen ~3.7x

Per GPU, polygen sustains about 3.7x (nearly 4x) the requests per second, so the same load runs on roughly a quarter of the GPUs. For precomputed / cached visual tokens (retrieval, repeated queries) -- the pattern the RAG demo shows.

~3.7x

Requests per GPU

roughly a quarter the GPUs, T=2, cached

~5x

Smaller stored representation

visual-token cache

+7.4pp

Accuracy vs baseline

ScienceQA, T=2 (above baseline)

Tiers, chosen per workload. T=3 is the accuracy peak (+8.4pp on ScienceQA); T=2, shown here, is the compression peak (+7.4pp, a statistical tie) where the throughput gain is largest; lower tiers trade accuracy for more compression.

What this is

The gain is throughput and storage: a shorter visual sequence is a smaller KV cache and less attention compute, so a GPU serves more concurrent requests and the cached representation is smaller. Measured for precomputed / cached visual tokens (retrieval, repeated queries) under vLLM continuous batching on an NVIDIA H200 with Qwen2-VL-7B.

""" def render_serving_story(nonce: str = "0") -> str: """Return the serving-economics panel HTML fragment. Args: nonce: A value that varies per render so a Gradio `gr.HTML` update sees a different string, replaces the DOM, and restarts the play-once bars. Returns: An HTML fragment (no doctype) suitable for a `gr.HTML` block. """ return _PANEL.replace("__NONCE__", str(nonce)) if __name__ == "__main__": # Regenerate the standalone browser preview from this single source of truth. with open("serving_story.html", "w", encoding="utf-8") as _fh: _fh.write('\n\n') _fh.write(render_serving_story("preview")) print("wrote serving_story.html")