Publish runtime v2 optimization, final comparisons and figures (weights unchanged)

.gitattributes

@@ -46,3 +46,5 @@ tokenizer.json filter=lfs diff=lfs merge=lfs -text
 examples/odyssey/odyssey-preview.gif filter=lfs diff=lfs merge=lfs -text
 figures/performance-summary.png filter=lfs diff=lfs merge=lfs -text
 figures/quality-summary.png filter=lfs diff=lfs merge=lfs -text
+figures/runtime-v2-performance.png filter=lfs diff=lfs merge=lfs -text
+figures/runtime-v2-quality.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -33,21 +33,73 @@ so MTP speculative decoding works, validated on an NVIDIA DGX Spark (GB10, SM121
 - **Weights and complete model files:** [https://huggingface.co/gitcommit90/SenseNova-Loop-9B-Preview-NVFP4](https://huggingface.co/gitcommit90/SenseNova-Loop-9B-Preview-NVFP4)
 - **Runtime, quantization recipe, evaluation harness and charts:** [https://github.com/gitcommit90/sensenova-loop-9b-nvfp4-dgx-spark](https://github.com/gitcommit90/sensenova-loop-9b-nvfp4-dgx-spark)
 
-This is the **October 30 functional-release baseline**, not the later speed-optimization work.
+**Updated October 31: runtime v2 is included below, alongside the preserved October 30 functional-release baseline.**
 Weights live on Hugging Face; the GitHub companion does not duplicate the 9.38 GB checkpoint.
 
-## Measured results at a glance
+## Runtime v2 — final optimization results (October 31)
 
-[![Measured single-user speed, TTFT and aggregate concurrency throughput](figures/performance-summary.png)](figures/performance-summary.pdf)
+**Same 9.38 GB NVFP4 weights; new runtime kernels and a smaller MTP draft vocabulary.**
+No retraining, recalibration, reduced loop passes, or changed KV/state precision. MTP is supported in all profiles.
+The target still verifies over the full vocabulary. Only greedy decoding was evaluated.
 
-[![Small-suite quality preservation and paired uncertainty](figures/quality-summary.png)](figures/quality-summary.pdf)
+[![Final runtime v2 single-user decode, concurrency and TTFT](figures/runtime-v2-performance.png)](figures/runtime-v2-performance.pdf)
 
-Figures are regenerated directly from the checked-in evaluation JSONs with
-[`figures/plot_benchmarks.py`](figures/plot_benchmarks.py), following the publication-figure conventions of
-[figures4papers](https://github.com/ChenLiu-1996/figures4papers).
-PNG previews, vector PDF/SVG exports, and plotting dependencies are included.
-**Aggregate throughput is not per-user decode speed.** Quality scores are small-subset regression measurements,
-not full-benchmark scores or a comparison against Qwen.
+| Final matched measurement | Frozen NVFP4 + MTP | v2 exact + MTP | v2 fast + MTP |
+| --- | ---: | ---: | ---: |
+| Single-request decode, ~1K input (tok/s, median) | 12.4 | **27.3** | **28.9** |
+| Single-request code decode, ~90 input (tok/s, median) | 16.0 | **33.4** | **35.9** |
+| Single-request document decode, ~4K input (tok/s, median) | 14.3 | **30.6** | **33.3** |
+| Eight-request aggregate, ~1K input (tok/s) | 64.3 | **117.2** | **119.4** |
+| Eight-request code aggregate (tok/s) | 97.4 | **172.7** | **196.7** |
+| Single-request TTFT, ~1K input (seconds, median) | 0.74 | 0.51 | **0.32** |
+
+Exactly 256 output tokens; unique prompts; no prefix-cache hits; zero errors in these final performance runs.
+**Aggregate throughput is combined across requests, not each user's speed.** These are the final matched runs,
+not the higher preliminary screening peaks. Full sample counts, token lengths and raw measurements:
+[`runtime-v2/RESULTS.md`](runtime-v2/RESULTS.md) and [`runtime-v2/runs/`](runtime-v2/runs/).
+
+### Which profile should I use?
+
+- **`MHC_MODE=exact` — recommended conservative profile:** fused Sinkhorn with reference operation order;
+  cuBLAS routing projection and reference merge retained. Reference routing math matched bitwise in differential
+  tests; the 12-case final feature check matched the frozen output. This is not a universal promise of identical
+  generations under every batch/hardware configuration.
+- **`MHC_MODE=fast` — faster, numerically different:** also fuses routing projection/merge. FP32 is preserved,
+  but summation order changes and greedy text can diverge. Choose this explicitly if that trade-off suits you.
+  The supplied v2 launcher's inherited default is `fast`; commands below explicitly select `exact`.
+- **`runtime/` — frozen v1:** preserved for reproduction and rollback. Both v2 profiles use the same pinned image.
+
+### Quality: report the losses and uncertainty, not just the speedup
+
+[![Runtime v2 quality results and paired uncertainty](figures/runtime-v2-quality.png)](figures/runtime-v2-quality.pdf)
+
+On the same 300-item suite, frozen v1 scored **251/300 (83.7%)**, v2 exact **249/300 (83.0%)**,
+and v2 fast **247/300 (82.3%)**. Paired-bootstrap 95% delta intervals vs frozen:
+exact **−0.67 pp [−3.00, +1.67]**; fast **−1.33 pp [−4.33, +1.67]**.
+These small, noisy subsets do **not** establish losslessness or full-benchmark equivalence.
+
+**Fast's first MMLU-Pro run fell from 70.0% to 58.3% (−11.7 pp)** and truncations rose from 14 to 20 across the suite.
+Those exceed the original first-run per-task and ref+3 truncation tripwires. Fast MMLU-Pro reruns scored 71.7% and
+65.0%; frozen reruns scored 70.0% and 65.0%. All runs are retained. Variability and the token budget complicate
+interpretation; they do not erase the first-run regression. See [`RESULTS.md`](runtime-v2/RESULTS.md).
+
+The improvements come from **fused FP32 mHC/Sinkhorn kernels** and a **65,536-token MTP draft shortlist** selected
+from training-split corpora. The draft is restricted; the verifying target is not. The exact kernel preserves all
+20 configured normalization iterations. No learned model weights changed.
+
+Source, numerical tests, optimized launcher, token-map builder, and experiment ledger are in
+[`runtime-v2/`](runtime-v2/). Numerical tests and final feature/quality/performance results are included.
+Figures follow [figures4papers](https://github.com/ChenLiu-1996/figures4papers), with
+[`figures/plot_runtime_v2.py`](figures/plot_runtime_v2.py), PNG previews and vector PDF/SVG exports.
+
+## Original quantization baseline (October 30; retained)
+
+[![Original BF16 versus NVFP4 performance baseline](figures/performance-summary.png)](figures/performance-summary.pdf)
+
+[![Original quantization quality comparison](figures/quality-summary.png)](figures/quality-summary.pdf)
+
+These older figures compare BF16 to the **original v1** NVFP4 runtime, not runtime v2.
+Their source script remains [`figures/plot_benchmarks.py`](figures/plot_benchmarks.py).
 
 ## Provenance
 | Item | Value |
@@ -65,6 +117,7 @@ not full-benchmark scores or a comparison against Qwen.
 - `mtp.*` (MTP head) is included in `model.safetensors` as BF16 (upstream shipped it as a separate FP32 `mtp.safetensors`).
 - Tokenizer, processor configs, chat template, generation config and `modeling_sensenova_loop.py` are byte-identical
   to the pinned upstream revision (see `SHA256SUMS`).
+- `runtime-v2/` adds fused routing kernels and a 64K draft token map, with no checkpoint weight changes.
 - `runtime/` contains SGLang source files from the open SGLang PR
   [#47281](https://github.com/sgl-project/sglang/pull/47281) (Apache-2.0, sgl-project) plus two small patches of ours
   (`runtime/patches/`), needed to serve this checkpoint.
@@ -125,7 +178,7 @@ Tool calls: 100 % well-formed for both. 0 request errors. Truncations at the max
 With MTP enabled the NVFP4 model scored GSM8K 96.3 % (n=80), HumanEval 80.0 % (n=40), tools 90 % (n=20); MTP is
 lossless in principle at temperature 0, small differences come from batch-dependent numerics.
 
-## Performance (DGX Spark GB10, SGLang, same flags for all rows)
+## Original v1 performance (DGX Spark GB10, SGLang, same flags for all rows)
 Prompt 1,051 tokens, exactly 256 output tokens (`ignore_eos`), greedy, unique prompts (no prefix-cache hits),
 `--mem-fraction-static 0.45`, CUDA graphs on. No MTP rows: 16 requests per level; MTP rows: 8 requests per level.
 Aggregate output tokens/s (per-request decode tok/s p50 in parentheses):
@@ -171,13 +224,13 @@ python3 -m venv .venv-download
 .venv-download/bin/hf download gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 --local-dir models/SenseNova-Loop-9B-Preview-NVFP4
 # Verify the downloaded release before serving:
 (cd models/SenseNova-Loop-9B-Preview-NVFP4 && sha256sum -c SHA256SUMS)
-bash runtime/serve_sglang.sh models/SenseNova-Loop-9B-Preview-NVFP4
+MHC_MODE=exact bash runtime-v2/serve_sglang.sh models/SenseNova-Loop-9B-Preview-NVFP4
 ```
 
 **From a complete Hugging Face snapshot** containing `model.safetensors`:
 ```bash
 # from the snapshot root
-bash runtime/serve_sglang.sh .                 # NVFP4 W4A4 + MTP (NEXTN, 3 steps), 127.0.0.1:30080
+MHC_MODE=exact bash runtime-v2/serve_sglang.sh . # conservative optimized W4A4 + MTP, 127.0.0.1:30080
 curl -s http://127.0.0.1:30080/v1/models
 python3 runtime/probe.py http://127.0.0.1:30080/v1 sensenova-loop-9b-nvfp4 [image.jpg]   # text / thinking / tool / image smoke test
 # options: --no-mtp ; W4A16: SGLANG_DISABLE_SILU_FP4_QUANT_FUSION=1 runtime/serve_sglang.sh . --fp4-gemm-backend marlin
@@ -185,14 +238,24 @@ python3 runtime/probe.py http://127.0.0.1:30080/v1 sensenova-loop-9b-nvfp4 [imag
 OpenAI-compatible API; thinking is toggled with `chat_template_kwargs: {"enable_thinking": false|true}`; tools use the
 `qwen3_coder` parser (the model's XML `<tool_call><function=…>` format).
 
+### Profile selection and rollback
+```bash
+MHC_MODE=fast bash runtime-v2/serve_sglang.sh /path/to/model  # fastest; see numerical/quality caveats
+MHC_MODE=exact bash runtime-v2/serve_sglang.sh /path/to/model # conservative optimized
+bash runtime/serve_sglang.sh /path/to/model                 # frozen v1 rollback
+```
+The launcher replaces only its named model container. No extra quantization step is required.
+Benchmark reproduction is documented in [`runtime-v2/bench/README.md`](runtime-v2/bench/README.md).
+
 ## Limitations
 - Upstream is a *Preview* model; all upstream limitations and intended-use terms apply.
 - The serving path depends on an **unmerged** SGLang PR plus our patches, pinned to one nightly image; it was tested
   only on GB10. Pipeline/tensor parallelism and other GPUs were not tested.
 - Evaluation is small (300 items, thinking off, contexts ≤ ~3 K tokens). Long context (> 32 K), video input, thinking-mode
   quality at scale and non-English quality were not evaluated. Only greedy decoding was benchmarked.
-- The mHC Sinkhorn routing runs as many small FP32 PyTorch kernels and dominates a large share of GPU time in both BF16 and
-  NVFP4; a fused kernel would likely improve speed further (not attempted).
+- The frozen v1 mHC path uses many small FP32 kernels. Runtime v2 fuses those operations; the remaining profile is more GEMM/output-head bound.
+- Runtime v2 fast is not bitwise equivalent to v1 and did not pass every original first-run quality tripwire. Exact is the conservative option.
+- The draft vocabulary shortlist was built from training corpora; no broad sampling-equivalence or full-context quality study was performed.
 - In this SGLang build, `--fp4-gemm-backend marlin` crashes unless `SGLANG_DISABLE_SILU_FP4_QUANT_FUSION=1` is set
   (upstream bug: the fused SiLU+FP4-quant path is enabled regardless of backend).
 - Quantization may change behaviour on inputs unlike the calibration mixture.
@@ -200,7 +263,8 @@ OpenAI-compatible API; thinking is toggled with `chat_template_kwargs: {"enable_
 ## Files
 - `model.safetensors`, `config.json`, `hf_quant_config.json` — NVFP4 checkpoint (+ BF16 MTP head, FP32 mHC).
 - Upstream (unchanged): `tokenizer*.json`, `chat_template.jinja`, `*processor_config.json`, `generation_config.json`, `modeling_sensenova_loop.py`.
-- `runtime/` — tested launcher, SGLang overlay and patches, probe script, compatibility notes.
+- `runtime-v2/` — optimized profiles, Triton source, draft token map, final results and differential tests.
+- `runtime/` — original tested launcher, SGLang overlay and patches, probe script, compatibility notes.
 - `quantization/` — quantization/verification scripts, stats, structural verification report, environment lock, calibration manifest.
 - `evaluation/` — quality gates, eval summaries, paired comparison, perf JSONs, kernel trace summary, reference checks.
 - `SHA256SUMS` — checksums of every file.

SHA256SUMS

@@ -1,6 +1,6 @@
 8951a99119cdcd06c699d91ccc5db0a40b595961041a5ec2e6103b88112757e8  .gitattributes
 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  LICENSE
-60e1c4899e3a776a61e0223abb3ed02317be0fae58d3beb24d86dfbc981fd408  README.md
+f7e05b28104f305f9fe6bf5cd25bc17e67c72cd8a7b0b29ea511f5f7f3a966bc  README.md
 a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715  chat_template.jinja
 5bec5643138a09f2917a0b7471a673baecfbf85f59af49cc048f4318f32e7a3d  config.json
 ccdb9203010fc0534d37b0b475d85f088f5e62568d3cbd76229af4d8e50ffb28  evaluation/QUALITY_GATES.md
@@ -33,15 +33,22 @@ ec88f909edcbf225936bbeac88ede9312520593eb53db976d5079d9dd6c45ed1  evaluation/sum
 c6d75330180b2d1bd5f2dda574afeee8844e18e44c46ba0f3a1128eb052d2326  evaluation/summary_sglang_nvfp4_v1.json
 aff7bcc52ad3e83e6394a8d31e733d8b5725d213836beb98824c397e726c66c6  evaluation/summary_sglang_nvfp4_v1_marlin_w4a16_gsm8k.json
 944d3ce06cffbfb7a8e10c1af186aaa62a196e6b96cd8c3a1e6800e5d998fa70  evaluation/summary_sglang_nvfp4_v1_mtp.json
-e6df880d4632abe34376bbf9d349ff694129e21067f4e58a757919715804a8b9  figures/README.md
+cc0be931111ecb00e1ceef1d24adfa6cd1e0af04167ce6d3a999a0cc08883d89  figures/README.md
 47eecb7dab94c2cbe44756977f2e4e81380a059f1661060c0ac5e0909d9b1c24  figures/performance-summary.pdf
 7012b65a258263617bcd3a9d57e93c4cb24836d9da35a69d723a0f6ef6416d68  figures/performance-summary.png
 975894bd93a506565d81573ecfc75b9e6202406c2f21940750ec1ad7c9635e8c  figures/performance-summary.svg
 058f67de2a71ee67cf48f4976b69d353cfaea0607bb2d91a20e33774c35c2df7  figures/plot_benchmarks.py
+277d66c8e57b9fe00c599a526d378e798d55be661d1a29da9ee6fb4eaaa691e0  figures/plot_runtime_v2.py
 ceca3cd14f612774aad2f9f076a1d77177bd6f6090c41f8a3ca985318ada3793  figures/quality-summary.pdf
 761cb0de5f120ec2934b7b900b0d4961464ba24686b4175095289e9d245b0a04  figures/quality-summary.png
 0315f8e622b4b9cb9ca36ed3718403ba64ea4f8f3df582853fd3e4c0b4fe15c5  figures/quality-summary.svg
 472fcc1fa6325424ea016e5fb386728d1bf661e21582f7e2e829387053eb9a0e  figures/requirements.txt
+a2e2fe69b92b12eaedab329f96b8434fb5f42cbfdbefb7a16742be9dd62e0906  figures/runtime-v2-performance.pdf
+6b0145854cf885771b77dd327e3d5cb0a742b04c9c744bf24006ba98f4569313  figures/runtime-v2-performance.png
+4de7bc0cd7d6f953bf1c864b0a66dbda1ef425d18c5f98d2ca14396df8e61f98  figures/runtime-v2-performance.svg
+8c26bbb48e9646822b64e2624b94cb4b4502c2d0987e74b6eff0b2cec17f4a09  figures/runtime-v2-quality.pdf
+c66a6044c2160795ef18de6906060864ed793adfe08a27a8fce20e5f59306695  figures/runtime-v2-quality.png
+d6422b140d441f9028b53b6f6035cb3296a4c55f96b1ecbfc249e0644136e6ab  figures/runtime-v2-quality.svg
 f288bb4281580138395c6acc7b3adb62f45c21530bc0708cb2cf9edf093f6a4f  generation_config.json
 b197af747a1f072419a8e7e2db3252d9aeea84d21f65db37f748430ab7f83bfd  hf_quant_config.json
 1e420ea69111b7bb3c7c529cdbbb4ff4e6ebb531c0cd2be5c634cd88cd40eaca  model.safetensors
@@ -57,6 +64,145 @@ dc54da6f2e37136c3172779d86ae0134a353942f341eaa8934d3000a01f34682  quantization/b
 4d71bc0045ee80c45f5eef89e5e721e064e2505020b327942501b017daf0fdf8  quantization/quantization_stats.json
 723419a376e5f2339f4f0d4f9376b16ea2d5359e54659ccb85c2bb2d5c3028fd  quantization/requirements-quant.lock
 be632c83213d44b4a18f367c59fcb55f5d0d738b8c67f64ccc61df3c6a9fb598  quantization/verify.json
+32dae3052f331ee34d628ef535709b301259a45df7c7522c4d35dcf49873f00b  runtime-v2/.gitignore
+4aa348318e825643f1edb8b4516862de1dca0cc6c56960b10619676a03b6a5ac  runtime-v2/LEDGER.md
+f43e21163bea7f9f6e9adcd73c8ac9914fc4b4988f0ef917b2071792c94879f4  runtime-v2/README.md
+5373b331a9ad0ca78e3bf362f4d3974808e8f820366781c76318f8abfa23c9d0  runtime-v2/RESULTS.md
+8d58f6918095d93a67b83b267ee51e48e5895b2d54e30ccdd5d2b7c7e60128ac  runtime-v2/bench/README.md
+4b5700057f8bbabf7a42e40fcf361710fb0a1976a9353abbdc1d79bc38ef564e  runtime-v2/bench/fp4_gemm_bench.py
+95a4ed870e9bb68e143fa459a55eb84963617ba5f0cda023a35f6e08082a17f3  runtime-v2/bench/prof_drive.py
+b95119c8ad457bf182319d4a8e10cc67d278efa9c5458a0caa2df4d941372e44  runtime-v2/bench/report_tables.py
+27e569ec5be6157c31e81ff6d09c62871542e82b1a397c95961a8a347e60de65  runtime-v2/bench/screen.py
+d548ad5c9741c59e51e32a78a39c305594a62a12bf13b06b06195b6f5d1f5c2f  runtime-v2/bench/trace_graphs.py
+8117abdb9b399c44a57ee0a5ad32831df0d83fc68880e60e24cb9ca65fee74fa  runtime-v2/bench/trace_summary.py
+5bb587d19dce89cbce317a812a17e30891150afca578f60aa94f1a77b4070c46  runtime-v2/bench/trace_summary_lib.py
+2f1d8b4d676321dce894f03f97fa17d99842a73054605d9e7f70c3f8426bc806  runtime-v2/bench/wait_ready.sh
+e00d2fafe780fcf4c04ab52b040e43ec7143c7bd6e070637563f2724b59888d1  runtime-v2/overlay/python/sglang/srt/configs/__init__.py
+bdc7fa2055b799ce07441a5d499fafa554b7b797d4c4a24fd251b846b51260fd  runtime-v2/overlay/python/sglang/srt/configs/model_config.py
+eb6df8f37ac06e6f439dd24f687e5bf1078599ade28acaa20d8799bf77c7e057  runtime-v2/overlay/python/sglang/srt/configs/sensenova_loop.py
+b3b4959f2336d1afdf4f0353a0a0eecfc38495226912022dd862489cbb532c98  runtime-v2/overlay/python/sglang/srt/layers/rotary_embedding/mrope_rope_index.py
+68bc3cfe9749f90ecc817ccede7fdbd9167684e902637dbec0b738a9d7ea7e25  runtime-v2/overlay/python/sglang/srt/layers/sensenova_mhc_fused.py
+87ccee5659fb8d95f7f54a038d7d0f71ba8afab04684c72c6bd1556e31d119ce  runtime-v2/overlay/python/sglang/srt/models/sensenova_loop.py
+7fc07bd18b974aa8f54013f2bb39536a65d32ad0cc1a1df2d239ead0dc136754  runtime-v2/overlay/python/sglang/srt/multimodal/processors/qwen_vl.py
+68e1076237ed67adfa06530139d79df0783a2519620443a58cce94c5143d56c4  runtime-v2/overlay/python/sglang/srt/utils/hf_transformers/common.py
+d93e6342f9487567a7847443eb47b41701b4510474cd5664c8dc51b37eb6f6fd  runtime-v2/patches/model_config_mtp_draft.patch
+029fa25bb4ea5a16c9fce8de412e7ea6481fc540508c63bdd629026b9a56f926  runtime-v2/patches/sensenova_loop_quant.patch
+cfd86c1b46c3ac7592ed50ec5641d36e1a69025791bd986907b3c6b54c31a4b6  runtime-v2/patches/v2_mhc_fused_router.patch
+aeabae0b23eb5eea12d4d5c9fc57b115bbf702d90019a61cb1bdd6b912cde07e  runtime-v2/runs/chain1.log
+8eac091fab019b0d871b8c20c4ef240f758ef512fefa7bdb2e5f3dd6267df276  runtime-v2/runs/chain2.log
+76984e7fc55c0065fb8c9d1646b4f61de0c24b9700fd21348fce82f1fffa5a22  runtime-v2/runs/chain3.log
+f82fb0102b5a9cea9aa39c720d35781128a7d4330d26e808a101ecbaef4c617a  runtime-v2/runs/final_chain.log
+3cb5e44353508026e19a34c728a0cfd62b234a1de02bb2e501bd904b856be246  runtime-v2/runs/final_frozen_v1/eval.log
+ae09e6590b834fe3d2eb4d0efbe1710f745fb3a2dfb894111035a81c1a0273bf  runtime-v2/runs/final_frozen_v1/eval/compare_vs_ref.json
+c7b22c9374b33ddbf580521917bb226e1535a28464d682690d4c0b4b74d0af65  runtime-v2/runs/final_frozen_v1/eval/raw_gsm8k.jsonl
+6c24a6a63e53f694029de6fd48bf4b0a31e8b3a8d4978ff0c1c2b1274627402f  runtime-v2/runs/final_frozen_v1/eval/raw_humaneval.jsonl
+d6102c5faacfc6d6460a076c4a4e73c09f4c3fc4b1f69c26694fa473c63a200c  runtime-v2/runs/final_frozen_v1/eval/raw_ifeval.jsonl
+723bc26360e96ea005eaf282c6c400104ad26a3df22a13328fbdc35b07164549  runtime-v2/runs/final_frozen_v1/eval/raw_mmlu_pro.jsonl
+37b7b5b550d8f3ced41d5fc0c33b3268b8112442412bc7e9216d58ccb0c2797f  runtime-v2/runs/final_frozen_v1/eval/raw_tools.jsonl
+fe1c19237fa5ab99b626d44830afb7dc7c8bc1ecacb1d5b6ac09729f3e6be0cd  runtime-v2/runs/final_frozen_v1/eval/raw_vqa.jsonl
+233917924301edfed636e906a80fbc7180b0c8f1465fd1df32b2acd6db9159e3  runtime-v2/runs/final_frozen_v1/eval/summary.json
+138edee8b67c2f8cdc9fbe34b48067ec99dc9b2b20729611e6a856af6f74af71  runtime-v2/runs/final_frozen_v1/eval_compare.txt
+69bf0e6859cd641d10341d0575012918944b49c59b9dada302324cb6b6df1476  runtime-v2/runs/final_frozen_v1/gpu_after.txt
+03ff9c79c916f05ec992381c0f635426e32cbe907d41f91d506d4cb1ffdf666c  runtime-v2/runs/final_frozen_v1/gpu_before.txt
+a3e9d1074d6b2b2d6af2eff5f1c35761e46067243a45c9d318562273dfea5417  runtime-v2/runs/final_frozen_v1/greedy.json
+5a29fa8c0bafdc6dcb66c6172008d892cb8144ec5066a2e79d3512b870164c37  runtime-v2/runs/final_frozen_v1/greedy_diff.txt
+3d3193ebc85d9703ceca209b0e9b2b8bdf8729dfd40bfb4ca36e1d6b05738ee6  runtime-v2/runs/final_frozen_v1/mem.txt
+d835c86c49b359ee814f608bc268ad1c843426bd4722d9b048ac3a7f7ec45579  runtime-v2/runs/final_frozen_v1/perf_matched.json
+e292a5bcb39f3793fbf94d7ca6bcaf80f7bd3427e8522fc9ef463646563dbacb  runtime-v2/runs/final_frozen_v1/perf_matched.log
+c9501741babdc2cbfc877be168b64621cfa1cc65bc4606ed93079b7d9bdbe71c  runtime-v2/runs/final_frozen_v1/screen_code.json
+6f6459cd295bbe224275e829e2379017f8e7a028a3f03e6de3b67b5ee14ac00d  runtime-v2/runs/final_frozen_v1/screen_doc.json
+4e2ead04c10b476e955991de1d1527dac816545fea5bac42042cdb12d0453ff4  runtime-v2/runs/final_frozen_v1/startup.txt
+1ab6b379ed646e189a284b44408989a79637a2c6f478aa7c03e47dc6b1418f8f  runtime-v2/runs/final_v2_exact/eval.log
+1be03124a62cea4fe99ea8dae682947570adf8e004061bec0a603bd04935ac42  runtime-v2/runs/final_v2_exact/eval/compare_vs_ref.json
+7193b8d845182d28f4f3f71cad94e3db146c65315384ac85b29e39c094e00ce7  runtime-v2/runs/final_v2_exact/eval/raw_gsm8k.jsonl
+0017ed83f0e36cd6ec678eb1158efae0bbf850715be242e11ba41fa2fc6a3b5d  runtime-v2/runs/final_v2_exact/eval/raw_humaneval.jsonl
+b2b5ea9fed29ff5317ee06aa4dee888cb2c60a64d33f188f49b0126797643ed5  runtime-v2/runs/final_v2_exact/eval/raw_ifeval.jsonl
+107687c7b517761160a2d8b35ccad29a10ac5667bdbbb3aff60fd20aeda19a92  runtime-v2/runs/final_v2_exact/eval/raw_mmlu_pro.jsonl
+975732afa2c61c16dc6e8877c72414adb4f97b2aa778c184ac32f9e33e56b2c1  runtime-v2/runs/final_v2_exact/eval/raw_tools.jsonl
+84a9e2908ce2dacc1b863e189ef1223f60f25256f46d3dc8ea0a20886c63106a  runtime-v2/runs/final_v2_exact/eval/raw_vqa.jsonl
+a50e1b698802fcdc0a1bac9e68a2e73a5fd57c53cdbe1e7bf4767cd776e284fe  runtime-v2/runs/final_v2_exact/eval/summary.json
+c0e7cb0b03610f45de18bd1ff73e7f9f5c766b1d687229857c69230f3e9e6596  runtime-v2/runs/final_v2_exact/eval_compare.txt
+ac7f0ee9c9c9d76b3f78530de8078f14f12b180bfb4bb87a8cb906101e5fc4de  runtime-v2/runs/final_v2_exact/gpu_after.txt
+33d8f8e097ac88d774be66b5375867309baf2cc1e71b7ddfa083499e04669270  runtime-v2/runs/final_v2_exact/gpu_before.txt
+5587c74db231fdb0f009e1fc521673c9464c8d16aa60df08d5729544315786fa  runtime-v2/runs/final_v2_exact/greedy.json
+5c6c788f0353dd10fb95bbd2cac62af3a774514b3d94acc6f0177f06e17f13cf  runtime-v2/runs/final_v2_exact/greedy_diff.txt
+4fb42769c273405b3cca964c55857838ec5ae5f505be5aa6608f9873d40abc1a  runtime-v2/runs/final_v2_exact/mem.txt
+0765ac69ab713c826222911c7a3b348ce6d63622b0990a3d858d6792bb01803a  runtime-v2/runs/final_v2_exact/perf_matched.json
+8d920e9d0edeeb83fc6f0500dff75a12225bc6d76c3ce59d4d4ceb593cb390b7  runtime-v2/runs/final_v2_exact/perf_matched.log
+83ec9f38a72972d41fcd91d40dd7feb13c2dd7a3925f92653b03b93ab750e0c2  runtime-v2/runs/final_v2_exact/screen_code.json
+c4e0990219587fe88a78488d08e32fe76eeb4d4d77535626f644e97387595128  runtime-v2/runs/final_v2_exact/screen_doc.json
+8ff918ad3e2c406fdc71be2997f998bbe8b71dd9330df1fbe8df4571f590e117  runtime-v2/runs/final_v2_exact/startup.txt
+92cbd830f90bd66caa59be28ec3c8a0494abc089100472392eca1636281e292b  runtime-v2/runs/final_v2_fast/eval.log
+dece7c45c38bcbdb721fabb340b72bc1410735df883146a440aff629bea1998d  runtime-v2/runs/final_v2_fast/eval/compare_vs_ref.json
+6b8f184b2b8a1af194a06edeaec8a46c5ec1c1deec246f5cf646fe7dcdef3d67  runtime-v2/runs/final_v2_fast/eval/raw_gsm8k.jsonl
+40dcf3d22f9c3727e2c6112561cbe128e72180a909ad8e51d8bb8e75d5cba75b  runtime-v2/runs/final_v2_fast/eval/raw_humaneval.jsonl
+e383f09c1a104b9f0413240e11456b962d9b736ea91fff64eec3daff762b9616  runtime-v2/runs/final_v2_fast/eval/raw_ifeval.jsonl
+ccfc6392d774720393c16ce659dfb50b58c110f4e5ea4cb745b3df1ee7bdb4f3  runtime-v2/runs/final_v2_fast/eval/raw_mmlu_pro.jsonl
+73d5fc4a7f459500e61fb1f0a2d073c69d2f5e9befd2c7fa2b04d1594efe9cd1  runtime-v2/runs/final_v2_fast/eval/raw_tools.jsonl
+771af56bd6e7f91dd9a20ba6d271f07846c0eccdac0d4ffe0cb82991c4d0a1ba  runtime-v2/runs/final_v2_fast/eval/raw_vqa.jsonl
+92df69d5c58818a00d2071d14dad8f0155789297b7582de15a583dd4a98c4a71  runtime-v2/runs/final_v2_fast/eval/summary.json
+1bf1c5ca785fce32bf0a7248f91e070e7b4c9588369e863d2438bf4d7c788276  runtime-v2/runs/final_v2_fast/eval_compare.txt
+e6c066419165ce26f5599e0efc5a41d4fc0b1f59bb862c15cf763a4f3ba3a0ea  runtime-v2/runs/final_v2_fast/gpu_after.txt
+f3da2a51469f7551f24bfe0833be0567b0374f5ce8caa0d9631fe0327d00221f  runtime-v2/runs/final_v2_fast/gpu_before.txt
+630d186e382f56a9d42a8f4cb3a5f2ff923f77a3876b58a72e99c44befc45631  runtime-v2/runs/final_v2_fast/greedy.json
+37eda03a85270d0c0629ee54c4965e003c06536063f4100a44baff3c31da965e  runtime-v2/runs/final_v2_fast/greedy_diff.txt
+6eb0bb17cf5b6b0e8a652ef7fb6d31a11485b1074182ce6752a970494de9acf1  runtime-v2/runs/final_v2_fast/mem.txt
+ec82b03916c6d8b88c47bb041136182d4139fd82b759b3190e680b0ec4bf5082  runtime-v2/runs/final_v2_fast/perf_matched.json
+efc50e8834d005b2eb1b7fb1d0179987eea4f1381c12bdcc932fef96d01d5f09  runtime-v2/runs/final_v2_fast/perf_matched.log
+3c5bf2f1dd670eb8dc3a48edd12e0a8685543541a146d25600dbd9c6f572c999  runtime-v2/runs/final_v2_fast/screen_code.json
+8f591e3b6c57b106db8e9fb4df8a99a54fcf3184ac68aebfb9f953e9dfc7891a  runtime-v2/runs/final_v2_fast/screen_doc.json
+0e35cc376c50fd8efe4dedff41a05c94e2619c0cee896250fed3bd0506555e50  runtime-v2/runs/final_v2_fast/startup.txt
+ea4b7427752378606219e1cf9357ba40aa67dd33235c12301d500c16a997a71b  runtime-v2/runs/hiconc_full_mtp_code.json
+0a9ba9984b6cbb131ab25fd6d8f8af8057141dbafcefe4d7f4b762943968a125  runtime-v2/runs/hiconc_full_mtp_synthetic.json
+6b93bc4ddd725b554bd3a4967135a8c099a5ad977362b553a780a724d1fbf896  runtime-v2/runs/hiconc_full_nomtp_code.json
+8a719835ac522fb33336a5e88a274165bd08ebb9fecf2c668fe699e8358139ec  runtime-v2/runs/hiconc_full_nomtp_synthetic.json
+66b3ffa838ab282f36ad13cae239e8b4f964ccfdbcc68e08ce706404bc8b01d0  runtime-v2/runs/screen_exact_nomtp_code.json
+7c42a88aff1c10777774242f8361cc06382c366d23e3e8e9e300952037b2e2ee  runtime-v2/runs/screen_exact_nomtp_synthetic.json
+1fb4c028f4786cac1dbf522a38a02a32855539b4dae5bf91add73b9d56722f89  runtime-v2/runs/screen_exact_tm32k_code.json
+d608a24eaec0b3124b9c0df90e8fb24a4cf62a3d04f931408f710ef954b0cf31  runtime-v2/runs/screen_exact_tm32k_synthetic.json
+1937cc250984d9af1c34fc22b5ca39158d8701eaec2cd78d1cd3b573db8287c8  runtime-v2/runs/screen_exact_tm64k_code.json
+d0aaaa27413ca37e111e647db68196948b3278af2f7153058851c168d691dbe3  runtime-v2/runs/screen_exact_tm64k_cutlass_code.json
+6f6feb4492963164dbc7dad10f086b96d70cbf68adb83fd27662459de492c422  runtime-v2/runs/screen_exact_tm64k_cutlass_synthetic.json
+064eeb61e42a384264749e2902bf0e7c255c6614c158f2c917e70f37f9dafe23  runtime-v2/runs/screen_exact_tm64k_marlin_code.json
+c0073c475b7cbd4a6182cecd30cffe17a56ddb98918b559a3d7a5a26453ce437  runtime-v2/runs/screen_exact_tm64k_marlin_synthetic.json
+0ad793cdb2cf70fd030c5968f0816772e86d0f0a780064b9b1632f2aff7850e3  runtime-v2/runs/screen_exact_tm64k_s2_code.json
+14d4d07b9a95d40f609b487921077ccef2bf57b2f9d25f3c5e020c3ae9fd2676  runtime-v2/runs/screen_exact_tm64k_s2_synthetic.json
+5d709710bb44b230b6ee08cf6a562ea45981949935df624b215ac17d4db5cd3b  runtime-v2/runs/screen_exact_tm64k_s4_code.json
+23c561017cbcff5f3a2730da4992f3f65910026ed310d1947f689e0292163c0b  runtime-v2/runs/screen_exact_tm64k_s4_synthetic.json
+e1b1fb5cc625e5aa44742c12982bef45e194ec3b59394f7ed110732fc470f2c9  runtime-v2/runs/screen_exact_tm64k_s5_code.json
+cdc7bbf511211f8b688d6abcb8278ae7ee45b38d9246c16dae205cacf9b43fdf  runtime-v2/runs/screen_exact_tm64k_s5_synthetic.json
+da53ac443d172f43721562867a1ccacc8775bf785dbd787816c7d5fd918aaf21  runtime-v2/runs/screen_exact_tm64k_synthetic.json
+42f790ca23bf3707695de96fdb2edc8ef31099433ed74073d847cc2c2c41db2c  runtime-v2/runs/screen_exactfused_code.json
+df8cc82b05501ccc78770b178ef3fcbb5a2c5e2b24d8aed72c0c732737a66a4e  runtime-v2/runs/screen_exactfused_synthetic.json
+68bff4c8a1d0c80151d734141d8e933c1e8ec64041cb00dca1357741fd8f6921  runtime-v2/runs/screen_frozen_code.json
+1ef02dcabdc8052687a7f10093589d3489791af6fdd7db57a61bed59de2640db  runtime-v2/runs/screen_frozen_synthetic.json
+c30ea79feba85166f0907f328e89173db73d859ba309a008ec48e7f3ebf800e3  runtime-v2/runs/screen_full_tm64k_code.json
+3759cbc1e65b4281836a19cfd9bd1ec2d885e6af3dd1d522b97257bfdb75acfb  runtime-v2/runs/screen_full_tm64k_synthetic.json
+fdebbccb4ccb10b29f43a3339671ac155f756afd6d27a8765c04ceb3f8320567  runtime-v2/runs/screen_fused_code.json
+3744bf31104e186420a0103a2aab4578a066d6ae189f75a745038a1e64b7bd2a  runtime-v2/runs/screen_fused_synthetic.json
+b2a551e06fbe60688f584d30d3c0155a9c362e0d94f677bc381a09df9868f43b  runtime-v2/runs/screen_tm32k_code.json
+a9cb5f518f107ebae9e68aef34ef4859432f5d3df28e056b21010a7c2080182a  runtime-v2/runs/screen_tm32k_synthetic.json
+0e4fed3fa486fa7785aff17c0d1bfffa2ff32a6995a48e12a8f81fa197bc8168  runtime-v2/runs/test_mhc_fused.log
+73341e398b509936823aeed0a988bf4979cadddc86dcf835967722741dbc68e0  runtime-v2/runs/var/frozen_mmlu_1.log
+fb767a933a78863687942d4d229c8a83db618402a788da3451fec189f1d2ea9f  runtime-v2/runs/var/frozen_mmlu_1/raw_mmlu_pro.jsonl
+96cc5e3107540e10a06de960a373db2a36c7197823da58487ff35d54daca4695  runtime-v2/runs/var/frozen_mmlu_1/summary.json
+882f35e5ea9cbdf01c9e384a51b0a73cbbd080093dfea7b09d5521bcaff6935f  runtime-v2/runs/var/frozen_mmlu_2.log
+cea910838afd0172f8c3996d936bda06d211ac390bf2955a5b67834dcb363b0c  runtime-v2/runs/var/frozen_mmlu_2/raw_mmlu_pro.jsonl
+3ad1cd1d31662f10549a4896e9f95ce5cced9df1aaabc584b932024fe4f180f4  runtime-v2/runs/var/frozen_mmlu_2/summary.json
+6eeac27f8871fcdebccdfeb68b967dd426471b000d386b2d91fd9e8bd9157f0d  runtime-v2/runs/var/v2_fast_mmlu_1.log
+e32b230c458f1f34167320e18a82c9f67d71f9be5595baf8fe329dc493dff65d  runtime-v2/runs/var/v2_fast_mmlu_1/raw_mmlu_pro.jsonl
+a123b42dd5e412d7b6ba769ed28f6baabc5d5e6eb991feb6701e61b3b2756c54  runtime-v2/runs/var/v2_fast_mmlu_1/summary.json
+ea40811ae5e7b86d215368593563b73043e9c81796a57a37df01801b5402e561  runtime-v2/runs/var/v2_fast_mmlu_2.log
+b9f451c6035eaafde49e67513e321c7b97301c019fc0e59a1053c08e2c46e67d  runtime-v2/runs/var/v2_fast_mmlu_2/raw_mmlu_pro.jsonl
+fd36f9094ea1f847fe0577c05954c4400d93720dba30011eb03ea36a7bf455da  runtime-v2/runs/var/v2_fast_mmlu_2/summary.json
+333515ef56dddab835aaf72d7d3e41e69acd13feddd29b3eea1a796cdba2c8cf  runtime-v2/serve_sglang.sh
+608f51ea2fc61860cd74e414c506310abd5bc196b6e98c36a34b58d6c61d08b3  runtime-v2/tests/greedy_outputs.py
+f70ffa501857fc0957c3ffbcf6a4143bea059dcf4a02cf7ba568adf5fe45e2ec  runtime-v2/tests/test_mhc_fused.py
+0e4fed3fa486fa7785aff17c0d1bfffa2ff32a6995a48e12a8f81fa197bc8168  runtime-v2/tests/test_mhc_fused.result.txt
+68ea210548cadb0adcc84c9c312bb22b6a4713656ee302c3b0d08f4d18dc1aef  runtime-v2/tokenmap/build_token_map.py
+3f15c5ab3623692cd2ac59a3fa4bd10cf41bfa6f36049b3a182bb82be99af31e  runtime-v2/tokenmap/hot_32k.pt
+119fa796636a94aa5bc9b3cc31e0410d34d4a6f4e023b9ee8ed783320c916a57  runtime-v2/tokenmap/hot_64k.pt
+e12571bb950e842d6ee21782ca5f15bc7b2375461800744db589c163bb5fec08  runtime-v2/tokenmap/token_map_stats.json
 1b95e589cd65a97a97fae93cdeec07fcadbf0ff204be746fe20db32550a2c248  runtime/COMPAT_NOTES.md
 e00d2fafe780fcf4c04ab52b040e43ec7143c7bd6e070637563f2724b59888d1  runtime/overlay/python/sglang/srt/configs/__init__.py
 bdc7fa2055b799ce07441a5d499fafa554b7b797d4c4a24fd251b846b51260fd  runtime/overlay/python/sglang/srt/configs/model_config.py

figures/README.md

@@ -15,3 +15,9 @@ Quality panels show task scores and paired-bootstrap 95% intervals from the orig
 These intervals reflect this small item set, not run-to-run throughput variation or full-benchmark accuracy.
 
 The originals are 300-DPI PNG plus vector PDF and SVG. No new GPU benchmarks were run to make these figures.
+
+## Runtime v2 final figures
+
+`python figures/plot_runtime_v2.py` reads only `runtime-v2/runs/final_*/` JSONs.
+Original v1 figures remain unchanged. New charts separate final matched runs from screening, and
+show the first complete quality runs and paired intervals, including fast-profile regressions.

figures/plot_runtime_v2.py

@@ -0,0 +1,52 @@
+#!/usr/bin/env python3
+"""Final runtime-v2 figures, read only from archived JSON evidence (figures4papers style)."""
+from pathlib import Path
+import json
+import matplotlib
+matplotlib.use('Agg')
+import matplotlib.pyplot as plt
+import numpy as np
+ROOT=Path(__file__).resolve().parent.parent;OUT=ROOT/'figures';DATA=ROOT/'runtime-v2/runs'
+plt.rcParams.update({'font.family':'sans-serif','font.sans-serif':['DejaVu Sans','Arial','Helvetica'],
+ 'font.size':12,'axes.titlesize':14,'axes.labelsize':12,'axes.linewidth':1.8,'axes.spines.right':False,
+ 'axes.spines.top':False,'legend.frameon':False,'svg.fonttype':'none','pdf.fonttype':42})
+NAMES=['Frozen v1 + MTP','v2 exact + MTP','v2 fast + MTP'];DIRS=['final_frozen_v1','final_v2_exact','final_v2_fast'];COLORS=['#767676','#3775BA','#0F4D92']
+def load(d,f):return json.loads((DATA/d/f).read_text())
+def save(fig,name):
+ for ext in ['png','pdf','svg']:fig.savefig(OUT/f'{name}.{ext}',dpi=300,bbox_inches='tight',facecolor='white')
+ plt.close(fig)
+fig=plt.figure(figsize=(13,8.4));gs=fig.add_gridspec(2,2,left=.085,right=.98,top=.83,bottom=.23,wspace=.30,hspace=.63)
+a=fig.add_subplot(gs[0,:]);b=fig.add_subplot(gs[1,0]);c=fig.add_subplot(gs[1,1])
+fig.suptitle('Runtime v2 — faster serving, unchanged NVFP4 weights',y=.98,fontsize=19,fontweight='bold')
+fig.text(.5,.925,'Final validation on one DGX Spark • W4A4 + MTP • October 31, 2026',ha='center',color='#4D4D4D')
+x=np.arange(3);w=.24
+for i,(d,col,lab) in enumerate(zip(DIRS,COLORS,NAMES)):
+ vals=[load(d,'perf_matched.json')['levels'][0]['decode_tok_s_per_req']['p50']]+[load(d,f'screen_{k}.json')['levels'][0]['decode_tok_s_per_req_p50'] for k in ['code','doc']]
+ bars=a.bar(x+(i-1)*w,vals,w,color=col,edgecolor='black',linewidth=.8,label=lab)
+ for q,v in zip(bars,vals):a.text(q.get_x()+q.get_width()/2,v+.55,f'{v:.1f}',ha='center',fontsize=11,fontweight='bold')
+a.set(title='A  Single-request decode (not aggregate throughput)',ylabel='Median decode tokens/s',xticks=x,xticklabels=['Matched synthetic (~1K input)','Short code (~90 input)','Document (~4K input)'],ylim=(0,44));a.legend(loc='upper left',ncol=3,fontsize=10);a.grid(axis='y',alpha=.16);a.set_axisbelow(True)
+for d,col,lab,m in zip(DIRS,COLORS,NAMES,['o','s','^']):
+ data=load(d,'perf_matched.json')['levels'];xs=[t['concurrency'] for t in data]
+ b.plot(xs,[t['aggregate_output_tok_s'] for t in data],color=col,marker=m,lw=2.5,ms=7,label=lab)
+ c.plot(xs,[t['ttft_s']['p50'] for t in data],color=col,marker=m,lw=2.5,ms=7)
+b.set(title='B  Matched-workload concurrency',xlabel='Concurrent requests',ylabel='Aggregate output tokens/s',xticks=[1,2,4,8],ylim=(0,145));b.grid(axis='y',alpha=.16)
+c.set(title='C  Matched-workload first token',xlabel='Concurrent requests',ylabel='Median TTFT (seconds)',xticks=[1,2,4,8],ylim=(0,6));c.grid(axis='y',alpha=.16)
+fig.text(.5,.075,'Exactly 256 output tokens • greedy • no prefix-cache hits • 0 errors in final performance runs\nMatched: 8 requests/level; code/document: 4 at c=1 • final results, not earlier screening peaks\nExact preserves reference routing math; fast changes FP32 reduction order and may change generated text.\nSource: runtime-v2/runs/final_*/{perf_matched,screen_code,screen_doc}.json',ha='center',va='center',fontsize=10,color='#4D4D4D',linespacing=1.6)
+save(fig,'runtime-v2-performance')
+keys=['ifeval','gsm8k','mmlu_pro','humaneval','tools','vqa'];labels=['IFEval*','GSM8K','MMLU-Pro','HumanEval','Tools**','TextVQA']
+fig,axes=plt.subplots(1,2,figsize=(13.8,6.9),gridspec_kw={'width_ratios':[1.6,1]});fig.subplots_adjust(left=.07,right=.98,top=.77,bottom=.37,wspace=.29)
+fig.suptitle('Runtime v2 quality — preserved evidence, explicit trade-offs',y=.98,fontsize=18,fontweight='bold')
+fig.text(.5,.914,'Same 300-item suite • thinking off • greedy • evaluation concurrency 16',ha='center',color='#4D4D4D')
+x=np.arange(6);w=.24
+for i,(d,col,lab) in enumerate(zip(DIRS,COLORS,NAMES)):
+ j=load(d,'eval/summary.json');vals=[100*j['tasks'][k]['accuracy'] for k in keys]
+ axes[0].bar(x+(i-1)*w,vals,w,color=col,edgecolor='black',linewidth=.7,label=lab)
+axes[0].set(title='A  First complete regression run',ylabel='Accuracy (%)',xticks=x,xticklabels=labels,ylim=(0,105));axes[0].tick_params(axis='x',labelrotation=25);fig.legend(handles=axes[0].get_legend_handles_labels()[0],labels=NAMES,loc='upper center',bbox_to_anchor=(.5,.865),ncol=3,fontsize=10);axes[0].grid(axis='y',alpha=.16);axes[0].set_axisbelow(True)
+for i,(d,col) in enumerate(zip(DIRS[1:],COLORS[1:])):
+ q=load(d,'eval/compare_vs_ref.json')['overall_micro'];v=q['delta']*100;lo,hi=np.array(q['delta_boot95'])*100
+ axes[1].errorbar(v,i,xerr=[[v-lo],[hi-v]],fmt='o',color=col,capsize=5,lw=2)
+ axes[1].text(-5.5,i+.22,f'{v:+.2f} pp [{lo:+.2f}, {hi:+.2f}]',fontsize=10)
+axes[1].axvline(0,color='#767676',lw=1,ls='--');axes[1].set(title='B  Pooled change vs frozen v1',xlabel='Percentage points (paired-bootstrap 95% CI)',yticks=[0,1],yticklabels=['v2 exact','v2 fast'],xlim=(-6,3),ylim=(1.55,-.55));axes[1].grid(axis='x',alpha=.16)
+fig.text(.5,.16,'Frozen: 251/300 (83.7%) • exact: 249/300 (83.0%) • fast: 247/300 (82.3%)\nFast MMLU-Pro first run: 58.3% vs 70.0% frozen (−11.7 pp); fast reruns: 71.7%, 65.0%.\nFast truncations: 20 vs 14 frozen. Fast fails original first-run per-task/stability tripwires; not “lossless.”\n*Re-implemented IFEval checker. **20 hand-written tool cases. Small subsets, not full-benchmark results.\nSource: runtime-v2/runs/final_*/eval; all reruns retained in runtime-v2/runs/var and RESULTS.md.',ha='center',va='center',fontsize=10,color='#4D4D4D',linespacing=1.7)
+save(fig,'runtime-v2-quality')
+print('Generated final v2 performance/quality PNG, PDF, SVG from JSONs.')

runtime-v2/.gitignore

@@ -0,0 +1 @@
+__pycache__/

runtime-v2/LEDGER.md

@@ -0,0 +1,19 @@
+# Experiment ledger — SenseNova-Loop-9B NVFP4 runtime optimization (GB10, 2026-10-30)
+Screen = bench/screen_set.sh: synthetic(1062 in)/code(90 in) prompts, 256 out, c=1 (4 req) + c=4 (8 req), unique prompts, temp 0, ignore_eos.
+Greedy check = tests/greedy_outputs.py 12 prompts (text/math/think/code/zh/json/long/tool/image) vs frozen first run on a fresh server.
+
+| # | Hypothesis | Change | Correctness | synth c1 / c4 | code c1 / c4 | Keep? |
+|---|---|---|---|---|---|---|
+| 0 | baseline | frozen release runtime, W4A4+MTP(3) | ref; same-server repeat 9/12 (prefix-cache rerun) | 11.6 / 40.8 | 15.4 / 58.4 | ref |
+| P | profile c1 | — | — | cycle 157 ms: verify 117 (mHC ~60 incl gaps, FP4 35, lm_head 8.7, router cuBLAS 5.5), draft 27.5+13.7 (BF16 lm_head x3) | | |
+| 1 | mHC is launch-bound (39k kernels/fwd) | fused Triton Sinkhorn (bitwise), cuBLAS linear + torch merge kept ("exact") | 11/12 identical (only random tool-call id) | 18.7 / 62.9 | 24.8 / 87.0 | KEEP |
+| 1b | router linear+merge also fused | single-kernel rmsnorm+linear+Sinkhorn+branch, fused merge ("full") | 2/12 identical; FP32 sum-order diffs (closer to fp64 than cuBLAS) flip near-ties | 21.2 / 73.8 | 25.7 / 92.6 | option (not bitwise) |
+| 2 | draft lm_head (2 GB BF16, 3x/cycle) dominates draft | FR-Spec hot-vocab draft head --speculative-token-map (train-split corpora), target verify stays full-vocab | 32k: 10/12; 64k: 11/12 | 32k 27.6/72.0; 64k 29.0/80.7 | 32k 32.9/105.8; 64k 32.2/106.7 | KEEP 64k |
+| X | target-only vs MTP (same math) | --no-mtp, exact | 1/12 identical vs MTP: M=1 decode vs M=4 verify kernels differ -> inherent spec/target near-tie flips (also true for frozen) | 19.5 / 63.0 | 20.0 / 70.2 | info |
+| P2 | profile exact+tm64k c1 | — | — | cycle ~69 ms: verify 56 (FP4 34.8, lm_head 8.7, router cuBLAS 5.8), draft 8.6 + extend 4.3 | | |
+| 3 | more/fewer MTP steps pay after draft got cheap | steps 2/4/5 (draft tokens = steps+1), exact+tm64k | — | s2 26.4/79.0; s4 25.9/68.5 | s2 32.8/108.9; s4 30.8/104.1; s5 31.6/94.3 | REJECT (keep 3) |
+| 4 | cutlass FP4 faster on down_proj (microbench -7% FP4) | --fp4-gemm-backend flashinfer_cutlass | 10/12 (different FP4 accumulation) | 26.3/80.8 | 33.2/108.6 | REJECT (noise-level) |
+| 5 | fused router linear+merge worth it once mHC is not dominant | full fused (default env) + tm64k | 2/12 (FP32 sum order, see 1b) | 31.0/90.6 (+7%/+12% vs exact) | 37.5/117.5 (+16%/+10%) | FINALIST (needs 300-item) |
+| 6 | Marlin W4A16 + MTP better single-user | marlin + SILU fusion off, exact+tm64k | 3/12 (W4A16 numerics) | 27.9/77.4, TTFT c4 2.2 s | 33.8/107.7 | REJECT (parity decode, worse TTFT) |
+| 7 | no-MTP throughput profile for high concurrency | --no-mtp vs MTP, fast+tm64k, c=8/16 | — | c8 129.4 vs 130.2; c16 194.3 vs 171.4 | c8 150.6 vs 198.4; c16 258.0 vs 295.0 | REJECT (single MTP profile) |
+| F | final validation | frozen_v1 vs v2_fast vs v2_exact (RESULTS.md) | 300-item: 251 / 247 / 249; exact == frozen 12/12; mmlu_pro reruns show frozen spans 0.65-0.70 | matched c1 12.0 -> 27.8 / 26.0; c8 64.3 -> 119.4 / 117.2 | code c1 15.6 -> 35.3; c8 97.4 -> 196.7 | SHIP v2 (default fast, exact switch) |

runtime-v2/README.md

@@ -0,0 +1,86 @@
+# SenseNova-Loop-9B-Preview-NVFP4 — runtime v2 (published companion update)
+
+Faster serving runtime for the **unchanged** published checkpoint
+[`gitcommit90/SenseNova-Loop-9B-Preview-NVFP4`](https://huggingface.co/gitcommit90/SenseNova-Loop-9B-Preview-NVFP4) (weights, config and the v1 `runtime/` are untouched).
+Same pinned SGLang image and PR #47281 overlay as v1; W4A4 NVFP4 (flashinfer cuDNN FP4 GEMMs) + MTP (NEXTN, 3 steps).
+
+## Use
+```bash
+runtime-v2/serve_sglang.sh /path/to/SenseNova-Loop-9B-Preview-NVFP4        # default: MHC_MODE=fast + 64k draft vocab + MTP
+MHC_MODE=exact runtime-v2/serve_sglang.sh <model>    # bitwise-identical target math to v1 (greedy outputs = v1)
+MHC_MODE=reference TOKEN_MAP=none runtime-v2/serve_sglang.sh <model>   # = v1 runtime behaviour and speed
+runtime-v2/serve_sglang.sh <model> --no-mtp           # target only (MTP still recommended, see results)
+```
+Env as in v1 (`PORT`, `HOST`, `MEMFRAC`, `CTX`, `MAXREQ`, `SERVED`, `NAME`) plus `MHC_MODE`, `TOKEN_MAP`, `SGL_CACHE`.
+**Recommended conservative command:** `MHC_MODE=exact bash runtime-v2/serve_sglang.sh /path/to/model`.
+The launcher’s existing default remains `fast`; choose explicitly. Exact describes target routing math, not a
+guarantee of identical sampled sequences on every workload/batch/hardware. Fast may change greedy outputs.
+
+Startup to ready: ~75 s (v1: ~90 s). Rollback: stop the container and start `bash runtime/serve_sglang.sh /path/to/model` — nothing is shared
+except the JIT cache dir (`~/.cache/sensenova-sglang`, safe to delete).
+
+## What changed (two things)
+1. **Fused mHC router** (`overlay/python/sglang/srt/layers/sensenova_mhc_fused.py`, wired in `models/sensenova_loop.py`,
+   diff: `patches/v2_mhc_fused_router.patch`). The original router ran FP32 log-space Sinkhorn (20 iterations on the 2x2 stream
+   matrix) as ~300 tiny ATen kernels per call; 96 router calls per forward (24 looped layers x 2 visits x 2 routers) = ~39k kernels,
+   ~60 ms of a 157 ms decode cycle. Now Triton:
+   * Sinkhorn/sigmoid kernel: same operation order, 20 iterations from config, torch.logsumexp semantics (NaN-propagating max,
+     inf-masked shift), precise libdevice exp/log/div, no FTZ: **bitwise equal** to eager (all T 1..4096, extreme values, CUDA graphs).
+   * `MHC_MODE=fast` (default) additionally fuses the FP32 RMS-normalize + 8x8192 FP32 router linear + branch, and the merge.
+     Still FP32 everywhere and the same dtype boundaries, but the FP32 dot-product summation order differs from cuBLAS
+     (coefficient diff <= 6.6e-7; vs a float64 reference the fused result is *more* accurate: 2.4e-7 vs cuBLAS 6.6e-7).
+     Not bitwise, so greedy text can diverge at near-ties (see correctness). +7..16% over `exact`.
+   * Eager reference kept verbatim (`_forward_reference`/`_merge_reference`, `MHC_MODE=reference`).
+2. **FR-Spec hot-vocab MTP draft head** (`--speculative-token-map tokenmap/hot_64k.pt`). The draft proposed tokens through the
+   full 2 GB BF16 lm_head 3-4x per cycle (~35 ms). The draft now scores only the 65,536 most frequent tokens (+ all 33 special
+   tokens) counted on TRAIN-split corpora (oasst2, glaive-fc-v2, mbpp, gsm8k train; 37 M tokens; 99.1% weighted coverage;
+   `tokenmap/build_token_map.py`). **The target still verifies over the full vocabulary**. Vocabulary restriction applies only to drafting.
+   Speculative decoding preserves the target distribution in exact arithmetic with correct verification; finite-precision
+   kernels/batching can change actual outputs. Only greedy decoding was validated here; sampling equivalence was not tested. No weights changed (the draft head is a row subset of the same lm_head).
+
+Nothing else: same image digest, same flags (cuDNN FP4, steps 3, mem 0.45, ctx 32768, max 16 running), no KV/state precision change,
+no change to loop passes, layers, routing, Sinkhorn iterations, sampling, context or output budgets.
+
+## Rejected after measurement (see LEDGER.md)
+MTP steps 2/4/5 (no gain), flashinfer_cutlass FP4 (noise-level), Marlin W4A16+MTP (decode parity, 2x worse TTFT), 32k token map
+(64k equal speed, better fidelity), no-MTP throughput profile (MTP wins on realistic text up to c=16; no-MTP only wins on
+random-word synthetic at c=16).
+
+## Results (full tables: RESULTS.md; screening history: LEDGER.md)
+Per-user decode speed and aggregate throughput, W4A4 + MTP, v1 frozen -> v2 default (`fast`) [`exact`]:
+
+| workload | c | aggregate tok/s | per-request decode tok/s p50 | TTFT p50 |
+|---|---:|---|---|---|
+| matched (1051 in / 256 out) | 1 | 12.0 -> **27.8** [26.0] | 12.4 -> **28.9** [27.3] | 0.74 -> 0.32 s |
+| matched | 2 | 23.4 -> **49.2** [45.5] | 12.2 -> 28.4 | 0.89 -> 0.41 s |
+| matched | 4 | 42.0 -> **87.0** [72.6] | 12.1 -> 25.4 | 1.84 -> 0.95 s |
+| matched | 8 | 64.3 -> **119.4** [117.2] | 12.1 -> 19.8 | 5.02 -> 2.90 s |
+| code (90 in / 256 out) | 1 | 15.6 -> **35.3** [32.8] | 16.0 -> 35.9 | 355 -> 134 ms |
+| code | 8 | 97.4 -> **196.7** [172.7] | 14.1 -> 26.3 | 744 -> 332 ms |
+| document (~4k in / 256 out) | 1 | 12.8 -> **29.1** [25.6] | 14.3 -> 33.3 | 2.03 -> 1.11 s |
+| document | 8 | 52.4 -> **92.3** [70.4] | 9.0 -> 16.1 | 5.49 -> 3.91 s |
+
+MTP acceptance unchanged within noise (code 2.43-2.51 -> 2.31-2.41; doc 2.16-2.26 -> 2.16-2.19). 0 errors; every request produced
+exactly the requested 256 tokens. 300-item regression: frozen 251/300, v2_exact 249, v2_fast 247. These are small, noisy regression tests, not proof of equivalence.
+The first fast MMLU-Pro run dropped 11.7 percentage points (70.0% → 58.3%); reruns varied substantially.
+See RESULTS.md for every run. Fast also had 20 truncated outputs vs 14 frozen, above the original ref+3 tripwire.
+**Fast does not pass every original per-task/stability gate on the first run; use exact for the conservative profile.**
+
+## Correctness evidence
+* `tests/test_mhc_fused.py` (result: `tests/test_mhc_fused.result.txt`): Sinkhorn kernel bitwise vs eager for T in
+  {1..4096} (decode, MTP verify 4/16/32, prefill 1051/4096), logit scales 0.1..1e4, +-inf, NaN, 3e38, denormals, non-contiguous
+  inputs, iterations 1/5/20/50, and under CUDA-graph capture/replay; exact router path bitwise end-to-end on real layer 4/15/27
+  router weights; fast path coefficient diff <= 6.6e-7 (closer to fp64 than cuBLAS), bf16 branch identical.
+* Greedy feature check `tests/greedy_outputs.py`: exact == frozen 12/12; fast valid on all 12 (text, thinking, code, zh, JSON, long,
+  tool call, image).
+* MTP verified throughout (acceptance logged per request); target-only vs MTP divergence characterised (inherent to v1 too).
+
+## Remaining bottleneck (profile, c=1, exact mode, ~63-69 ms per speculative cycle; GPU ~100% busy, no launch gaps left)
+* FP4 W4A4 GEMMs ~35 ms (55%): 6.8 GB of FP4 weights per forward because the 24 looped layers are read twice; cuDNN/CUTLASS
+  already at 85-95% of the 273 GB/s DRAM roofline in isolation. Floor ~25 ms.
+* Target BF16 lm_head 8.7 ms (2 GB, required for verification) — only reducible by quantizing lm_head (a weight change; needs approval).
+* Draft (BF16 MTP layer + 64k head) ~13 ms; GDN/attention/norms ~6 ms; router linear ~5.8 ms in exact mode (~0.5 ms in fast).
+* Honest ceiling: ~8.8 GB of weights per target forward => <=31 forwards/s at 100% bandwidth; with accept length 2.3-2.9 that is
+  ~70-90 tok/s theoretical, realistically ~40-45 with this stack. Measured: 28-36 tok/s single-user. Reaching 60 tok/s single-user
+  would need changes outside this brief: FP8/NVFP4 lm_head and/or draft weights (weight changes), a better-trained draft (higher
+  acceptance), or a fused-weight persistent decode kernel. Each is a separate approval/effort decision.

runtime-v2/RESULTS.md

@@ -0,0 +1,83 @@
+# Runtime v2 — final validation results (GB10, 2026-10-31, same session, clocks 2.4 GHz, 47-65 C)
+
+frozen_v1 = published release runtime (W4A4 + MTP, eager mHC). v2_fast = default launcher. v2_exact = MHC_MODE=exact. All W4A4 NVFP4 + MTP(3), identical flags/budgets, unique prompts (no prefix-cache hits), temp 0.
+
+### Matched workload (eval/perf_bench.py: ~1051 prompt tok, 256 out, unique prompts, 8 req/level, 2 warmup)
+
+| runtime | c | aggregate tok/s | per-req decode tok/s p50 | TTFT p50 / p90 (s) | ITL p50 / p90 (ms) | E2E p50 / p90 (s) | out tok min-max | errors |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| frozen_v1 | 1 | 12.0 | 12.4 | 0.74 / 0.74 | 81 / 87 | 21.3 / 22.8 | 256-256 | 0 |
+| frozen_v1 | 2 | 23.4 | 12.2 | 0.89 / 1.27 | 82 / 89 | 21.9 / 23.5 | 256-256 | 0 |
+| frozen_v1 | 4 | 42.0 | 12.1 | 1.84 / 2.78 | 82 / 92 | 22.9 / 24.4 | 256-256 | 0 |
+| frozen_v1 | 8 | 64.3 | 12.1 | 5.02 / 5.11 | 82 / 94 | 26.0 / 29.1 | 256-256 | 0 |
+| v2_fast | 1 | 27.8 | 28.9 | 0.32 / 0.34 | 35 / 39 | 9.2 / 10.2 | 256-256 | 0 |
+| v2_fast | 2 | 49.2 | 28.4 | 0.41 / 0.57 | 35 / 42 | 9.5 / 11.1 | 256-256 | 0 |
+| v2_fast | 4 | 87.0 | 25.4 | 0.95 / 1.23 | 39 / 44 | 10.9 / 12.2 | 256-256 | 0 |
+| v2_fast | 8 | 119.4 | 19.8 | 2.90 / 2.91 | 51 / 55 | 15.8 / 17.0 | 256-256 | 0 |
+| v2_exact | 1 | 26.0 | 27.3 | 0.51 / 0.51 | 37 / 39 | 9.8 / 10.5 | 256-256 | 0 |
+| v2_exact | 2 | 45.5 | 25.7 | 0.58 / 0.98 | 39 / 47 | 10.5 / 12.6 | 256-256 | 0 |
+| v2_exact | 4 | 72.6 | 21.7 | 1.15 / 1.81 | 46 / 50 | 12.7 / 14.0 | 256-256 | 0 |
+| v2_exact | 8 | 117.2 | 21.2 | 4.35 / 4.36 | 47 / 51 | 16.4 / 17.4 | 256-256 | 0 |
+
+### code: short interactive coding/reasoning prompt (~90 tok in, 256 out)
+
+| runtime | c | n | aggregate tok/s | per-req decode p50 | TTFT p50 (ms) | ITL p50 / p95 (ms) | MTP accept len | errors |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| frozen_v1 | 1 | 4 | 15.6 | 16.0 | 355 | 63 / 66 | 2.51 | 0 |
+| frozen_v1 | 4 | 8 | 56.3 | 15.0 | 615 | 67 / 70 | 2.45 | 0 |
+| frozen_v1 | 8 | 16 | 97.4 | 14.1 | 744 | 71 / 81 | 2.43 | 0 |
+| v2_fast | 1 | 4 | 35.3 | 35.9 | 134 | 28 / 29 | 2.31 | 0 |
+| v2_fast | 4 | 8 | 116.7 | 30.9 | 264 | 32 / 34 | 2.33 | 0 |
+| v2_fast | 8 | 16 | 196.7 | 26.3 | 332 | 38 / 41 | 2.41 | 0 |
+| v2_exact | 1 | 4 | 32.8 | 33.4 | 148 | 30 / 31 | 2.37 | 0 |
+| v2_exact | 4 | 8 | 106.5 | 29.0 | 282 | 35 / 38 | 2.41 | 0 |
+| v2_exact | 8 | 16 | 172.7 | 23.4 | 398 | 43 / 46 | 2.37 | 0 |
+
+### doc: long document (~4k tok in, 256 out)
+
+| runtime | c | n | aggregate tok/s | per-req decode p50 | TTFT p50 (ms) | ITL p50 / p95 (ms) | MTP accept len | errors |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| frozen_v1 | 1 | 4 | 12.8 | 14.3 | 2029 | 70 / 74 | 2.26 | 0 |
+| frozen_v1 | 4 | 8 | 36.9 | 12.1 | 5490 | 83 / 108 | 2.16 | 0 |
+| frozen_v1 | 8 | 16 | 52.4 | 9.0 | 5491 | 111 / 156 | 2.18 | 0 |
+| v2_fast | 1 | 4 | 29.1 | 33.3 | 1111 | 30 / 31 | 2.19 | 0 |
+| v2_fast | 4 | 8 | 69.9 | 25.0 | 4012 | 40 / 51 | 2.16 | 0 |
+| v2_fast | 8 | 16 | 92.3 | 16.1 | 3909 | 62 / 85 | 2.16 | 0 |
+| v2_exact | 1 | 4 | 25.6 | 30.6 | 1791 | 33 / 33 | 2.26 | 0 |
+| v2_exact | 4 | 8 | 56.1 | 20.3 | 4736 | 49 / 64 | 2.13 | 0 |
+| v2_exact | 8 | 16 | 70.4 | 14.2 | 7989 | 71 / 121 | 2.16 | 0 |
+
+### 300-item regression (thinking off, temp 0, eval concurrency 16)
+
+| runtime | ifeval | gsm8k | mmlu_pro | humaneval | tools | vqa | pooled | truncated | errors |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| frozen_v1 | 0.767 | 0.950 | 0.700 | 0.825 | 0.900 | 0.900 | 251/300 = 0.837 | 14 | 0 |
+| v2_fast | 0.817 | 0.963 | 0.583 | 0.775 | 0.900 | 0.925 | 247/300 = 0.823 | 20 | 0 |
+| v2_exact | 0.750 | 0.975 | 0.683 | 0.800 | 0.900 | 0.875 | 249/300 = 0.830 | 9 | 0 |
+
+### MMLU-Pro run-to-run spread (identical settings, no changes between reps; 60 items, 1024-token budget)
+
+| runtime | runs (acc) | mean | truncated per run |
+|---|---|---:|---|
+| frozen_v1 | 0.700, 0.700, 0.650 | 0.683 | 9, 7, 10 |
+| v2_fast | 0.583, 0.717, 0.650 | 0.650 | 14, 10, 9 |
+| v2_exact | 0.683 | — | 6 |
+| BF16 original (v1 eval) | 0.650 | — | 13 |
+
+The first v2_fast mmlu_pro result (8 losses / 1 gain vs frozen, McNemar p=0.039) did not reproduce: the same config scored 0.717 and
+0.650 on reruns, and frozen itself spans 0.650-0.700. Losses are dominated by long CoT hitting the 1024-token budget; v2_fast's mean
+length (~600-618 tok) is between frozen (~572-594) and BF16 (622). Pooled 300: frozen 251, v2_exact 249, v2_fast 247 (delta -1.3 pp,
+bootstrap 95% CI [-4.3, +1.7]).
+
+### Greedy feature check (12 prompts: text x2, math, thinking, code x2, translate, zh, JSON, long, tool call, image)
+* v2_exact vs frozen_v1 (same validation round): **12/12 identical** (tool-call function + arguments; ids are random).
+* v2_fast vs frozen_v1: 3/12 identical; all 12 outputs valid (tool call correct, image description correct, thinking present).
+* Reference variance: frozen vs frozen across rounds 10/12; frozen target-only vs frozen MTP 1/12 (decode M=1 vs verify M=4 kernels).
+
+### Startup / memory (same --mem-fraction-static 0.45)
+| | frozen_v1 | v2_fast |
+|---|---:|---:|
+| launch -> ready | 92 s | 77 s |
+| target-verify CUDA graph capture | 22.4 s, 6.16 GB | 4.4 s, 1.14 GB |
+| KV capacity (tokens) | 402,460 | 393,742 (-2%: 0.5 GB sliced draft head) |
+| 300-item eval wall time | 549 s | 332 s |

runtime-v2/bench/README.md

@@ -0,0 +1,36 @@
+# Reproduce runtime v2 measurements
+
+Run one GPU workload at a time. Historical `runs/` contains the original final JSONs, per-item evaluation outputs,
+screening history and logs; paths inside archived JSONs may refer to the original machine. No trace files or
+session-local GPU-lock/orchestration wrappers are required for serving.
+
+From the repository/snapshot root, after downloading model files as in the main README:
+
+```bash
+python3 -m venv .venv-eval
+.venv-eval/bin/pip install -r evaluation/harness/requirements.lock
+MODEL=/absolute/path/to/SenseNova-Loop-9B-Preview-NVFP4
+# Select one runtime, wait for readiness, then measure. Stop it before switching to another.
+bash runtime/serve_sglang.sh "$MODEL"                          # frozen v1
+# MHC_MODE=exact bash runtime-v2/serve_sglang.sh "$MODEL"      # conservative optimized
+# MHC_MODE=fast bash runtime-v2/serve_sglang.sh "$MODEL"       # fastest, different summation order
+curl -fsS http://127.0.0.1:30080/v1/models
+mkdir -p results
+.venv-eval/bin/python evaluation/harness/perf_bench.py --base-url http://127.0.0.1:30080/v1 \
+  --model sensenova-loop-9b-nvfp4 --concurrency 1,2,4,8 --num-requests 8 --warmup 2 \
+  --input-tokens 1024 --output-tokens 256 --tokenizer "$MODEL/tokenizer.json" --out results/matched.json
+.venv-eval/bin/python runtime-v2/bench/screen.py --base-url http://127.0.0.1:30080 \
+  --model sensenova-loop-9b-nvfp4 --concurrency 1,4,8 --output-tokens 256 --prompt-kind code \
+  --tokenizer "$MODEL/tokenizer.json" --out results/code.json
+# Repeat with --prompt-kind doc for ~4K input tokens.
+# Build the pinned small evaluation set into evaluation/harness/data/ before running quality tests:
+.venv-eval/bin/python evaluation/harness/build_eval_data.py
+.venv-eval/bin/python evaluation/harness/run_eval.py --base-url http://127.0.0.1:30080/v1 \
+  --model sensenova-loop-9b-nvfp4 --out results/quality --concurrency 16
+```
+
+For the 12-case feature test, supply `--image` (the recorded image was TextVQA validation image 34730):
+`python runtime-v2/tests/greedy_outputs.py results/features.json --image /path/to/image.jpg`.
+Kernel differential tests run in the pinned SGLang container; see `tests/test_mhc_fused.py`.
+All run counts and exact prompts/lengths are in saved JSONs. Code/doc final tests use 4/8/16 requests at c=1/4/8;
+matched tests use 8 per level. Screening c=16 results are not final matched-sweep numbers.

runtime-v2/bench/fp4_gemm_bench.py

@@ -0,0 +1,47 @@
+"""Microbench NVFP4 W4A4 GEMM backends at SenseNova-Loop decoder shapes (run inside the SGLang image)."""
+import torch, sys
+from flashinfer import fp4_quantize, mm_fp4
+torch.manual_seed(0); dev = "cuda"
+SHAPES = {"gate_up": (4096, 24576), "down": (12288, 4096), "gdn_qkvz": (4096, 12288), "attn_qkv": (4096, 10240), "o_proj": (4096, 4096)}
+MS = [int(m) for m in (sys.argv[1] if len(sys.argv) > 1 else "1,4,8,16,32,64").split(",")]
+BACKENDS = ["cudnn", "cutlass", "b12x", "cake", "cutedsl_low_latency", "auto"]
+def gs(t): return (6.0 * 448.0 / t.abs().max().float()).float()
+def timeit(fn, n=50):
+    for _ in range(3): fn()
+    g = torch.cuda.CUDAGraph()
+    with torch.cuda.graph(g):
+        for _ in range(n): fn()
+    torch.cuda.synchronize(); s, e = torch.cuda.Event(True), torch.cuda.Event(True)
+    s.record(); g.replay(); e.record(); torch.cuda.synchronize(); return s.elapsed_time(e) / n * 1e3
+kda = None
+try:
+    import sglang.kernels.kda_kernels.qwen3x_nvfp4_gemm as q
+    q._has_runtime = lambda d: True
+    from sglang.kernels.ops.gemm import try_qwen3x_nvfp4_gemm as kda
+except Exception as ex: print("kda import fail", ex)
+tot = {}
+for name, (K, N) in SHAPES.items():
+    w = torch.randn(N, K, device=dev, dtype=torch.bfloat16) * 0.02; wg = gs(w)
+    wq, ws = fp4_quantize(w, wg, 16, False, True)
+    gbytes = N * K * (0.5 + 1 / 16) / 1e9
+    for M in MS:
+        x = torch.randn(M, K, device=dev, dtype=torch.bfloat16); xg = gs(x)
+        xq, xs = fp4_quantize(x, xg, 16, False, True); alpha = (1.0 / (xg * wg)).reshape(1)
+        ref = (x.float() @ w.float().t())
+        row = []
+        for be in BACKENDS + (["kda"] if kda else []):
+            try:
+                out = torch.empty(M, N, device=dev, dtype=torch.bfloat16)
+                if be == "kda":
+                    f = lambda: kda(xq, wq, xs, ws, alpha, torch.bfloat16, N)
+                    if f() is None: raise RuntimeError("unsupported shape")
+                else:
+                    f = lambda: mm_fp4(xq, wq.t(), xs, ws.t(), alpha, torch.bfloat16, out, backend=be)
+                o = f(); o = out if o is None else o
+                err = ((o.float() - ref).norm() / ref.norm()).item()
+                us = timeit(f); row.append(f"{be}={us:6.1f}us({gbytes/us*1e6:4.0f}GB/s,e{err:.3f})")
+                tot.setdefault((be, M), 0); tot[(be, M)] += us
+            except Exception as ex:
+                row.append(f"{be}=FAIL[{str(ex)[:40]}]")
+        print(f"{name:9s} M={M:3d} " + " ".join(row), flush=True)
+print("sum over 5 shapes (us):", {f"{b}@{m}": round(v, 1) for (b, m), v in sorted(tot.items(), key=lambda t: (t[0][1], t[1]))})

runtime-v2/bench/prof_drive.py

@@ -0,0 +1,19 @@
+"""Drive a short torch-profiler capture: warmup, then start_profile(num_steps) + N concurrent requests."""
+import sys, time, json, threading, requests
+sys.path.insert(0, "/home/tux/sensenova-loop-nvfp4/eval")
+from perf_bench import build_prompt_text
+base = "http://127.0.0.1:30080"
+conc = int(sys.argv[1]); steps = int(sys.argv[2]); prefix = sys.argv[3]
+text, _, _ = build_prompt_text(1024, "/home/tux/sensenova-loop-nvfp4/upstream/SenseNova-Loop-9B-Preview/tokenizer.json")
+def req(i, n=64):
+    r = requests.post(base + "/v1/chat/completions", json={"model": "x", "messages": [{"role": "user", "content": f"[{time.time()}-{i}] " + text}],
+        "max_tokens": n, "min_tokens": n, "ignore_eos": True, "temperature": 0, "chat_template_kwargs": {"enable_thinking": False}}, timeout=600)
+    return r.json()["usage"]
+print("warm", req(-1, 32))
+ths = [threading.Thread(target=req, args=(i, 600)) for i in range(conc)]
+for t in ths: t.start()
+time.sleep(2.0 + 0.5 * conc)  # past prefill, into decode
+r = requests.post(base + "/start_profile", json={"num_steps": steps, "activities": ["CPU", "GPU"], "profile_prefix": prefix})
+print("profile", r.status_code, r.text[:200])
+for t in ths: t.join()
+print(json.dumps(requests.get(base + "/get_server_info").json().get("internal_states", [{}])[0].get("avg_spec_accept_length")))

runtime-v2/bench/report_tables.py

@@ -0,0 +1,41 @@
+"""Before/after markdown tables from runs/final_<label>/ (perf_matched.json, screen_{code,doc}.json, eval)."""
+import json, sys
+from pathlib import Path
+labels = sys.argv[1:] or ["frozen_v1", "v2_fast", "v2_exact"]
+R = Path(__file__).resolve().parents[1] / "runs"
+def load(l, f):
+    p = R / f"final_{l}" / f
+    return json.load(open(p)) if p.exists() else None
+out = []
+out.append("### Matched workload (eval/perf_bench.py: ~1051 prompt tok, 256 out, unique prompts, 8 req/level, 2 warmup)\n")
+out.append("| runtime | c | aggregate tok/s | per-req decode tok/s p50 | TTFT p50 / p90 (s) | ITL p50 / p90 (ms) | E2E p50 / p90 (s) | out tok min-max | errors |")
+out.append("|---|---:|---:|---:|---:|---:|---:|---:|---:|")
+for l in labels:
+    d = load(l, "perf_matched.json")
+    if not d: continue
+    for lv in d["levels"]:
+        out.append(f"| {l} | {lv['concurrency']} | {lv['aggregate_output_tok_s']:.1f} | {lv['decode_tok_s_per_req']['p50']:.1f} | "
+                   f"{lv['ttft_s']['p50']:.2f} / {lv['ttft_s']['p90']:.2f} | {lv['itl_s']['p50']*1e3:.0f} / {lv['itl_s']['p90']*1e3:.0f} | "
+                   f"{lv['e2e_s']['p50']:.1f} / {lv['e2e_s']['p90']:.1f} | {lv['output_tokens_min']}-{lv['output_tokens_max']} | {lv['n_errors']} |")
+for kind, desc in (("code", "short interactive coding/reasoning prompt (~90 tok in, 256 out)"), ("doc", "long document (~4k tok in, 256 out)")):
+    out.append(f"\n### {kind}: {desc}\n")
+    out.append("| runtime | c | n | aggregate tok/s | per-req decode p50 | TTFT p50 (ms) | ITL p50 / p95 (ms) | MTP accept len | errors |")
+    out.append("|---|---:|---:|---:|---:|---:|---:|---:|---:|")
+    for l in labels:
+        d = load(l, f"screen_{kind}.json")
+        if not d: continue
+        for lv in d["levels"]:
+            a = lv.get("spec_accept_len_req_mean")
+            out.append(f"| {l} | {lv['concurrency']} | {lv['n_requests']} | {lv['aggregate_output_tok_s']:.1f} | {lv['decode_tok_s_per_req_p50']:.1f} | "
+                       f"{lv['ttft_ms_p50']:.0f} | {lv['itl_ms_p50']:.0f} / {lv['itl_ms_p95']:.0f} | {a:.2f} | {lv['n_errors']} |" if a else
+                       f"| {l} | {lv['concurrency']} | {lv['n_requests']} | {lv['aggregate_output_tok_s']:.1f} | {lv['decode_tok_s_per_req_p50']:.1f} | {lv['ttft_ms_p50']:.0f} | {lv['itl_ms_p50']:.0f} / {lv['itl_ms_p95']:.0f} | - | {lv['n_errors']} |")
+out.append("\n### 300-item regression (thinking off, temp 0, eval concurrency 16)\n")
+out.append("| runtime | ifeval | gsm8k | mmlu_pro | humaneval | tools | vqa | pooled | truncated | errors |")
+out.append("|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|")
+for l in labels:
+    s = load(l, "eval/summary.json")
+    if not s: continue
+    t = s["tasks"]; c = sum(round(v["accuracy"] * v["n"]) for v in t.values()); n = sum(v["n"] for v in t.values())
+    tr = sum(v.get("n_truncated", v.get("truncated", 0)) or 0 for v in t.values()); er = sum(v.get("n_errors", 0) for v in t.values())
+    out.append(f"| {l} | " + " | ".join(f"{t[k]['accuracy']:.3f}" for k in ("ifeval", "gsm8k", "mmlu_pro", "humaneval", "tools", "vqa")) + f" | {c}/{n} = {c/n:.3f} | {tr} | {er} |")
+print("\n".join(out))

runtime-v2/bench/screen.py

@@ -0,0 +1,166 @@
+#!/usr/bin/env python
+"""Fast A/B screening benchmark for the SGLang server (OpenAI chat API, streaming).
+
+  .venv-eval/bin/python runtime-v2/bench/screen.py --base-url http://127.0.0.1:30080 \
+      --model sensenova-loop-9b-nvfp4 --concurrency 1,4,8 --num-requests 4 --output-tokens 256 \
+      --prompt-kind synthetic --repeat 2 --out runtime-v2/runs/screen_baseline.json
+
+Prompts follow eval/perf_bench.py: unique per-request prefix (plus a per-run tag, so no prefix-cache
+hits even across runs), temperature 0, ignore_eos + min_tokens = max_tokens, thinking off.
+--prompt-kind: synthetic (perf_bench text, default 1024 tok), code (fixed coding prompt), doc (~4k-token
+document + summarization instruction). MTP acceptance: per-request `sglext.spec_tokens_details`
+(requested via return_spec_tokens_details) and the server's cumulative /server_info avg_spec_accept_length.
+"""
+from __future__ import annotations
+
+import argparse, json, statistics, sys, time, uuid
+from concurrent.futures import ThreadPoolExecutor
+from pathlib import Path
+
+import requests
+
+ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(ROOT / "evaluation" / "harness"))
+from perf_bench import DEFAULT_TOK, build_prompt_text, pct  # noqa: E402
+
+CODE_PROMPT = ("Implement an LRU cache in Python with O(1) get and put (no functools), explain the design "
+               "and its complexity, then write a thorough pytest test suite covering eviction order, updates "
+               "of existing keys, capacity 1 and invalid capacity. Finally discuss thread-safety trade-offs.")
+
+
+def make_prompt(args):
+    if args.prompt_kind == "code":
+        return CODE_PROMPT, ""
+    n = args.input_tokens or (4096 if args.prompt_kind == "doc" else 1024)
+    text, _, _ = build_prompt_text(n, args.tokenizer)
+    if args.prompt_kind == "doc":
+        return "Document:\n" + text, "\n\nSummarize the document above in five bullet points, then list its recurring themes."
+    return text, "\n\nContinue this text."
+
+
+def one_request(args, s, body_text, suffix, tag):
+    body = {"model": args.model, "stream": True, "stream_options": {"include_usage": True},
+            "messages": [{"role": "user", "content": f"[req {tag}] " + body_text + suffix}],
+            "max_tokens": args.output_tokens, "temperature": 0.0, "ignore_eos": True,
+            "min_tokens": args.output_tokens, "return_spec_tokens_details": True,
+            "chat_template_kwargs": {"enable_thinking": False}}
+    t0 = time.perf_counter()
+    times, usage, spec = [], None, None
+    try:
+        with s.post(args.api + "/chat/completions", json=body, stream=True, timeout=args.timeout) as r:
+            if r.status_code != 200:
+                return {"ok": False, "error": f"HTTP {r.status_code}: {r.text[:300]}"}
+            for line in r.iter_lines():
+                if not line.startswith(b"data:"):
+                    continue
+                data = line[5:].strip()
+                if data == b"[DONE]":
+                    break
+                j = json.loads(data)
+                usage = j.get("usage") or usage
+                spec = ((j.get("sglext") or {}).get("spec_tokens_details")) or spec
+                for ch in j.get("choices") or []:
+                    d = ch.get("delta") or {}
+                    if d.get("content") or d.get("reasoning_content") or d.get("reasoning"):
+                        times.append(time.perf_counter())
+    except Exception as e:  # noqa: BLE001
+        return {"ok": False, "error": f"{type(e).__name__}: {e}"}
+    if not times:
+        return {"ok": False, "error": "no tokens received"}
+    n_out = (usage or {}).get("completion_tokens") or len(times)
+    dt = times[-1] - times[0]
+    if isinstance(spec, list):
+        spec = spec[0] if spec else None
+    return {"ok": True, "ttft_s": times[0] - t0, "e2e_s": time.perf_counter() - t0, "out": n_out,
+            "prompt_tokens": (usage or {}).get("prompt_tokens"),
+            "decode_tok_s": (n_out - 1) / dt if dt > 0 and n_out > 1 else None,
+            "itl_s": dt / (n_out - 1) if dt > 0 and n_out > 1 else None,
+            "gaps": [b - a for a, b in zip(times, times[1:])],
+            "accept_len": (spec or {}).get("spec_accept_length")}
+
+
+def server_accept(args):
+    for path in ("/server_info", "/get_server_info"):
+        try:
+            j = requests.get(args.root + path, timeout=10).json()
+            for st in j.get("internal_states") or []:
+                if "avg_spec_accept_length" in st:
+                    return st["avg_spec_accept_length"]
+            return None
+        except Exception:  # noqa: BLE001
+            continue
+    return None
+
+
+def run_level(args, s, prompt, c, rep):
+    n = args.num_requests or max(4, 2 * c)
+    tags = [f"{args.run_tag}-c{c}-r{rep}-{i:05d}" for i in range(n)]
+    acc0 = server_accept(args)
+    t0 = time.perf_counter()
+    with ThreadPoolExecutor(c) as ex:
+        res = list(ex.map(lambda t: one_request(args, s, prompt[0], prompt[1], t), tags))
+    wall = time.perf_counter() - t0
+    ok = [r for r in res if r["ok"]]
+    col = lambda k: [r[k] for r in ok if r.get(k) is not None]  # noqa: E731
+    gaps = [g for r in ok for g in r["gaps"]]
+    ms = lambda x: None if x is None else 1000 * x  # noqa: E731
+    acc = col("accept_len")
+    return {"concurrency": c, "repeat": rep, "n_requests": n, "n_ok": len(ok), "n_errors": n - len(ok),
+            "errors": [r["error"] for r in res if not r["ok"]][:5], "wall_s": wall,
+            "aggregate_output_tok_s": sum(col("out")) / wall if wall else None,
+            "decode_tok_s_per_req_p50": pct(col("decode_tok_s"), 0.5),
+            "ttft_ms_p50": ms(pct(col("ttft_s"), 0.5)),
+            "itl_ms_p50": ms(pct(col("itl_s"), 0.5)), "itl_ms_p95": ms(pct(col("itl_s"), 0.95)),
+            "chunk_gap_ms_p50": ms(pct(gaps, 0.5)), "chunk_gap_ms_p95": ms(pct(gaps, 0.95)),
+            "out_tokens_mean": statistics.fmean(col("out")) if ok else None,
+            "prompt_tokens_mean": statistics.fmean(col("prompt_tokens")) if col("prompt_tokens") else None,
+            "spec_accept_len_req_mean": statistics.fmean(acc) if acc else None,
+            "server_avg_accept_len_before": acc0, "server_avg_accept_len_after": server_accept(args)}
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--base-url", required=True, help="server root or .../v1")
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--concurrency", default="1,4,8")
+    ap.add_argument("--num-requests", type=int, default=0, help="per level; 0 => max(4, 2*c)")
+    ap.add_argument("--input-tokens", type=int, default=0, help="0 => 1024 (synthetic) / 4096 (doc)")
+    ap.add_argument("--output-tokens", type=int, default=256)
+    ap.add_argument("--warmup", type=int, default=1)
+    ap.add_argument("--repeat", type=int, default=1)
+    ap.add_argument("--prompt-kind", choices=["synthetic", "code", "doc"], default="synthetic")
+    ap.add_argument("--tokenizer", default=str(DEFAULT_TOK))
+    ap.add_argument("--timeout", type=float, default=600)
+    ap.add_argument("--label", default="")
+    ap.add_argument("--out", required=True)
+    args = ap.parse_args()
+    args.root = args.base_url.rstrip("/").removesuffix("/v1")
+    args.api = args.root + "/v1"
+    args.run_tag = uuid.uuid4().hex[:8]
+    prompt = make_prompt(args)
+    levels = [int(x) for x in args.concurrency.split(",")]
+    s = requests.Session()
+    s.mount("http://", requests.adapters.HTTPAdapter(pool_maxsize=max(levels) * 2 + 4))
+    for i in range(args.warmup):
+        w = one_request(args, s, prompt[0], prompt[1], f"{args.run_tag}-warm{i}")
+        if not w["ok"]:
+            print("warmup error:", w["error"], file=sys.stderr)
+    out = {"label": args.label, "time": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+           "config": {k: v for k, v in vars(args).items()}, "levels": []}
+    f = lambda x: "  n/a" if x is None else f"{x:7.2f}"  # noqa: E731
+    for c in levels:
+        for rep in range(args.repeat):
+            lv = run_level(args, s, prompt, c, rep)
+            out["levels"].append(lv)
+            print(f"c={c:2d} r={rep} ok={lv['n_ok']}/{lv['n_requests']} agg={f(lv['aggregate_output_tok_s'])} tok/s "
+                  f"dec/req={f(lv['decode_tok_s_per_req_p50'])} TTFT={f(lv['ttft_ms_p50'])}ms "
+                  f"ITL p50/p95={f(lv['itl_ms_p50'])}/{f(lv['itl_ms_p95'])}ms "
+                  f"accept={f(lv['spec_accept_len_req_mean'])} srv={f(lv['server_avg_accept_len_after'])}",
+                  flush=True)
+    Path(args.out).parent.mkdir(parents=True, exist_ok=True)
+    Path(args.out).write_text(json.dumps(out, indent=2))
+    print("wrote", args.out)
+
+
+if __name__ == "__main__":
+    main()

runtime-v2/bench/trace_graphs.py

@@ -0,0 +1,28 @@
+"""Attribute GPU kernels to each cudaGraphLaunch (via correlation id) and summarize per replay site."""
+import gzip, json, sys, collections
+sys.path.insert(0, __file__.rsplit("/", 1)[0]); from trace_summary_lib import cat
+d = json.load(gzip.open(sys.argv[1])); ev = [e for e in d["traceEvents"] if e.get("ph") == "X"]
+launch = {e["args"].get("correlation"): e for e in ev if e.get("cat") == "cuda_runtime" and e["name"] == "cudaGraphLaunch"}
+k = [e for e in ev if e.get("cat") == "kernel"]
+per = collections.defaultdict(list)
+for e in k:
+    c = e["args"].get("correlation")
+    per[c if c in launch else None].append(e)
+# order launches by time; label by position in a cycle
+rows = []
+for c, L in launch.items():
+    ks = per.get(c, [])
+    if not ks: continue
+    t0 = min(x["ts"] for x in ks); t1 = max(x["ts"] + x["dur"] for x in ks)
+    cc = collections.Counter(); g = collections.Counter()
+    for x in ks:
+        cc[cat(x["name"])] += x["dur"]
+        if "gemvx" in x["name"] or "wmma" in x["name"]: g[(x["name"][60:100], x["dur"] // 100 * 100)] += 1
+    rows.append((L["ts"], len(ks), (t1 - t0) / 1e3, sum(x["dur"] for x in ks) / 1e3, dict((a, round(b / 1e3, 1)) for a, b in cc.most_common(5)), g.most_common(4)))
+rows.sort()
+for r in rows[:9]: print(f"n={r[1]:6d} span={r[2]:7.1f}ms busy={r[3]:7.1f}ms {r[4]}\n      {r[5]}")
+nk = per.get(None, [])
+print("non-graph kernels:", len(nk), round(sum(x['dur'] for x in nk) / 1e3, 1), "ms")
+cc = collections.Counter()
+for x in nk: cc[x["name"][:90]] += x["dur"]
+for n, v in cc.most_common(8): print(f"   {v/1e3:7.1f} {n}")

runtime-v2/bench/trace_summary.py

@@ -0,0 +1,36 @@
+"""Summarize a torch-profiler chrome trace: GPU kernel time by category, busy vs wall, per-graph-replay breakdown."""
+import gzip, json, sys, re, collections
+d = json.load(gzip.open(sys.argv[1]))
+ev = [e for e in d["traceEvents"] if e.get("ph") == "X"]
+k = [e for e in ev if e.get("cat") == "kernel"]
+cpu = [e for e in ev if e.get("cat") in ("cpu_op", "user_annotation", "python_function")]
+def cat(n):
+    n2 = n.lower()
+    if "block_scaled" in n2 or "fp4" in n2 or "nvfp4" in n2 or "cudnn" in n2 and "gemm" in n2 or "sm120" in n2 or "quantize_with_block" in n2 or "cvt_fp16_to_fp4" in n2 or "e2m1" in n2: return "FP4 GEMM/quant"
+    if "gemv" in n2 or "gemm" in n2 or "sgemm" in n2 or "cublas" in n2 or "dot_kernel" in n2 or "wmma" in n2 or "splitk" in n2 or "nvjet" in n2: return "other GEMM/GEMV"
+    if "delta_rule" in n2 or "causal_conv" in n2 or "gdn" in n2 or "fla" in n2 or "chunk_" in n2: return "GDN/recurrent"
+    if "attn" in n2 or "attention" in n2 or "flash" in n2 and "norm" not in n2 or "decode" in n2 and "rope" not in n2: return "attention"
+    if any(s in n2 for s in ("exp_kernel", "log_kernel", "masked_fill", "absfunctor", "compareeq", "maxnan", "sum_functor", "add<float>", "sigmoid", "functor_add", "logsumexp")): return "elementwise/reduce (fp32, mHC-like)"
+    if "norm" in n2: return "norm"
+    return "other"
+tot = collections.Counter(); cnt = collections.Counter(); byname = collections.Counter(); bync = collections.Counter()
+for e in k:
+    c = cat(e["name"]); tot[c] += e["dur"]; cnt[c] += 1; byname[e["name"][:110]] += e["dur"]; bync[e["name"][:110]] += 1
+t0 = min(e["ts"] for e in k); t1 = max(e["ts"] + e["dur"] for e in k)
+busy = sum(e["dur"] for e in k)
+print(f"kernels={len(k)} window={((t1-t0)/1e3):.1f}ms gpu_kernel_sum={busy/1e3:.1f}ms ({100*busy/(t1-t0):.0f}% of window)")
+for c, v in tot.most_common(): print(f"  {c:40s} {v/1e3:9.1f} ms  {100*v/busy:5.1f}%  n={cnt[c]}")
+print("top kernels:")
+for n, v in byname.most_common(22): print(f"  {v/1e3:8.1f} ms {bync[n]:7d}x  {n}")
+# graph replays / forward annotations
+ann = collections.Counter(); annd = collections.Counter()
+for e in cpu:
+    n = e["name"]
+    if any(s in n for s in ("replay", "forward_", "run_batch", "verify", "draft", "cudaGraphLaunch")):
+        ann[n[:80]] += 1; annd[n[:80]] += e["dur"]
+print("cpu annotations (count, total ms):")
+for n, c in ann.most_common(30): print(f"  {c:5d} {annd[n]/1e3:9.1f}  {n}")
+rt = [e for e in ev if e.get("cat") == "cuda_runtime"]
+rc = collections.Counter(); rd = collections.Counter()
+for e in rt: rc[e["name"]] += 1; rd[e["name"]] += e["dur"]
+print("cuda runtime:", [(n, rc[n], round(rd[n]/1e3, 1)) for n, _ in rd.most_common(8)])

runtime-v2/bench/trace_summary_lib.py

@@ -0,0 +1,9 @@
+def cat(n):
+    n2 = n.lower()
+    if "block_scaled" in n2 or "fp4" in n2 or "nvfp4" in n2 or "cudnn" in n2 and "gemm" in n2 or "sm120" in n2 or "quantize_with_block" in n2 or "cvt_fp16_to_fp4" in n2 or "e2m1" in n2: return "FP4 GEMM/quant"
+    if "gemv" in n2 or "gemm" in n2 or "sgemm" in n2 or "cublas" in n2 or "dot_kernel" in n2 or "wmma" in n2 or "splitk" in n2 or "nvjet" in n2: return "other GEMM/GEMV"
+    if "delta_rule" in n2 or "causal_conv" in n2 or "gdn" in n2 or "fla" in n2 or "chunk_" in n2: return "GDN/recurrent"
+    if "attn" in n2 or "attention" in n2 or "flash" in n2 and "norm" not in n2 or "decode" in n2 and "rope" not in n2: return "attention"
+    if any(s in n2 for s in ("exp_kernel", "log_kernel", "masked_fill", "absfunctor", "compareeq", "maxnan", "sum_functor", "add<float>", "sigmoid", "functor_add", "logsumexp")): return "elementwise/reduce (fp32, mHC-like)"
+    if "norm" in n2: return "norm"
+    return "other"

runtime-v2/bench/wait_ready.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# wait until server on ${PORT:-30080} answers a generation; prints seconds waited
+P=${PORT:-30080}; t0=$(date +%s)
+for i in $(seq 1 360); do
+  if curl -sf -m 5 http://127.0.0.1:$P/v1/models >/dev/null; then echo "ready after $(( $(date +%s)-t0 ))s"; exit 0; fi
+  if ! docker ps --format '{{.Names}}' | grep -qx "${NAME:-sensenova-loop-nvfp4}"; then echo "container died"; docker logs --tail 40 "${NAME:-sensenova-loop-nvfp4}"; exit 1; fi
+  sleep 5
+done; echo timeout; exit 1

runtime-v2/overlay/python/sglang/srt/configs/__init__.py

@@ -0,0 +1,187 @@
+from sglang.srt.configs.afmoe import AfmoeConfig
+from sglang.srt.configs.bailing_hybrid import BailingHybridConfig, BailingMoeV3VLConfig
+from sglang.srt.configs.bailing_moe_v2 import BailingMM2Config
+from sglang.srt.configs.beam import BeamConfig
+from sglang.srt.configs.chatglm import ChatGLMConfig
+from sglang.srt.configs.cohere2_moe import Cohere2MoeConfig
+from sglang.srt.configs.cosmos3 import (
+    Cosmos3Config,
+    Cosmos3EdgeConfig,
+    Cosmos3EdgeProjectorConfig,
+    Cosmos3EdgeTextConfig,
+    Cosmos3EdgeVisionConfig,
+)
+from sglang.srt.configs.dbrx import DbrxConfig
+from sglang.srt.configs.deepseekvl2 import DeepseekVL2Config
+from sglang.srt.configs.dots3 import Dots3Config
+from sglang.srt.configs.dots_ocr import DotsOCRConfig
+from sglang.srt.configs.dots_vlm import DotsVLMConfig
+from sglang.srt.configs.exaone import ExaoneConfig
+from sglang.srt.configs.falcon_h1 import FalconH1Config
+from sglang.srt.configs.flint import FlintConfig
+from sglang.srt.configs.gigachat35 import GigaChat35Config
+from sglang.srt.configs.glm5_next import Glm5NextConfig, Glm5NextTextConfig
+from sglang.srt.configs.granitemoehybrid import GraniteMoeHybridConfig
+from sglang.srt.configs.hy_v4 import HYV4Config
+from sglang.srt.configs.inkling import (
+    InklingAudioConfig,
+    InklingMMConfig,
+    InklingModelConfig,
+    InklingVisionConfig,
+)
+from sglang.srt.configs.interns2_mobius import (
+    InternS2MobiusConfig,
+    InternS2MobiusTextConfig,
+    InternS2MobiusVisionConfig,
+)
+from sglang.srt.configs.interns2preview import InternS2PreviewConfig
+from sglang.srt.configs.iquest_q1 import (
+    IQuestQ1Config,
+    IQuestQ1MTPConfig,
+)
+from sglang.srt.configs.janus_pro import MultiModalityConfig
+from sglang.srt.configs.jet_nemotron import JetNemotronConfig
+from sglang.srt.configs.jet_vlm import JetVLMConfig
+from sglang.srt.configs.k2_horizon import K2HorizonConfig, XllmConfig
+from sglang.srt.configs.kimi_k3 import KimiK3Config
+from sglang.srt.configs.kimi_k25 import KimiK25Config
+from sglang.srt.configs.kimi_linear import KimiLinearConfig
+from sglang.srt.configs.kimi_vl import KimiVLConfig
+from sglang.srt.configs.kimi_vl_moonvit import MoonViTConfig
+from sglang.srt.configs.laguna import LagunaConfig
+from sglang.srt.configs.lfm2 import Lfm2Config
+from sglang.srt.configs.lfm2_moe import Lfm2MoeConfig
+from sglang.srt.configs.lfm2_vl import Lfm2VlConfig
+from sglang.srt.configs.locate_anything import LocateAnythingConfig
+from sglang.srt.configs.longcat_flash import LongcatFlashConfig
+from sglang.srt.configs.mamba import FalconMambaConfig, MambaConfig
+from sglang.srt.configs.mamba2 import Mamba2Config
+from sglang.srt.configs.minicpm import MiniCPMHybridConfig
+from sglang.srt.configs.minicpmv4_6 import MiniCPMV4_6Config, MiniCPMV4_6VisionConfig
+from sglang.srt.configs.minicpmv4_7 import MiniCPMV4_7Config
+from sglang.srt.configs.minimax_vl import MiniMaxM3VLConfig
+from sglang.srt.configs.muse_glimmer import (
+    MuseGlimmerAssistantConfig,
+    MuseGlimmerConfig,
+)
+from sglang.srt.configs.muse_spark import (
+    MuseSparkConfig,
+    MuseSparkTextConfig,
+)
+from sglang.srt.configs.nanbeige import NanbeigeConfig
+from sglang.srt.configs.nano_nemotron_vl import (
+    NemotronH_Nano_Omni_Reasoning_V3_Config,
+    NemotronH_Nano_VL_V2_Config,
+    NemotronH_Omni_Reasoning_V3_Config,
+)
+from sglang.srt.configs.nemotron_h import NemotronHConfig, NemotronHPuzzleConfig
+from sglang.srt.configs.olmo3 import Olmo3Config
+from sglang.srt.configs.qwen3_5 import (
+    Qwen3_5Config,
+    Qwen3_5MoeConfig,
+    Qwen3_5MoeTextConfig,
+    Qwen3_5TextConfig,
+)
+from sglang.srt.configs.qwen3_asr import Qwen3ASRConfig
+from sglang.srt.configs.qwen3_next import Qwen3NextConfig
+from sglang.srt.configs.qwen4_exp import Qwen4ExpConfig, Qwen4ExpTextConfig
+from sglang.srt.configs.sensenova_loop import SenseNovaLoopConfig
+from sglang.srt.configs.spark2_5 import Spark2_5Config
+from sglang.srt.configs.step3_vl import (
+    Step3TextConfig,
+    Step3VisionEncoderConfig,
+    Step3VLConfig,
+)
+from sglang.srt.configs.step3p5 import Step3p5Config
+from sglang.srt.configs.step3p7 import Step3p7Config
+from sglang.srt.configs.unlimited_ocr import UnlimitedVLConfig
+from sglang.srt.configs.zaya import ZayaConfig
+
+__all__ = [
+    "AfmoeConfig",
+    "BailingHybridConfig",
+    "BailingMM2Config",
+    "BailingMoeV3VLConfig",
+    "BeamConfig",
+    "ExaoneConfig",
+    "ChatGLMConfig",
+    "Cosmos3Config",
+    "Cosmos3EdgeConfig",
+    "Cosmos3EdgeTextConfig",
+    "Cosmos3EdgeVisionConfig",
+    "Cosmos3EdgeProjectorConfig",
+    "DbrxConfig",
+    "DeepseekVL2Config",
+    "LongcatFlashConfig",
+    "MultiModalityConfig",
+    "KimiVLConfig",
+    "MoonViTConfig",
+    "K2HorizonConfig",
+    "XllmConfig",
+    "Step3VLConfig",
+    "Step3TextConfig",
+    "Step3VisionEncoderConfig",
+    "Olmo3Config",
+    "MuseGlimmerConfig",
+    "MuseGlimmerAssistantConfig",
+    "MuseSparkConfig",
+    "MuseSparkTextConfig",
+    "Glm5NextConfig",
+    "Glm5NextTextConfig",
+    "KimiLinearConfig",
+    "KimiK3Config",
+    "KimiK25Config",
+    "LagunaConfig",
+    "Qwen3NextConfig",
+    "Qwen4ExpConfig",
+    "Qwen4ExpTextConfig",
+    "Qwen3_5Config",
+    "Qwen3_5MoeConfig",
+    "Qwen3_5TextConfig",
+    "Qwen3_5MoeTextConfig",
+    "SenseNovaLoopConfig",
+    "InternS2PreviewConfig",
+    "IQuestQ1Config",
+    "FlintConfig",
+    "IQuestQ1MTPConfig",
+    "InternS2MobiusConfig",
+    "InternS2MobiusTextConfig",
+    "InternS2MobiusVisionConfig",
+    "DotsVLMConfig",
+    "DotsOCRConfig",
+    "Dots3Config",
+    "FalconH1Config",
+    "FalconMambaConfig",
+    "GigaChat35Config",
+    "GraniteMoeHybridConfig",
+    "HYV4Config",
+    "MambaConfig",
+    "Mamba2Config",
+    "Lfm2Config",
+    "Lfm2MoeConfig",
+    "Lfm2VlConfig",
+    "LocateAnythingConfig",
+    "MiniCPMV4_6Config",
+    "MiniCPMV4_6VisionConfig",
+    "MiniCPMV4_7Config",
+    "NemotronHConfig",
+    "NemotronHPuzzleConfig",
+    "NemotronH_Nano_VL_V2_Config",
+    "NemotronH_Nano_Omni_Reasoning_V3_Config",
+    "NemotronH_Omni_Reasoning_V3_Config",
+    "NanbeigeConfig",
+    "JetNemotronConfig",
+    "JetVLMConfig",
+    "MiniCPMHybridConfig",
+    "Step3p5Config",
+    "MiniMaxM3VLConfig",
+    "Spark2_5Config",
+    "Step3p7Config",
+    "Qwen3ASRConfig",
+    "InklingAudioConfig",
+    "InklingMMConfig",
+    "InklingModelConfig",
+    "InklingVisionConfig",
+    "UnlimitedVLConfig",
+    "ZayaConfig",
+]

runtime-v2/overlay/python/sglang/srt/configs/sensenova_loop.py

@@ -0,0 +1,77 @@
+# Copyright 2026 SGLang Team
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+from sglang.srt.configs.linear_attn_model_registry import (
+    LinearAttnModelSpec,
+    register_linear_attn_model,
+)
+from sglang.srt.configs.qwen3_5 import Qwen3_5Config, Qwen3_5TextConfig
+
+
+class SenseNovaLoopTextConfig(Qwen3_5TextConfig):
+    """Serving slots follow execution order; checkpoint layers remain physical."""
+
+    def __init__(self, execution_plan=(), mhc=None, layer_types=None, **kwargs):
+        super().__init__(layer_types=layer_types, **kwargs)
+        self.execution_plan = tuple(execution_plan)
+        self.mhc = mhc
+        self.layer_types = layer_types
+
+    @property
+    def layers_block_type(self):
+        return [
+            "attention" if kind == "full_attention" else "linear_attention"
+            for kind in self.layer_types
+        ]
+
+
+class SenseNovaLoopConfig(Qwen3_5Config):
+    model_type = "sensenova_loop"
+    has_no_defaults_at_init = True
+    sub_configs = {
+        **Qwen3_5Config.sub_configs,
+        "text_config": SenseNovaLoopTextConfig,
+    }
+
+    def __init__(self, loop_settings=None, mhc=None, **kwargs):
+        super().__init__(**kwargs)
+        self.loop_settings = loop_settings
+        self.mhc = mhc
+        text = self.text_config.to_dict()
+        depth = text.get("physical_num_hidden_layers", text["num_hidden_layers"])
+        kinds = text.get("physical_layer_types", text["layer_types"])
+        start, end, count = (
+            loop_settings[key] for key in ("start", "end", "eval_count")
+        )
+        self.execution_plan = (
+            tuple(range(start))
+            + tuple(range(start, end)) * count
+            + tuple(range(end, depth))
+        )
+        text.update(
+            physical_num_hidden_layers=depth,
+            physical_layer_types=kinds,
+            num_hidden_layers=len(self.execution_plan),
+            layer_types=[kinds[i] for i in self.execution_plan],
+            execution_plan=self.execution_plan,
+            mhc=mhc,
+        )
+        self.text_config = SenseNovaLoopTextConfig(**text)
+        self.architectures = ["SenseNovaLoopForConditionalGeneration"]
+
+
+register_linear_attn_model(
+    LinearAttnModelSpec(
+        config_class=SenseNovaLoopTextConfig,
+        backend_class_name="sglang.srt.layers.attention.linear.gdn_backend.GDNAttnBackend",
+        arch_names=["SenseNovaLoopForConditionalGeneration"],
+        unwrap_text_config=True,
+        uses_mamba_radix_cache=True,
+        support_mamba_cache=True,
+        support_mamba_cache_extra_buffer=True,
+    )
+)

runtime-v2/overlay/python/sglang/srt/layers/rotary_embedding/mrope_rope_index.py

@@ -0,0 +1,837 @@
+"""get_rope_index implementations for Qwen2-VL/Qwen3-VL, Qwen3-Omni, GLM4V, Ernie4.5."""
+
+from __future__ import annotations
+
+import itertools
+from typing import Any, List, Optional, Tuple, Union
+
+import torch
+
+
+def _get_feat_extract_output_lengths(input_lengths):
+    """
+    Computes the output length of the convolutional layers and the output length of the audio encoder
+    """
+    input_lengths_leave = input_lengths % 100
+    feat_lengths = (input_lengths_leave - 1) // 2 + 1
+    output_lengths = (
+        ((feat_lengths - 1) // 2 + 1 - 1) // 2 + 1 + (input_lengths // 100) * 13
+    )
+    return output_lengths
+
+
+def _get_llm_pos_ids_for_vision(
+    st_idx, vision_idx, spatial_merge_size, t_index, grid_hs, grid_ws, device
+):
+    grid_h = grid_hs[vision_idx] // spatial_merge_size
+    grid_w = grid_ws[vision_idx] // spatial_merge_size
+
+    h_index = (
+        torch.arange(grid_h, device=device)
+        .view(1, -1, 1)
+        .expand(len(t_index), -1, grid_w)
+        .flatten()
+    )
+    w_index = (
+        torch.arange(grid_w, device=device)
+        .view(1, 1, -1)
+        .expand(len(t_index), grid_h, -1)
+        .flatten()
+    )
+    t_index = t_index.view(-1, 1).expand(-1, grid_h * grid_w).flatten()
+
+    llm_pos_ids = torch.stack([t_index, h_index, w_index], dim=0) + st_idx
+    return llm_pos_ids
+
+
+def get_rope_index(
+    spatial_merge_size: int,
+    image_token_id: int,
+    video_token_id: int,
+    vision_start_token_id: int,
+    model_type: str,
+    tokens_per_second: Optional[int] = None,
+    input_ids: Optional[torch.LongTensor] = None,
+    image_grid_thw: Optional[torch.LongTensor] = None,
+    video_grid_thw: Optional[torch.LongTensor] = None,
+    second_per_grid_ts: Optional[torch.Tensor] = None,
+    **kwargs,
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    if model_type == "qwen3_omni_moe":
+        return get_rope_index_qwen3_omni(
+            spatial_merge_size,
+            image_token_id,
+            video_token_id,
+            vision_start_token_id,
+            tokens_per_second,
+            input_ids,
+            image_grid_thw,
+            video_grid_thw,
+            second_per_grid_ts,
+            **kwargs,
+        )
+    if (
+        model_type.startswith("qwen3_vl")
+        or model_type.startswith("qwen3_vl_moe")
+        or model_type.startswith("qwen3_5")
+        or model_type in ("qwen4_exp", "sensenova_loop")
+        or model_type.startswith("interns2_mobius")
+        or model_type.startswith("cosmos3_omni")
+        or model_type.startswith("cosmos3_edge")
+        or model_type == "bailing_moe_v3_vl"
+    ) and video_grid_thw is not None:
+        video_grid_thw = torch.repeat_interleave(
+            video_grid_thw, video_grid_thw[:, 0], dim=0
+        )
+        video_grid_thw[:, 0] = 1
+
+    mrope_position_deltas = []
+    if input_ids is not None and (
+        image_grid_thw is not None or video_grid_thw is not None
+    ):
+        total_input_ids = input_ids
+        position_ids = torch.ones(
+            3,
+            input_ids.shape[0],
+            input_ids.shape[1],
+            dtype=input_ids.dtype,
+            device=input_ids.device,
+        )
+        image_index, video_index = 0, 0
+        for i, input_ids in enumerate(total_input_ids):
+            image_nums, video_nums = 0, 0
+            vision_start_indices = torch.argwhere(
+                input_ids == vision_start_token_id
+            ).squeeze(1)
+            vision_tokens = input_ids[vision_start_indices + 1]
+            image_nums = (vision_tokens == image_token_id).sum()
+            video_nums = (vision_tokens == video_token_id).sum()
+            input_tokens = input_ids.tolist()
+            llm_pos_ids_list: list = []
+            st = 0
+            remain_images, remain_videos = image_nums, video_nums
+            for _ in range(image_nums + video_nums):
+                if image_token_id in input_tokens and remain_images > 0:
+                    ed_image = input_tokens.index(image_token_id, st)
+                else:
+                    ed_image = len(input_tokens) + 1
+                if video_token_id in input_tokens and remain_videos > 0:
+                    ed_video = input_tokens.index(video_token_id, st)
+                else:
+                    ed_video = len(input_tokens) + 1
+                if ed_image < ed_video:
+                    t, h, w = (
+                        image_grid_thw[image_index][0],
+                        image_grid_thw[image_index][1],
+                        image_grid_thw[image_index][2],
+                    )
+                    second_per_grid_t = 0
+                    image_index += 1
+                    remain_images -= 1
+                    ed = ed_image
+                else:
+                    t, h, w = (
+                        video_grid_thw[video_index][0],
+                        video_grid_thw[video_index][1],
+                        video_grid_thw[video_index][2],
+                    )
+                    if second_per_grid_ts is not None:
+                        second_per_grid_t = second_per_grid_ts[video_index]
+                    else:
+                        second_per_grid_t = 1.0
+                    video_index += 1
+                    remain_videos -= 1
+                    ed = ed_video
+                t_int, h_int, w_int = int(t), int(h), int(w)
+                llm_grid_t = t_int
+                llm_grid_h = h_int // spatial_merge_size
+                llm_grid_w = w_int // spatial_merge_size
+                text_len = ed - st
+                st_idx = (
+                    llm_pos_ids_list[-1].max() + 1 if len(llm_pos_ids_list) > 0 else 0
+                )
+                llm_pos_ids_list.append(
+                    torch.arange(text_len).view(1, -1).expand(3, -1) + st_idx
+                )
+                if model_type in ("qwen2_5_vl", "paddleocr_vl"):
+                    range_tensor = torch.arange(llm_grid_t).view(-1, 1)
+                    expanded_range = range_tensor.expand(-1, llm_grid_h * llm_grid_w)
+                    time_tensor = expanded_range * second_per_grid_t * tokens_per_second
+                    t_index = time_tensor.long().flatten()
+                elif model_type in (
+                    "qwen2_vl",
+                    "qwen3_vl",
+                    "qwen3_vl_moe",
+                    "qwen3_5",
+                    "qwen3_5_moe",
+                    "sensenova_loop",
+                    "qwen4_exp",
+                    "intern_s2_preview",
+                    "interns2_mobius",
+                    "cosmos3_omni",
+                    "cosmos3_edge",
+                ):
+                    t_index = (
+                        torch.arange(llm_grid_t, device=position_ids.device)
+                        .view(-1, 1)
+                        .expand(llm_grid_t, llm_grid_h * llm_grid_w)
+                        .reshape(-1)
+                    )
+                else:
+                    raise RuntimeError(f"Unimplemented model type: {model_type}")
+                h_index = (
+                    torch.arange(llm_grid_h, device=position_ids.device)
+                    .view(1, -1, 1)
+                    .expand(llm_grid_t, llm_grid_h, llm_grid_w)
+                    .reshape(-1)
+                )
+                w_index = (
+                    torch.arange(llm_grid_w, device=position_ids.device)
+                    .view(1, 1, -1)
+                    .expand(llm_grid_t, llm_grid_h, llm_grid_w)
+                    .reshape(-1)
+                )
+                llm_pos_ids_list.append(
+                    torch.stack([t_index, h_index, w_index]) + text_len + st_idx
+                )
+                st = ed + llm_grid_t * llm_grid_h * llm_grid_w
+            if st < len(input_tokens):
+                st_idx = (
+                    llm_pos_ids_list[-1].max() + 1 if len(llm_pos_ids_list) > 0 else 0
+                )
+                text_len = len(input_tokens) - st
+                llm_pos_ids_list.append(
+                    torch.arange(text_len).view(1, -1).expand(3, -1) + st_idx
+                )
+            llm_positions = torch.cat(llm_pos_ids_list, dim=1).reshape(3, -1)
+            position_ids[..., i, :] = llm_positions.to(position_ids.device)
+            mrope_position_deltas.append(
+                llm_positions.max() + 1 - len(total_input_ids[i])
+            )
+        mrope_position_deltas = torch.tensor(
+            mrope_position_deltas, device=input_ids.device
+        ).unsqueeze(1)
+        return position_ids, mrope_position_deltas
+    else:
+        s = input_ids.shape[1]
+        position_ids = torch.arange(s)
+        position_ids = position_ids.unsqueeze(0).expand(3, -1, -1).to(input_ids.device)
+        max_position_ids = position_ids.amax(dim=0, keepdim=False)
+        mrope_position_deltas = max_position_ids.amax(-1, keepdim=True) + 1 - s
+        return position_ids, mrope_position_deltas
+
+
+def get_rope_index_qwen3_omni(
+    spatial_merge_size: int,
+    image_token_id: int,
+    video_token_id: int,
+    vision_start_token_id: int,
+    tokens_per_second: Optional[int] = None,
+    input_ids: Optional[torch.LongTensor] = None,
+    image_grid_thw: Optional[torch.LongTensor] = None,
+    video_grid_thw: Optional[torch.LongTensor] = None,
+    second_per_grid_ts: Optional[torch.Tensor] = None,
+    **kwargs,
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    audio_token_id = kwargs["audio_token_id"]
+    audio_start_token_id = kwargs["audio_start_token_id"]
+    position_id_per_seconds = kwargs["position_id_per_seconds"]
+    use_audio_in_video = kwargs.get("use_audio_in_video", False)
+    audio_seqlens = kwargs.get("audio_seqlens", None)
+    second_per_grids = second_per_grid_ts
+
+    mrope_position_deltas = []
+    if input_ids is not None and (
+        image_grid_thw is not None or video_grid_thw is not None
+    ):
+        total_input_ids = input_ids
+        position_ids = torch.zeros(
+            3,
+            input_ids.shape[0],
+            input_ids.shape[1],
+            dtype=torch.float,
+            device=input_ids.device,
+        )
+        image_idx, video_idx, audio_idx = 0, 0, 0
+        for i, current_input_ids in enumerate(total_input_ids):
+            image_nums, video_nums, audio_nums = 0, 0, 0
+            vision_start_indices = torch.argwhere(
+                current_input_ids == vision_start_token_id
+            ).squeeze(1)
+            if vision_start_indices.numel() > 0:
+                vision_tokens = current_input_ids[vision_start_indices + 1]
+                image_nums = (vision_tokens == image_token_id).sum()
+                video_nums = (
+                    (vision_tokens == audio_start_token_id).sum()
+                    if use_audio_in_video
+                    else (vision_tokens == video_token_id).sum()
+                )
+            audio_nums = torch.sum(current_input_ids == audio_start_token_id)
+            input_tokens = current_input_ids.tolist()
+            llm_pos_ids_list: list = []
+            st = 0
+            remain_images, remain_videos, remain_audios = (
+                image_nums,
+                video_nums,
+                audio_nums,
+            )
+            multimodal_nums = (
+                image_nums + audio_nums
+                if use_audio_in_video
+                else image_nums + video_nums + audio_nums
+            )
+            for _ in range(multimodal_nums):
+                st_idx = (
+                    llm_pos_ids_list[-1].max() + 1 if len(llm_pos_ids_list) > 0 else 0
+                )
+                ed_vision_start = (
+                    input_tokens.index(vision_start_token_id, st)
+                    if (
+                        (
+                            image_token_id in input_tokens
+                            or video_token_id in input_tokens
+                        )
+                        and (remain_videos > 0 or remain_images > 0)
+                    )
+                    else len(input_tokens) + 1
+                )
+                ed_audio_start = (
+                    input_tokens.index(audio_start_token_id, st)
+                    if (audio_token_id in input_tokens and remain_audios > 0)
+                    else len(input_tokens) + 1
+                )
+                min_ed = min(ed_vision_start, ed_audio_start)
+                text_len = min_ed - st
+                if text_len != 0:
+                    llm_pos_ids_list.append(
+                        torch.arange(text_len).view(1, -1).expand(3, -1) + st_idx
+                    )
+                    st_idx += text_len
+                if min_ed == ed_vision_start and ed_vision_start + 1 == ed_audio_start:
+                    bos_len, eos_len = 2, 2
+                else:
+                    bos_len, eos_len = 1, 1
+                llm_pos_ids_list.append(
+                    torch.arange(bos_len).view(1, -1).expand(3, -1) + st_idx
+                )
+                st_idx += bos_len
+                # Audio Only
+                if min_ed == ed_audio_start:
+                    audio_len = _get_feat_extract_output_lengths(
+                        audio_seqlens[audio_idx]
+                    )
+                    llm_pos_ids = (
+                        torch.arange(audio_len).view(1, -1).expand(3, -1) + st_idx
+                    )
+                    llm_pos_ids_list.append(llm_pos_ids)
+                    st += int(text_len + bos_len + audio_len + eos_len)
+                    audio_idx += 1
+                    remain_audios -= 1
+                # Image Only
+                elif (
+                    min_ed == ed_vision_start
+                    and current_input_ids[ed_vision_start + 1] == image_token_id
+                ):
+                    grid_t = image_grid_thw[image_idx][0]
+                    grid_hs = image_grid_thw[:, 1]
+                    grid_ws = image_grid_thw[:, 2]
+                    t_index = (
+                        torch.arange(grid_t) * 1 * position_id_per_seconds
+                    ).float()
+                    llm_pos_ids = _get_llm_pos_ids_for_vision(
+                        st_idx,
+                        image_idx,
+                        spatial_merge_size,
+                        t_index,
+                        grid_hs,
+                        grid_ws,
+                        input_ids.device,
+                    )
+                    image_len = image_grid_thw[image_idx].prod() // (
+                        spatial_merge_size**2
+                    )
+                    llm_pos_ids_list.append(llm_pos_ids)
+                    st += int(text_len + bos_len + image_len + eos_len)
+                    image_idx += 1
+                    remain_images -= 1
+                # Video Only
+                elif (
+                    min_ed == ed_vision_start
+                    and current_input_ids[ed_vision_start + 1] == video_token_id
+                ):
+                    grid_t = video_grid_thw[video_idx][0]
+                    grid_hs = video_grid_thw[:, 1]
+                    grid_ws = video_grid_thw[:, 2]
+                    t_index = (
+                        torch.arange(grid_t)
+                        * second_per_grids[video_idx].cpu().float()
+                        * position_id_per_seconds
+                    ).float()
+                    llm_pos_ids = _get_llm_pos_ids_for_vision(
+                        st_idx,
+                        video_idx,
+                        spatial_merge_size,
+                        t_index,
+                        grid_hs,
+                        grid_ws,
+                        input_ids.device,
+                    )
+                    video_len = video_grid_thw[video_idx].prod() // (
+                        spatial_merge_size**2
+                    )
+                    llm_pos_ids_list.append(llm_pos_ids)
+                    st += int(text_len + bos_len + video_len + eos_len)
+                    video_idx += 1
+                    remain_videos -= 1
+                # Audio in Video
+                elif (
+                    min_ed == ed_vision_start and ed_vision_start + 1 == ed_audio_start
+                ):
+                    audio_len = _get_feat_extract_output_lengths(
+                        audio_seqlens[audio_idx]
+                    )
+                    audio_llm_pos_ids = (
+                        torch.arange(audio_len).view(1, -1).expand(3, -1) + st_idx
+                    )
+                    grid_t = video_grid_thw[video_idx][0]
+                    grid_hs = video_grid_thw[:, 1]
+                    grid_ws = video_grid_thw[:, 2]
+                    t_index = (
+                        torch.arange(grid_t)
+                        * second_per_grids[video_idx].cpu().float()
+                        * position_id_per_seconds
+                    ).float()
+                    video_llm_pos_ids = _get_llm_pos_ids_for_vision(
+                        st_idx,
+                        video_idx,
+                        spatial_merge_size,
+                        t_index,
+                        grid_hs,
+                        grid_ws,
+                        input_ids.device,
+                    )
+                    video_data_index, audio_data_index = 0, 0
+                    while (
+                        video_data_index < video_llm_pos_ids.shape[-1]
+                        and audio_data_index < audio_llm_pos_ids.shape[-1]
+                    ):
+                        if (
+                            video_llm_pos_ids[0][video_data_index]
+                            <= audio_llm_pos_ids[0][audio_data_index]
+                        ):
+                            llm_pos_ids_list.append(
+                                video_llm_pos_ids[
+                                    :, video_data_index : video_data_index + 1
+                                ]
+                            )
+                            video_data_index += 1
+                        else:
+                            llm_pos_ids_list.append(
+                                audio_llm_pos_ids[
+                                    :, audio_data_index : audio_data_index + 1
+                                ]
+                            )
+                            audio_data_index += 1
+                    if video_data_index < video_llm_pos_ids.shape[-1]:
+                        llm_pos_ids_list.append(
+                            video_llm_pos_ids[
+                                :, video_data_index : video_llm_pos_ids.shape[-1]
+                            ]
+                        )
+                    if audio_data_index < audio_llm_pos_ids.shape[-1]:
+                        llm_pos_ids_list.append(
+                            audio_llm_pos_ids[
+                                :, audio_data_index : audio_llm_pos_ids.shape[-1]
+                            ]
+                        )
+                    video_len = video_grid_thw[video_idx].prod() // (
+                        spatial_merge_size**2
+                    )
+                    st += int(text_len + bos_len + audio_len + video_len + eos_len)
+                    audio_idx += 1
+                    video_idx += 1
+                    remain_videos -= 1
+                    remain_audios -= 1
+                st_idx = (
+                    llm_pos_ids_list[-1].max() + 1 if len(llm_pos_ids_list) > 0 else 0
+                )
+                llm_pos_ids_list.append(
+                    torch.arange(eos_len).view(1, -1).expand(3, -1) + st_idx
+                )
+            if st < len(input_tokens):
+                st_idx = (
+                    llm_pos_ids_list[-1].max() + 1 if len(llm_pos_ids_list) > 0 else 0
+                )
+                text_len = len(input_tokens) - st
+                llm_pos_ids_list.append(
+                    torch.arange(text_len).view(1, -1).expand(3, -1) + st_idx
+                )
+            llm_positions = torch.cat(
+                [item.float() for item in llm_pos_ids_list], dim=1
+            ).reshape(3, -1)
+            position_ids[..., i, :] = llm_positions.to(position_ids.device)
+            mrope_position_deltas.append(
+                llm_positions.max() + 1 - len(current_input_ids)
+            )
+        mrope_position_deltas = torch.tensor(
+            mrope_position_deltas, device=input_ids.device
+        ).unsqueeze(1)
+        return position_ids, mrope_position_deltas
+    else:
+        s = input_ids.shape[1]
+        position_ids = torch.arange(s)
+        position_ids = position_ids.unsqueeze(0).expand(3, -1, -1).to(input_ids.device)
+        max_position_ids = position_ids.max(0, keepdim=False)[0].max(-1, keepdim=True)[
+            0
+        ]
+        mrope_position_deltas = max_position_ids + 1 - s
+        return position_ids, mrope_position_deltas
+
+
+def get_rope_index_glm4v(
+    input_ids: torch.Tensor,
+    hf_config: Any,
+    image_grid_thw: Union[List[List[int]], torch.Tensor],
+    video_grid_thw: Union[List[List[int]], torch.Tensor],
+    attention_mask: torch.Tensor,
+    **kwargs,
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    """Get mrope input positions and delta value for GLM4V."""
+    image_token_id = hf_config.image_token_id
+    video_start_token_id = hf_config.video_start_token_id
+    video_end_token_id = hf_config.video_end_token_id
+    spatial_merge_size = hf_config.vision_config.spatial_merge_size
+
+    mrope_position_deltas = []
+
+    if input_ids is not None and (
+        image_grid_thw is not None or video_grid_thw is not None
+    ):
+        total_input_ids = input_ids
+
+        if attention_mask is None:
+            attention_mask = torch.ones_like(total_input_ids)
+
+        position_ids = torch.ones(
+            3,
+            input_ids.shape[0],
+            input_ids.shape[1],
+            dtype=input_ids.dtype,
+            device=input_ids.device,
+        )
+
+        image_index, video_index = 0, 0
+        video_group_index = 0
+        attention_mask = attention_mask.to(total_input_ids.device)
+
+        for i, ids in enumerate(total_input_ids):
+            curr_mask = attention_mask[i]
+            ids_masked = ids[curr_mask == 1]
+
+            input_tokens = ids_masked.tolist()
+            input_token_type = [""] * len(input_tokens)
+
+            video_check_flg = False
+            for j, token in enumerate(input_tokens):
+                if token == video_start_token_id:
+                    video_check_flg = True
+                elif token == video_end_token_id:
+                    video_check_flg = False
+
+                if token == image_token_id and not video_check_flg:
+                    input_token_type[j] = "image"
+                elif token == image_token_id and video_check_flg:
+                    input_token_type[j] = "video"
+                else:
+                    input_token_type[j] = "text"
+
+            input_type_group = []
+            for key, group in itertools.groupby(
+                enumerate(input_token_type), lambda x: x[1]
+            ):
+                group = list(group)
+                start_index = group[0][0]
+                end_index = group[-1][0] + 1
+                input_type_group.append((key, start_index, end_index))
+
+            llm_pos_ids_list = []
+            video_frame_num = 1
+
+            for modality_type, start_idx, end_idx in input_type_group:
+                if llm_pos_ids_list:
+                    st_idx = llm_pos_ids_list[-1].max().item() + 1
+                else:
+                    st_idx = 0
+
+                if modality_type == "image":
+                    t, h, w = (
+                        image_grid_thw[image_index][0],
+                        image_grid_thw[image_index][1],
+                        image_grid_thw[image_index][2],
+                    )
+                    t_int, h_int, w_int = int(t), int(h), int(w)
+                    llm_grid_t = t_int
+                    llm_grid_h = h_int // spatial_merge_size
+                    llm_grid_w = w_int // spatial_merge_size
+
+                    t_index = (
+                        torch.arange(llm_grid_t, device=position_ids.device)
+                        .view(-1, 1)
+                        .expand(llm_grid_t, llm_grid_h * llm_grid_w)
+                        .reshape(-1)
+                    )
+                    h_index = (
+                        torch.arange(llm_grid_h, device=position_ids.device)
+                        .view(1, -1, 1)
+                        .expand(llm_grid_t, llm_grid_h, llm_grid_w)
+                        .reshape(-1)
+                    )
+                    w_index = (
+                        torch.arange(llm_grid_w, device=position_ids.device)
+                        .view(1, 1, -1)
+                        .expand(llm_grid_t, llm_grid_h, llm_grid_w)
+                        .reshape(-1)
+                    )
+                    llm_pos_ids_list.append(
+                        torch.stack([t_index, h_index, w_index]) + st_idx
+                    )
+                    image_index += 1
+                    video_frame_num = 1
+
+                elif modality_type == "video":
+                    t = video_frame_num
+                    h = video_grid_thw[video_index][1]
+                    w = video_grid_thw[video_index][2]
+                    h_int, w_int = int(h), int(w)
+                    llm_grid_h = h_int // spatial_merge_size
+                    llm_grid_w = w_int // spatial_merge_size
+
+                    for t_idx in range(t):
+                        t_index = (
+                            torch.tensor(t_idx, device=position_ids.device)
+                            .view(-1, 1)
+                            .expand(1, llm_grid_h * llm_grid_w)
+                            .reshape(-1)
+                        )
+                        h_index = (
+                            torch.arange(llm_grid_h, device=position_ids.device)
+                            .view(1, -1, 1)
+                            .expand(1, llm_grid_h, llm_grid_w)
+                            .reshape(-1)
+                        )
+                        w_index = (
+                            torch.arange(llm_grid_w, device=position_ids.device)
+                            .view(1, 1, -1)
+                            .expand(1, llm_grid_h, llm_grid_w)
+                            .reshape(-1)
+                        )
+                        llm_pos_ids_list.append(
+                            torch.stack([t_index, h_index, w_index]) + st_idx
+                        )
+
+                    video_group_index += 1
+                    if video_group_index >= video_grid_thw[video_index][0]:
+                        video_index += 1
+                        video_group_index = 0
+                    video_frame_num += 1
+
+                else:  # text
+                    text_len = end_idx - start_idx
+                    text_range = torch.arange(text_len, device=position_ids.device)
+                    text_pos = text_range.view(1, -1).expand(3, text_len) + st_idx
+                    llm_pos_ids_list.append(text_pos)
+                    video_frame_num = 1
+
+            llm_positions = torch.cat(llm_pos_ids_list, dim=1).reshape(3, -1)
+            idx_mask = curr_mask == 1
+            position_ids[..., i, idx_mask] = llm_positions.to(position_ids.device)
+            mrope_position_deltas.append(
+                llm_positions.max() + 1 - len(total_input_ids[i])
+            )
+        mrope_position_deltas = torch.tensor(
+            mrope_position_deltas, device=input_ids.device
+        ).unsqueeze(1)
+        return position_ids, mrope_position_deltas
+    else:
+        if attention_mask is not None:
+            position_ids = attention_mask.long().cumsum(-1) - 1
+            position_ids.masked_fill_(attention_mask == 0, 1)
+            position_ids = (
+                position_ids.unsqueeze(0).expand(3, -1, -1).to(attention_mask.device)
+            )
+            max_position_ids = position_ids.amax(dim=0, keepdim=False)
+            mrope_position_deltas = (
+                max_position_ids.amax(-1, keepdim=True) + 1 - attention_mask.shape[-1]
+            )
+        else:
+            length = input_ids.shape[1]
+            batch_size = input_ids.shape[0]
+            arange_ids = torch.arange(length, device=input_ids.device).view(1, 1, -1)
+            position_ids = arange_ids.expand(3, batch_size, length)
+            mrope_position_deltas = torch.zeros(
+                [batch_size, 1],
+                device=input_ids.device,
+                dtype=input_ids.dtype,
+            )
+        return position_ids, mrope_position_deltas
+
+
+def get_rope_index_ernie45(
+    input_ids: torch.Tensor,
+    hf_config: Any,
+    image_grid_thw: Union[List[List[int]], torch.Tensor],
+    video_grid_thw: Union[List[List[int]], torch.Tensor],
+    **kwargs,
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    """Get mrope input positions and delta value for Ernie VL."""
+    image_token_id = hf_config.im_patch_id
+    video_start_token_id = hf_config.video_start_token_id
+    video_end_token_id = hf_config.video_end_token_id
+    spatial_conv_size = hf_config.spatial_conv_size
+    temporal_conv_size = hf_config.temporal_conv_size
+
+    mrope_position_deltas = []
+    if input_ids is not None and (
+        image_grid_thw is not None or video_grid_thw is not None
+    ):
+        total_input_ids = input_ids
+        position_ids = torch.ones(
+            3,
+            input_ids.shape[0],
+            input_ids.shape[1],
+            dtype=input_ids.dtype,
+            device=input_ids.device,
+        )
+        image_index, video_index = 0, 0
+        for i, input_ids in enumerate(total_input_ids):
+            input_tokens = input_ids.tolist()
+
+            input_token_type = []
+            video_check_flg = False
+            for token in input_tokens:
+                if token == video_start_token_id:
+                    video_check_flg = True
+                elif token == video_end_token_id:
+                    video_check_flg = False
+
+                if token == image_token_id and not video_check_flg:
+                    input_token_type.append("image")
+                elif token == image_token_id and video_check_flg:
+                    input_token_type.append("video")
+                else:
+                    input_token_type.append("text")
+
+            input_type_group = []
+            for key, group in itertools.groupby(
+                enumerate(input_token_type), lambda x: x[1]
+            ):
+                group = list(group)
+                start_index = group[0][0]
+                end_index = group[-1][0] + 1
+                input_type_group.append((key, start_index, end_index))
+
+            llm_pos_ids_list = []
+            video_frame_num = 1
+            for modality_type, start_idx, end_idx in input_type_group:
+                st_idx = (
+                    llm_pos_ids_list[-1].max() + 1 if len(llm_pos_ids_list) > 0 else 0
+                )
+
+                if modality_type == "image":
+                    t, h, w = (
+                        image_grid_thw[image_index][0],
+                        image_grid_thw[image_index][1],
+                        image_grid_thw[image_index][2],
+                    )
+                    llm_grid_t, llm_grid_h, llm_grid_w = (
+                        t.item(),
+                        h.item() // spatial_conv_size,
+                        w.item() // spatial_conv_size,
+                    )
+
+                    t_index = (
+                        torch.arange(llm_grid_t)
+                        .view(-1, 1)
+                        .expand(-1, llm_grid_h * llm_grid_w)
+                        .flatten()
+                    )
+                    h_index = (
+                        torch.arange(llm_grid_h)
+                        .view(1, -1, 1)
+                        .expand(llm_grid_t, -1, llm_grid_w)
+                        .flatten()
+                    )
+                    w_index = (
+                        torch.arange(llm_grid_w)
+                        .view(1, 1, -1)
+                        .expand(llm_grid_t, llm_grid_h, -1)
+                        .flatten()
+                    )
+                    llm_pos_ids_list.append(
+                        torch.stack([t_index, h_index, w_index]) + st_idx
+                    )
+                    image_index += 1
+                    video_frame_num = 1
+
+                elif modality_type == "video":
+                    t, h, w = (
+                        video_grid_thw[video_index][0],
+                        video_grid_thw[video_index][1],
+                        video_grid_thw[video_index][2],
+                    )
+                    llm_grid_t, llm_grid_h, llm_grid_w = (
+                        t.item() // temporal_conv_size,
+                        h.item() // spatial_conv_size,
+                        w.item() // spatial_conv_size,
+                    )
+
+                    for t_idx in range(llm_grid_t):
+                        t_index = (
+                            torch.tensor(t_idx)
+                            .view(-1, 1)
+                            .expand(-1, llm_grid_h * llm_grid_w)
+                            .flatten()
+                        )
+                        h_index = (
+                            torch.arange(llm_grid_h)
+                            .view(1, -1, 1)
+                            .expand(1, -1, llm_grid_w)
+                            .flatten()
+                        )
+                        w_index = (
+                            torch.arange(llm_grid_w)
+                            .view(1, 1, -1)
+                            .expand(1, llm_grid_h, -1)
+                            .flatten()
+                        )
+                        llm_pos_ids_list.append(
+                            torch.stack([t_index, h_index, w_index]) + st_idx
+                        )
+                    video_index += 1
+                    video_frame_num += 1
+
+                else:
+                    text_len = end_idx - start_idx
+                    llm_pos_ids_list.append(
+                        torch.arange(text_len).view(1, -1).expand(3, -1) + st_idx
+                    )
+                    video_frame_num = 1
+
+            llm_positions = torch.cat(llm_pos_ids_list, dim=1).reshape(3, -1)
+            position_ids[..., i, :] = llm_positions.to(position_ids.device)
+            mrope_position_deltas.append(
+                llm_positions.max() + 1 - len(total_input_ids[i])
+            )
+        mrope_position_deltas = torch.tensor(
+            mrope_position_deltas, device=input_ids.device
+        ).unsqueeze(1)
+        return position_ids, mrope_position_deltas
+    else:
+        s = input_ids.shape[1]
+        position_ids = torch.arange(s)
+        position_ids = position_ids.unsqueeze(0).expand(3, -1, -1).to(input_ids.device)
+        max_position_ids = position_ids.max(0, keepdim=False)[0].max(-1, keepdim=True)[
+            0
+        ]
+        mrope_position_deltas = max_position_ids + 1 - s
+        return position_ids, mrope_position_deltas

runtime-v2/overlay/python/sglang/srt/layers/sensenova_mhc_fused.py

@@ -0,0 +1,462 @@
+"""Fused Triton kernels for the SenseNova Loop mHC router (streams == 2).
+
+The reference router (SenseNovaMHCRouter._forward_reference) turns its FP32
+logits [T, 8] into
+
+    pre  = sigmoid(l[:, 0:2])
+    post = 2 * sigmoid(l[:, 2:4])
+    mix  = exp(Sinkhorn^iterations(l[:, 4:8].view(T, 2, 2)))   (log space)
+
+with torch.logsumexp, i.e. ~300 tiny CUDA kernels per call. ``mhc_coeffs``
+does the same in one kernel, replicating the eager op order exactly:
+
+  * torch.logsumexp(x, dim): m = amax(x) (NaN-propagating), m := 0 where
+    |m| == inf, r = log(sum(exp(x - m))) + m.  A 2-element fp32 sum is a
+    single correctly rounded add, so its order cannot differ.
+  * torch.sigmoid (CUDA, fp32): 1 / (1 + exp(-x)) with IEEE division.
+  * exp / log / division use the correctly rounded libdevice routines that
+    nvcc uses for expf / logf / '/' in ATen (no ex2.approx / lg2.approx /
+    div.full), so results are bitwise identical to the eager path.
+
+The iteration count is a constexpr taken from the model config.
+"""
+
+import torch
+import triton
+import triton.language as tl
+from triton.language.extra import libdevice
+
+
+@triton.jit
+def _lse2(x, y):
+    # torch.logsumexp over two elements.
+    m = tl.maximum(x, y, propagate_nan=tl.PropagateNan.ALL)
+    m = tl.where(tl.abs(m) == float("inf"), 0.0, m)
+    s = libdevice.exp(x - m) + libdevice.exp(y - m)
+    return libdevice.log(s) + m
+
+
+@triton.jit
+def _sigmoid(x):
+    # ATen sigmoid_kernel_cuda: one / (one + std::exp(-a)) in opmath (fp32).
+    return libdevice.div_rn(1.0, 1.0 + libdevice.exp(-x))
+
+
+@triton.jit
+def _mhc_coeffs_kernel(
+    logits_ptr,
+    pre_ptr,
+    post_ptr,
+    mix_ptr,
+    T,
+    stride_t,
+    stride_k,
+    ITERS: tl.constexpr,
+    BLOCK_T: tl.constexpr,
+):
+    pid = tl.program_id(0)
+    rows = pid * BLOCK_T + tl.arange(0, BLOCK_T)
+    mask = rows < T
+    base = logits_ptr + rows.to(tl.int64) * stride_t
+    l0 = tl.load(base + 0 * stride_k, mask=mask, other=0.0)
+    l1 = tl.load(base + 1 * stride_k, mask=mask, other=0.0)
+    l2 = tl.load(base + 2 * stride_k, mask=mask, other=0.0)
+    l3 = tl.load(base + 3 * stride_k, mask=mask, other=0.0)
+    # mix[t] = [[a, b], [c, d]]; dim -1 = rows (a,b),(c,d); dim -2 = cols (a,c),(b,d)
+    a = tl.load(base + 4 * stride_k, mask=mask, other=0.0)
+    b = tl.load(base + 5 * stride_k, mask=mask, other=0.0)
+    c = tl.load(base + 6 * stride_k, mask=mask, other=0.0)
+    d = tl.load(base + 7 * stride_k, mask=mask, other=0.0)
+
+    for _ in tl.static_range(ITERS):
+        r0 = _lse2(a, b)
+        r1 = _lse2(c, d)
+        a = a - r0
+        b = b - r0
+        c = c - r1
+        d = d - r1
+        c0 = _lse2(a, c)
+        c1 = _lse2(b, d)
+        a = a - c0
+        c = c - c0
+        b = b - c1
+        d = d - c1
+
+    out2 = rows.to(tl.int64) * 2
+    tl.store(pre_ptr + out2, _sigmoid(l0), mask=mask)
+    tl.store(pre_ptr + out2 + 1, _sigmoid(l1), mask=mask)
+    tl.store(post_ptr + out2, 2.0 * _sigmoid(l2), mask=mask)
+    tl.store(post_ptr + out2 + 1, 2.0 * _sigmoid(l3), mask=mask)
+    out4 = rows.to(tl.int64) * 4
+    tl.store(mix_ptr + out4 + 0, libdevice.exp(a), mask=mask)
+    tl.store(mix_ptr + out4 + 1, libdevice.exp(b), mask=mask)
+    tl.store(mix_ptr + out4 + 2, libdevice.exp(c), mask=mask)
+    tl.store(mix_ptr + out4 + 3, libdevice.exp(d), mask=mask)
+
+
+def supported(streams: int, fixed_identity: bool) -> bool:
+    return streams == 2 and not fixed_identity
+
+
+def mhc_coeffs(logits: torch.Tensor, iterations: int):
+    """FP32 logits [..., 8] -> (pre [..., 2], post [..., 2], mix [..., 2, 2]),
+    all FP32 and contiguous, bitwise equal to the reference router."""
+    assert logits.dtype == torch.float32 and logits.shape[-1] == 8, logits.shape
+    lead = logits.shape[:-1]
+    flat = logits if logits.dim() == 2 else logits.reshape(-1, 8)
+    T = flat.shape[0]
+    pre = torch.empty((*lead, 2), device=logits.device, dtype=torch.float32)
+    post = torch.empty((*lead, 2), device=logits.device, dtype=torch.float32)
+    mix = torch.empty((*lead, 2, 2), device=logits.device, dtype=torch.float32)
+    if T == 0:
+        return pre, post, mix
+    BLOCK_T = 128 if T >= 128 else max(32, triton.next_power_of_2(T))
+    grid = (triton.cdiv(T, BLOCK_T),)
+    _mhc_coeffs_kernel[grid](
+        flat,
+        pre,
+        post,
+        mix,
+        T,
+        flat.stride(0),
+        flat.stride(1),
+        ITERS=int(iterations),
+        BLOCK_T=BLOCK_T,
+        num_warps=max(1, min(4, BLOCK_T // 32)),
+        # ATen keeps denormals; Triton's default links libdevice in FTZ mode,
+        # which would flush exp() / sigmoid() results below 2**-126 to zero.
+        enable_reflect_ftz=False,
+    )
+    return pre, post, mix
+
+
+# ---------------------------------------------------------------------------
+# Router tail + pre-branch in one kernel, and the merge in one kernel.
+#
+# bf16 bmm with K == 2 (cuBLAS, fp32 accumulate): both bf16 x bf16 products are
+# exact in fp32, so the result is round_bf16(round_fp32(p0 + p1)) whatever the
+# accumulation order or FMA use. Elementwise bf16 ops in ATen compute in fp32
+# and round once to bf16. The kernels below reproduce exactly that.
+# ---------------------------------------------------------------------------
+
+
+@triton.jit
+def _bf16(x):
+    # torch .to(torch.bfloat16) (round to nearest even), read back as fp32.
+    return x.to(tl.bfloat16).to(tl.float32)
+
+
+@triton.jit
+def _mhc_pre_kernel(
+    logits_ptr,
+    hs_ptr,
+    branch_ptr,
+    pre_ptr,
+    post_ptr,
+    mix_ptr,
+    H,
+    stride_lt,
+    stride_lk,
+    stride_ht,
+    stride_bt,
+    ITERS: tl.constexpr,
+    BLOCK_H: tl.constexpr,
+):
+    t = tl.program_id(0).to(tl.int64)
+    base = logits_ptr + t * stride_lt
+    l0 = tl.load(base + 0 * stride_lk)
+    l1 = tl.load(base + 1 * stride_lk)
+    l2 = tl.load(base + 2 * stride_lk)
+    l3 = tl.load(base + 3 * stride_lk)
+    a = tl.load(base + 4 * stride_lk)
+    b = tl.load(base + 5 * stride_lk)
+    c = tl.load(base + 6 * stride_lk)
+    d = tl.load(base + 7 * stride_lk)
+    for _ in tl.static_range(ITERS):
+        r0 = _lse2(a, b)
+        r1 = _lse2(c, d)
+        a = a - r0
+        b = b - r0
+        c = c - r1
+        d = d - r1
+        c0 = _lse2(a, c)
+        c1 = _lse2(b, d)
+        a = a - c0
+        c = c - c0
+        b = b - c1
+        d = d - c1
+    p0 = _sigmoid(l0)
+    p1 = _sigmoid(l1)
+    tl.store(pre_ptr + t * 2, p0)
+    tl.store(pre_ptr + t * 2 + 1, p1)
+    tl.store(post_ptr + t * 2, 2.0 * _sigmoid(l2))
+    tl.store(post_ptr + t * 2 + 1, 2.0 * _sigmoid(l3))
+    tl.store(mix_ptr + t * 4 + 0, libdevice.exp(a))
+    tl.store(mix_ptr + t * 4 + 1, libdevice.exp(b))
+    tl.store(mix_ptr + t * 4 + 2, libdevice.exp(c))
+    tl.store(mix_ptr + t * 4 + 3, libdevice.exp(d))
+
+    # branch = bmm(pre.to(bf16)[t, None, :], hs[t].view(2, H))
+    w0 = _bf16(p0)
+    w1 = _bf16(p1)
+    offs = tl.arange(0, BLOCK_H)
+    row = hs_ptr + t * stride_ht
+    out = branch_ptr + t * stride_bt
+    for h in range(0, H, BLOCK_H):
+        m = h + offs < H
+        x0 = tl.load(row + h + offs, mask=m, other=0.0).to(tl.float32)
+        x1 = tl.load(row + H + h + offs, mask=m, other=0.0).to(tl.float32)
+        tl.store(out + h + offs, (w0 * x0 + w1 * x1).to(tl.bfloat16), mask=m)
+
+
+@triton.jit
+def _mhc_merge_kernel(
+    branch_ptr,
+    res_ptr,
+    mix_ptr,
+    post_ptr,
+    out_ptr,
+    H,
+    stride_bt,
+    stride_rt,
+    stride_ot,
+    stride_mt,
+    stride_mi,
+    stride_mj,
+    stride_pt,
+    stride_pk,
+    BLOCK_H: tl.constexpr,
+):
+    t = tl.program_id(0).to(tl.int64)
+    hb = tl.program_id(1)
+    mb = mix_ptr + t * stride_mt
+    m00 = _bf16(tl.load(mb))
+    m01 = _bf16(tl.load(mb + stride_mj))
+    m10 = _bf16(tl.load(mb + stride_mi))
+    m11 = _bf16(tl.load(mb + stride_mi + stride_mj))
+    q0 = _bf16(tl.load(post_ptr + t * stride_pt))
+    q1 = _bf16(tl.load(post_ptr + t * stride_pt + stride_pk))
+    offs = hb * BLOCK_H + tl.arange(0, BLOCK_H)
+    m = offs < H
+    row = res_ptr + t * stride_rt
+    r0 = tl.load(row + offs, mask=m, other=0.0).to(tl.float32)
+    r1 = tl.load(row + H + offs, mask=m, other=0.0).to(tl.float32)
+    br = tl.load(branch_ptr + t * stride_bt + offs, mask=m, other=0.0).to(tl.float32)
+    # mixed = bmm(mix.bf16, residual); update = post.bf16[:, :, None] * branch[:, None]
+    o0 = _bf16(m00 * r0 + m01 * r1) + _bf16(q0 * br)
+    o1 = _bf16(m10 * r0 + m11 * r1) + _bf16(q1 * br)
+    out = out_ptr + t * stride_ot
+    tl.store(out + offs, o0.to(tl.bfloat16), mask=m)
+    tl.store(out + H + offs, o1.to(tl.bfloat16), mask=m)
+
+
+def mhc_pre(logits: torch.Tensor, hidden_states: torch.Tensor, iterations: int):
+    """Router tail for streams == 2: FP32 logits [T, 8] and BF16 hidden_states
+    [T, 2H] -> (branch [T, H] bf16, pre [T, 2], post [T, 2], mix [T, 2, 2])."""
+    assert logits.dim() == 2 and logits.shape[1] == 8 and logits.dtype == torch.float32
+    assert hidden_states.dim() == 2 and hidden_states.dtype == torch.bfloat16
+    T, H2 = hidden_states.shape
+    H = H2 // 2
+    if hidden_states.stride(1) != 1:
+        hidden_states = hidden_states.contiguous()
+    dev = logits.device
+    branch = torch.empty((T, H), device=dev, dtype=hidden_states.dtype)
+    pre = torch.empty((T, 2), device=dev, dtype=torch.float32)
+    post = torch.empty((T, 2), device=dev, dtype=torch.float32)
+    mix = torch.empty((T, 2, 2), device=dev, dtype=torch.float32)
+    if T == 0:
+        return branch, pre, post, mix
+    _mhc_pre_kernel[(T,)](
+        logits,
+        hidden_states,
+        branch,
+        pre,
+        post,
+        mix,
+        H,
+        logits.stride(0),
+        logits.stride(1),
+        hidden_states.stride(0),
+        branch.stride(0),
+        ITERS=int(iterations),
+        BLOCK_H=1024,
+        num_warps=4,
+        enable_reflect_ftz=False,
+    )
+    return branch, pre, post, mix
+
+
+def mhc_merge(branch, residual, mix, post):
+    """SenseNovaMHCRouter.merge for streams == 2 in one kernel (bitwise equal)."""
+    T, H2 = residual.shape
+    H = H2 // 2
+    if residual.stride(1) != 1:
+        residual = residual.contiguous()
+    if branch.stride(-1) != 1:
+        branch = branch.contiguous()
+    out = torch.empty((T, H2), device=residual.device, dtype=residual.dtype)
+    if T == 0:
+        return out
+    BLOCK_H = 1024
+    _mhc_merge_kernel[(T, triton.cdiv(H, BLOCK_H))](
+        branch,
+        residual,
+        mix,
+        post,
+        out,
+        H,
+        branch.stride(0),
+        residual.stride(0),
+        out.stride(0),
+        mix.stride(0),
+        mix.stride(1),
+        mix.stride(2),
+        post.stride(0),
+        post.stride(1),
+        BLOCK_H=BLOCK_H,
+        num_warps=4,
+        enable_reflect_ftz=False,
+        # Otherwise the bf16-rounded update product is contracted into an fma
+        # with the following add and its rounding is lost (not bitwise).
+        enable_fp_fusion=False,
+    )
+    return out
+
+
+# ---------------------------------------------------------------------------
+# Whole router forward in one kernel: FP32 RMS-normalize + 8 FP32 dot products
+# + bias + sigmoid / Sinkhorn + pre-branch. One program per token row.
+#
+# NOT bitwise equal to the eager path: the sum of squares and the 8192-long dot
+# products are FP32 but summed in a different order than torch.mean / cuBLAS
+# (which itself changes order with M). Everything after the logits is exact.
+# ---------------------------------------------------------------------------
+
+
+@triton.jit
+def _pick(v, idx, j: tl.constexpr):
+    return tl.sum(tl.where(idx == j, v, 0.0), axis=0)
+
+
+@triton.jit
+def _mhc_router_kernel(
+    hs_ptr,
+    w_ptr,
+    bias_ptr,
+    branch_ptr,
+    pre_ptr,
+    post_ptr,
+    mix_ptr,
+    stride_ht,
+    stride_bt,
+    H: tl.constexpr,
+    EPS: tl.constexpr,
+    ITERS: tl.constexpr,
+    BLOCK_K: tl.constexpr,
+):
+    K: tl.constexpr = 2 * H
+    t = tl.program_id(0).to(tl.int64)
+    row = hs_ptr + t * stride_ht
+    offs = tl.arange(0, BLOCK_K)
+
+    # mean(x^2): squares of bf16 values are exact in fp32.
+    sq = tl.zeros([BLOCK_K], dtype=tl.float32)
+    for k in range(0, K, BLOCK_K):
+        x = tl.load(row + k + offs).to(tl.float32)
+        sq += x * x
+    ms = libdevice.div_rn(tl.sum(sq, axis=0), K * 1.0)
+    r = libdevice.rsqrt(ms + EPS)
+
+    # logits[j] = sum_k (x_k * r) * W[j, k] + b[j]   (normalized rounded as in eager)
+    jj = tl.arange(0, 8)
+    acc = tl.zeros([8, BLOCK_K], dtype=tl.float32)
+    for k in range(0, K, BLOCK_K):
+        n = tl.load(row + k + offs).to(tl.float32) * r
+        w = tl.load(w_ptr + jj[:, None] * K + k + offs[None, :])
+        acc += n[None, :] * w
+    logits = tl.sum(acc, axis=1) + tl.load(bias_ptr + jj)
+    l0 = _pick(logits, jj, 0)
+    l1 = _pick(logits, jj, 1)
+    l2 = _pick(logits, jj, 2)
+    l3 = _pick(logits, jj, 3)
+    a = _pick(logits, jj, 4)
+    b = _pick(logits, jj, 5)
+    c = _pick(logits, jj, 6)
+    d = _pick(logits, jj, 7)
+
+    for _ in tl.static_range(ITERS):
+        r0 = _lse2(a, b)
+        r1 = _lse2(c, d)
+        a = a - r0
+        b = b - r0
+        c = c - r1
+        d = d - r1
+        c0 = _lse2(a, c)
+        c1 = _lse2(b, d)
+        a = a - c0
+        c = c - c0
+        b = b - c1
+        d = d - c1
+    p0 = _sigmoid(l0)
+    p1 = _sigmoid(l1)
+    tl.store(pre_ptr + t * 2, p0)
+    tl.store(pre_ptr + t * 2 + 1, p1)
+    tl.store(post_ptr + t * 2, 2.0 * _sigmoid(l2))
+    tl.store(post_ptr + t * 2 + 1, 2.0 * _sigmoid(l3))
+    tl.store(mix_ptr + t * 4 + 0, libdevice.exp(a))
+    tl.store(mix_ptr + t * 4 + 1, libdevice.exp(b))
+    tl.store(mix_ptr + t * 4 + 2, libdevice.exp(c))
+    tl.store(mix_ptr + t * 4 + 3, libdevice.exp(d))
+
+    w0 = _bf16(p0)
+    w1 = _bf16(p1)
+    out = branch_ptr + t * stride_bt
+    for h in range(0, H, BLOCK_K):
+        x0 = tl.load(row + h + offs).to(tl.float32)
+        x1 = tl.load(row + H + h + offs).to(tl.float32)
+        tl.store(out + h + offs, (w0 * x0 + w1 * x1).to(tl.bfloat16))
+
+
+def mhc_router(
+    hidden_states: torch.Tensor,
+    weight: torch.Tensor,
+    bias: torch.Tensor,
+    iterations: int,
+    eps: float = 1e-6,
+):
+    """Whole streams == 2 router forward: BF16 hidden_states [T, 2H] ->
+    (branch [T, H] bf16, pre [T, 2], post [T, 2], mix [T, 2, 2])."""
+    T, K = hidden_states.shape
+    H = K // 2
+    assert weight.shape == (8, K) and weight.dtype == torch.float32
+    assert hidden_states.dtype == torch.bfloat16
+    BLOCK_K = 1024
+    assert H % BLOCK_K == 0, H
+    if hidden_states.stride(1) != 1:
+        hidden_states = hidden_states.contiguous()
+    weight = weight.contiguous()
+    dev = hidden_states.device
+    branch = torch.empty((T, H), device=dev, dtype=hidden_states.dtype)
+    pre = torch.empty((T, 2), device=dev, dtype=torch.float32)
+    post = torch.empty((T, 2), device=dev, dtype=torch.float32)
+    mix = torch.empty((T, 2, 2), device=dev, dtype=torch.float32)
+    if T == 0:
+        return branch, pre, post, mix
+    _mhc_router_kernel[(T,)](
+        hidden_states,
+        weight,
+        bias,
+        branch,
+        pre,
+        post,
+        mix,
+        hidden_states.stride(0),
+        branch.stride(0),
+        H=H,
+        EPS=eps,
+        ITERS=int(iterations),
+        BLOCK_K=BLOCK_K,
+        num_warps=8,
+        enable_reflect_ftz=False,
+    )
+    return branch, pre, post, mix

runtime-v2/overlay/python/sglang/srt/models/sensenova_loop.py

@@ -0,0 +1,362 @@
+# Copyright 2026 SGLang Team
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+"""Shared-weight SenseNova Loop with independent attention state per visit."""
+
+import copy
+import os
+from collections.abc import Iterable
+
+import torch
+from torch import nn
+from torch.nn import functional as F
+
+from sglang.kernels.ops.layernorm.mhc import hc_expand
+from sglang.srt.configs.sensenova_loop import (
+    SenseNovaLoopConfig,
+    SenseNovaLoopTextConfig,
+)
+from sglang.srt.layers.layer_boundary import (
+    MHCState,
+    append_stages,
+    declare_attn,
+    declare_ffn,
+)
+from sglang.srt.layers.layer_boundary.residual import batch as residual_batch
+from sglang.srt.layers.layernorm import GemmaRMSNorm
+from sglang.srt.layers import sensenova_mhc_fused
+from sglang.srt.layers.quantization.base_config import QuantizationConfig
+from sglang.srt.model_executor.forward_batch_info import ForwardBatch, PPProxyTensors
+from sglang.srt.models import qwen3_5
+from sglang.srt.runtime_context import get_parallel, get_stream
+from sglang.srt.utils import add_prefix, make_pp_layers
+
+_MHC_FUSED = os.environ.get("SENSENOVA_MHC_FUSED", "1") != "0"
+_MHC_FUSED_LINEAR = os.environ.get("SENSENOVA_MHC_FUSED_LINEAR", "1") != "0"
+# The single-kernel router beat torch-norm + cuBLAS at every T measured (1..4096).
+_MHC_FUSED_LINEAR_MAX_T = int(
+    os.environ.get("SENSENOVA_MHC_FUSED_LINEAR_MAX_T", str(1 << 62))
+)
+# Fused merge rounds the K=2 bf16 bmm exactly once (IEEE RN); bitwise equal to
+# torch for T <= 32, ~1e-6 of elements differ by 1-2 bf16 ulp at T >= 64 where
+# cuBLAS picks tensor-core kernels with non-RN fp32 accumulation.
+_MHC_FUSED_MERGE = _MHC_FUSED and os.environ.get("SENSENOVA_MHC_FUSED_MERGE", "1") != "0"
+
+
+class SenseNovaMHCRouter(nn.Module):
+    """Canonical FP32 router with log-space Sinkhorn and BF16 residual math."""
+
+    def __init__(self, *, hidden_size: int, settings: dict) -> None:
+        super().__init__()
+        self.streams = settings["streams"]
+        self.iterations = settings["iterations"]
+        self.fixed_identity = settings["fixed_identity"]
+        self.pre_scale = settings.get("pre_scale", 1.0)
+        output_size = 2 * self.streams + (
+            0 if self.fixed_identity else self.streams * self.streams
+        )
+        self.weight = nn.Parameter(
+            torch.empty(output_size, self.streams * hidden_size, dtype=torch.float32)
+        )
+        self.bias = nn.Parameter(torch.empty(output_size, dtype=torch.float32))
+        # SENSENOVA_MHC_FUSED=1 (default): Triton router kernels; 0: eager reference.
+        self.use_fused = _MHC_FUSED and sensenova_mhc_fused.supported(
+            self.streams, self.fixed_identity
+        )
+        # SENSENOVA_MHC_FUSED_LINEAR=1 (default): for T <= ..._MAX_T the whole
+        # forward (FP32 RMS-normalize + FP32 linear + coefficients + branch) is
+        # one kernel; FP32 but not cuBLAS's summation order, so not bitwise.
+        self.use_fused_linear = self.use_fused and _MHC_FUSED_LINEAR
+        self.fused_linear_max_t = _MHC_FUSED_LINEAR_MAX_T
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        out_norm_weight: torch.Tensor | None = None,
+        out_norm_eps: float | None = None,
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, bool]:
+        if self.use_fused:
+            return self._forward_fused(hidden_states, out_norm_weight, out_norm_eps)
+        return self._forward_reference(hidden_states, out_norm_weight, out_norm_eps)
+
+    def _forward_fused(
+        self,
+        hidden_states: torch.Tensor,
+        out_norm_weight: torch.Tensor | None = None,
+        out_norm_eps: float | None = None,
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, bool]:
+        """Same math and dtype boundaries as _forward_reference."""
+        if self.use_fused_linear and hidden_states.shape[0] <= self.fused_linear_max_t:
+            branch, pre, post, mix = sensenova_mhc_fused.mhc_router(
+                hidden_states, self.weight, self.bias, self.iterations
+            )
+        else:
+            flat = hidden_states.float()
+            normalized = flat * torch.rsqrt(
+                flat.square().mean(-1, keepdim=True) + 1e-6
+            )
+            logits = F.linear(normalized, self.weight, self.bias)
+            # Bitwise equal to the eager sigmoid / Sinkhorn / exp / bf16 bmm.
+            branch, pre, post, mix = sensenova_mhc_fused.mhc_pre(
+                logits, hidden_states, self.iterations
+            )
+        if self.pre_scale != 1.0:
+            branch = branch * self.pre_scale
+        return branch, mix, post, False
+
+    def _forward_reference(
+        self,
+        hidden_states: torch.Tensor,
+        out_norm_weight: torch.Tensor | None = None,
+        out_norm_eps: float | None = None,
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, bool]:
+        """Original eager implementation (SENSENOVA_MHC_FUSED=0), kept verbatim."""
+        residual = hidden_states.unflatten(-1, (self.streams, -1))
+        flat = hidden_states.float()
+        normalized = flat * torch.rsqrt(flat.square().mean(-1, keepdim=True) + 1e-6)
+        logits = F.linear(normalized, self.weight, self.bias)
+        pre = logits[..., : self.streams].sigmoid()
+        post = 2 * logits[..., self.streams : 2 * self.streams].sigmoid()
+        if self.fixed_identity:
+            mix = torch.eye(self.streams, device=flat.device, dtype=torch.float32)
+            mix = mix.expand(*flat.shape[:-1], self.streams, self.streams)
+        else:
+            mix = logits[..., 2 * self.streams :].unflatten(
+                -1, (self.streams, self.streams)
+            )
+            for _ in range(self.iterations):
+                mix = mix - torch.logsumexp(mix, dim=-1, keepdim=True)
+                mix = mix - torch.logsumexp(mix, dim=-2, keepdim=True)
+            mix = mix.exp()
+        branch = torch.bmm(pre.to(residual.dtype).unsqueeze(-2), residual).squeeze(-2)
+        if self.pre_scale != 1.0:
+            branch = branch * self.pre_scale
+        # MHCState supplies the norm arguments; False leaves norm to the caller.
+        return branch, mix, post, False
+
+    @staticmethod
+    def merge(
+        branch: torch.Tensor,
+        residual: torch.Tensor,
+        mix: torch.Tensor,
+        post: torch.Tensor,
+    ) -> torch.Tensor:
+        if _MHC_FUSED_MERGE and post.shape[-1] == 2 and residual.dim() == 2:
+            # One kernel (see _MHC_FUSED_MERGE for exactness).
+            return sensenova_mhc_fused.mhc_merge(branch, residual, mix, post)
+        return SenseNovaMHCRouter._merge_reference(branch, residual, mix, post)
+
+    @staticmethod
+    def _merge_reference(
+        branch: torch.Tensor,
+        residual: torch.Tensor,
+        mix: torch.Tensor,
+        post: torch.Tensor,
+    ) -> torch.Tensor:
+        streams = post.shape[-1]
+        residual = residual.unflatten(-1, (streams, -1))
+        mixed = torch.bmm(mix.to(residual.dtype), residual)
+        update = post.to(branch.dtype).unsqueeze(-1) * branch.unsqueeze(-2)
+        return (mixed + update).flatten(-2)
+
+
+class SenseNovaLoopModel(qwen3_5.Qwen3_5ForCausalLM):
+    def __init__(
+        self,
+        config: SenseNovaLoopTextConfig,
+        quant_config: QuantizationConfig | None = None,
+        prefix: str = "",
+    ) -> None:
+        nn.Module.__init__(self)
+        self.config = config
+        self.hidden_size = config.hidden_size
+        self.pp_group = get_parallel().pp_group
+        self.embed_tokens = self._build_embed_tokens(config)
+        physical_layers = {}
+        alt_stream = get_stream("alt")
+
+        def get_layer(idx: int, prefix: str):
+            execution_layer_id = idx
+            physical_layer_id = config.execution_plan[execution_layer_id]
+            layer_type = config.layers_block_type[execution_layer_id]
+            is_linear_attention = layer_type == "linear_attention"
+            settings = config.mhc
+            is_mhc = (
+                settings is not None
+                and settings["start"] <= physical_layer_id < settings["end"]
+            )
+            if physical_layer_id not in physical_layers:
+                layer_class = self.decoder_layer_types[layer_type]
+                # NVFP4 patch: (1) pass quant_config so ModelOpt FP4 linears are
+                # built; (2) name modules by PHYSICAL layer id so the checkpoint's
+                # exclude_modules (physical names, e.g. layers.30.linear_attn.in_proj_a)
+                # match layers 28..31, which run at execution ids 52..55.
+                physical_prefix = f"{prefix.rsplit('.', 1)[0]}.{physical_layer_id}"
+                layer = layer_class(
+                    config=config,
+                    layer_id=execution_layer_id,
+                    quant_config=quant_config,
+                    prefix=add_prefix(
+                        "linear_attn" if is_linear_attention else "self_attn",
+                        physical_prefix,
+                    ),
+                    alt_stream=alt_stream,
+                    build_stages=False,
+                )
+                if is_mhc:
+                    layer.mhc_attn = SenseNovaMHCRouter(
+                        hidden_size=config.hidden_size, settings=settings
+                    )
+                    layer.mhc_ffn = SenseNovaMHCRouter(
+                        hidden_size=config.hidden_size, settings=settings
+                    )
+                physical_layers[physical_layer_id] = layer
+            else:
+                layer = self._new_visit(
+                    physical_layers[physical_layer_id],
+                    execution_layer_id=execution_layer_id,
+                    is_linear_attention=is_linear_attention,
+                )
+
+            residual = None
+            layer.starts_mhc = is_mhc and physical_layer_id == settings["start"]
+            if is_mhc:
+                residual = MHCState(
+                    hc_mult=config.mhc["streams"],
+                    hc_attn_pre=layer.mhc_attn,
+                    hc_ffn_pre=layer.mhc_ffn,
+                    hc_post=SenseNovaMHCRouter.merge,
+                    is_last_layer=physical_layer_id == config.mhc["end"] - 1,
+                ).residual_ops()
+            attn, ffn = self._stage_facts(config, execution_layer_id, residual=residual)
+            layer.attn_boundary, layer.ffn_boundary = append_stages(
+                (attn, layer.input_layernorm),
+                (ffn, layer.post_attention_layernorm),
+            )
+            return layer
+
+        self.layers, self._start_layer, self._end_layer = make_pp_layers(
+            config.num_hidden_layers,
+            get_layer,
+            prefix=f"{prefix}.layers",
+            stage_facts=lambda execution_layer_id: self._stage_facts(
+                config, execution_layer_id
+            ),
+        )
+        self.norm = GemmaRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.flashinfer_mnnvl_cutedsl_fusion = None
+        self.layers_to_capture = []
+        self._weight_layer_indices = {
+            physical_layer_id: config.execution_plan.index(physical_layer_id)
+            for physical_layer_id in range(config.physical_num_hidden_layers)
+        }
+
+    @staticmethod
+    def _stage_facts(
+        config: SenseNovaLoopTextConfig, execution_layer_id: int, *, residual=None
+    ):
+        physical_layer_id = config.execution_plan[execution_layer_id]
+        settings = config.mhc
+        if (
+            settings is None
+            or not settings["start"] <= physical_layer_id < settings["end"]
+        ):
+            layer_class = qwen3_5.Qwen3_5ForCausalLM.decoder_layer_types[
+                config.layers_block_type[execution_layer_id]
+            ]
+            return layer_class.stage_facts(config, execution_layer_id)
+        residual = MHCState.facts() if residual is None else residual
+        return (
+            declare_attn(read=residual.attn_readout, update=residual.attn_update),
+            declare_ffn(read=residual.ffn_readout, update=residual.ffn_update),
+        )
+
+    @staticmethod
+    def _new_visit(
+        physical_layer: nn.Module,
+        *,
+        execution_layer_id: int,
+        is_linear_attention: bool,
+    ) -> nn.Module:
+        # Visits share all projections, norms and router parameters. Only cache
+        # identities and the subsequently bound residual state belong to a visit.
+        layer = copy.copy(physical_layer)
+        layer._modules = layer._modules.copy()
+        if is_linear_attention:
+            layer.linear_attn = copy.copy(layer.linear_attn)
+            layer.linear_attn._modules = layer.linear_attn._modules.copy()
+            layer.linear_attn.layer_id = execution_layer_id
+            layer.linear_attn.attn = copy.copy(layer.linear_attn.attn)
+            layer.linear_attn.attn.layer_id = execution_layer_id
+        else:
+            layer.attn = copy.copy(layer.attn)
+            layer.attn.layer_id = execution_layer_id
+        layer.layer_id = execution_layer_id
+        return layer
+
+    @torch.no_grad()
+    def forward(
+        self,
+        input_ids: torch.Tensor | None,
+        positions: torch.Tensor,
+        forward_batch: ForwardBatch,
+        input_embeds: torch.Tensor | None = None,
+        pp_proxy_tensors: PPProxyTensors | None = None,
+        input_deepstack_embeds: torch.Tensor | None = None,
+    ) -> torch.Tensor:
+        hidden_states = (
+            self.embed_tokens(input_ids) if input_embeds is None else input_embeds
+        )
+        residual_batch.start(forward_batch)
+        for layer in self.layers:
+            if layer.starts_mhc:
+                hidden_states = residual_batch.fold(hidden_states, forward_batch)
+                hidden_states = residual_batch.set_written(
+                    hc_expand(hidden_states, layer.mhc_attn.streams), forward_batch
+                )
+            hidden_states = layer(
+                positions=positions,
+                hidden_states=hidden_states,
+                forward_batch=forward_batch,
+            )
+        return residual_batch.final_norm(
+            hidden_states, forward_batch, self.norm, skip_empty=True
+        )
+
+
+class SenseNovaLoopForConditionalGeneration(qwen3_5.Qwen3_5ForConditionalGeneration):
+    def __init__(
+        self,
+        config: SenseNovaLoopConfig,
+        quant_config: QuantizationConfig | None = None,
+        prefix: str = "",
+    ) -> None:
+        super().__init__(
+            config=config,
+            quant_config=quant_config,
+            prefix=prefix,
+            language_model_cls=SenseNovaLoopModel,
+        )
+
+    def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]) -> set[str]:
+        def execution_weights() -> Iterable[tuple[str, torch.Tensor]]:
+            prefix = "model.language_model.layers."
+            for name, weight in weights:
+                if name == "model.language_model.loop_gate.logits":
+                    continue
+                if name.startswith(prefix):
+                    physical_layer_id, suffix = name[len(prefix) :].split(".", 1)
+                    execution_layer_id = self.model._weight_layer_indices[
+                        int(physical_layer_id)
+                    ]
+                    name = f"{prefix}{execution_layer_id}.{suffix}"
+                yield name, weight
+
+        return super().load_weights(execution_weights())
+
+
+EntryClass = SenseNovaLoopForConditionalGeneration

runtime-v2/overlay/python/sglang/srt/multimodal/processors/qwen_vl.py

@@ -0,0 +1,1234 @@
+import math
+import os
+import re
+import time
+from copy import deepcopy
+from dataclasses import dataclass, replace
+from functools import lru_cache
+from typing import Any, List, Optional, Union
+
+import numpy as np
+import torch
+import torchvision
+from PIL import Image
+from torchvision.transforms import InterpolationMode
+from transformers import BaseImageProcessor
+
+from sglang.srt.environ import envs
+from sglang.srt.layers.rotary_embedding import MRotaryEmbedding
+from sglang.srt.managers.schedule_batch import (
+    Modality,
+    MultimodalDataItem,
+    MultimodalProcessorOutput,
+)
+from sglang.srt.models.cosmos3 import Cosmos3ForConditionalGeneration
+from sglang.srt.models.interns2_mobius import (
+    InternS2MobiusForConditionalGeneration,
+)
+from sglang.srt.models.interns2preview import InternS2PreviewForConditionalGeneration
+from sglang.srt.models.qwen2_5_vl import Qwen2_5_VLForConditionalGeneration
+from sglang.srt.models.qwen2_vl import Qwen2VLForConditionalGeneration
+from sglang.srt.models.qwen3_5 import (
+    Qwen3_5ForConditionalGeneration,
+    Qwen3_5MoeForConditionalGeneration,
+)
+from sglang.srt.models.qwen3_5_mtp import Qwen3_5ForCausalLMMTP
+from sglang.srt.models.qwen3_omni_moe import Qwen3OmniMoeForConditionalGeneration
+from sglang.srt.models.qwen3_vl import Qwen3VLForConditionalGeneration
+from sglang.srt.models.qwen3_vl_moe import Qwen3VLMoeForConditionalGeneration
+from sglang.srt.models.qwen4_exp import Qwen4ExpForConditionalGeneration
+from sglang.srt.models.sensenova_loop import SenseNovaLoopForConditionalGeneration
+from sglang.srt.multimodal.cache import resolve_multimodal_item_hash
+from sglang.srt.multimodal.media_artifacts.base import (
+    MediaArtifactCacheMixin,
+    MediaArtifactInput,
+)
+from sglang.srt.multimodal.processors.base_processor import (
+    BaseMultimodalProcessor as SGLangBaseProcessor,
+)
+from sglang.srt.multimodal.processors.base_processor import (
+    MultimodalSpecialTokens,
+)
+from sglang.srt.multimodal.transport.cuda_ipc import (
+    DEFER_CUDA_IPC_FEATURE_RECONSTRUCTION_KEY,
+)
+from sglang.srt.runtime_context import get_mm, get_parallel
+from sglang.srt.utils import cpu_has_amx_support, is_cpu
+from sglang.srt.utils.video_decoder import VideoDecoderWrapper
+from sglang.utils import logger
+
+IMAGE_FACTOR = 28
+MIN_PIXELS = 4 * 28 * 28
+MAX_PIXELS = envs.SGLANG_IMAGE_MAX_PIXELS.get()
+MAX_RATIO = 200
+RESIZE_RESAMPLE = getattr(Image, envs.SGLANG_RESIZE_RESAMPLE.get(), None)
+if envs.SGLANG_RESIZE_RESAMPLE.is_set() and RESIZE_RESAMPLE is None:
+    logger.warning(
+        f"Invalid RESIZE_RESAMPLE value: '{envs.SGLANG_RESIZE_RESAMPLE.get()}'. "
+        f"Ignoring and using default."
+    )
+VIDEO_TOTAL_PIXELS = int(
+    float(os.environ.get("VIDEO_MAX_PIXELS", 128000 * 28 * 28 * 0.9))
+)
+
+VIDEO_MIN_PIXELS = 128 * 28 * 28
+VIDEO_MAX_PIXELS = 768 * 28 * 28
+FRAME_FACTOR = 2
+FPS = 2.0
+FPS_MIN_FRAMES = 4
+FPS_MAX_FRAMES = 768
+
+
+@dataclass(frozen=True)
+class QwenVLImagePreprocessArtifact:
+    """Prompt-independent Qwen-VL processor output for one image."""
+
+    content_digest: str
+    artifact_key: str
+    feature_hash: int
+    feature: Optional[torch.Tensor]
+    model_specific_data: dict[str, Any]
+
+    @property
+    def has_feature(self) -> bool:
+        return self.feature is not None
+
+    def cache_value(self) -> "QwenVLImagePreprocessArtifact":
+        """Keep CPU processor outputs but never retain a CUDA tensor."""
+        if self.feature is None or self.feature.device.type == "cpu":
+            return self
+        return replace(self, feature=None)
+
+    def cache_size_items(self) -> tuple:
+        return (
+            self.content_digest,
+            self.artifact_key,
+            self.feature_hash,
+            self.feature,
+            self.model_specific_data,
+        )
+
+
+QWEN_VIDEO_PREPROCESS_CONFIG_KEYS = frozenset(
+    {
+        "fps",
+        "nframes",
+        "min_frames",
+        "max_frames",
+        "min_pixels",
+        "max_pixels",
+        "total_pixels",
+        "resized_height",
+        "resized_width",
+    }
+)
+
+
+def _get_processor_video_config(video_config, video_metadata):
+    if video_metadata and all(metadata is not None for metadata in video_metadata):
+        return {
+            key: value
+            for key, value in video_config.items()
+            if key not in QWEN_VIDEO_PREPROCESS_CONFIG_KEYS
+        }
+    return None
+
+
+_is_cpu_amx_available = cpu_has_amx_support()
+_is_cpu = is_cpu()
+if _is_cpu and _is_cpu_amx_available:
+    try:
+        import transformers
+
+        from sglang.srt.layers.amx_utils import fast_preprocess_cpu
+
+        transformers.models.qwen2_vl.image_processing_qwen2_vl_fast.Qwen2VLImageProcessorFast._preprocess = fast_preprocess_cpu
+    except Exception as e:
+        logger.warning(
+            f"Failed to hack Qwen2VLImageProcessorFast with AMX optimization: {e}"
+        )
+
+
+def smart_resize(
+    height: int,
+    width: int,
+    factor: int = IMAGE_FACTOR,
+    min_pixels: int = MIN_PIXELS,
+    max_pixels: int = MAX_PIXELS,
+) -> tuple[int, int]:
+    """
+    Rescales the image so that the following conditions are met:
+
+    1. Both dimensions (height and width) are divisible by 'factor'.
+
+    2. The total number of pixels is within the range ['min_pixels', 'max_pixels'].
+
+    3. The aspect ratio of the image is maintained as closely as possible.
+    """
+    if max(height, width) / min(height, width) > MAX_RATIO:
+        raise ValueError(
+            f"absolute aspect ratio must be smaller than {MAX_RATIO}, got {max(height, width) / min(height, width)}"
+        )
+    h_bar = max(factor, round_by_factor(height, factor))
+    w_bar = max(factor, round_by_factor(width, factor))
+    if h_bar * w_bar > max_pixels:
+        beta = math.sqrt((height * width) / max_pixels)
+        h_bar = floor_by_factor(height / beta, factor)
+        w_bar = floor_by_factor(width / beta, factor)
+    elif h_bar * w_bar < min_pixels:
+        beta = math.sqrt(min_pixels / (height * width))
+        h_bar = ceil_by_factor(height * beta, factor)
+        w_bar = ceil_by_factor(width * beta, factor)
+    return h_bar, w_bar
+
+
+def round_by_factor(number: int, factor: int) -> int:
+    """Returns the closest integer to 'number' that is divisible by 'factor'."""
+    return round(number / factor) * factor
+
+
+def ceil_by_factor(number: int, factor: int) -> int:
+    """Returns the smallest integer greater than or equal to 'number' that is divisible by 'factor'."""
+    return math.ceil(number / factor) * factor
+
+
+def floor_by_factor(number: int, factor: int) -> int:
+    """Returns the largest integer less than or equal to 'number' that is divisible by 'factor'."""
+    return math.floor(number / factor) * factor
+
+
+def smart_nframes(
+    ele: dict,
+    total_frames: int,
+    video_fps: int | float,
+) -> int:
+    """calculate the number of frames for video used for model inputs.
+
+    Args:
+        ele (dict): a dict contains the configuration of video.
+            support either `fps` or `nframes`:
+                - nframes: the number of frames to extract for model inputs.
+                - fps: the fps to extract frames for model inputs.
+                    - min_frames: the minimum number of frames of the video, only used when fps is provided.
+                    - max_frames: the maximum number of frames of the video, only used when fps is provided.
+        total_frames (int): the original total number of frames of the video.
+        video_fps (int | float): the original fps of the video.
+
+    Raises:
+        ValueError: nframes should in interval [FRAME_FACTOR, total_frames].
+
+    Returns:
+        int: the number of frames for video used for model inputs.
+    """
+    assert not ("fps" in ele and "nframes" in ele), (
+        "Only accept either `fps` or `nframes`"
+    )
+    if "nframes" in ele:
+        nframes = round_by_factor(ele["nframes"], FRAME_FACTOR)
+    else:
+        fps = ele.get("fps", FPS)
+        min_frames = ceil_by_factor(ele.get("min_frames", FPS_MIN_FRAMES), FRAME_FACTOR)
+        max_frames = floor_by_factor(
+            ele.get("max_frames", min(FPS_MAX_FRAMES, total_frames)), FRAME_FACTOR
+        )
+        nframes = total_frames / video_fps * fps
+        if nframes > total_frames:
+            logger.warning(
+                f"smart_nframes: nframes[{nframes}] > total_frames[{total_frames}]"
+            )
+        nframes = min(min(max(nframes, min_frames), max_frames), total_frames)
+        nframes = floor_by_factor(nframes, FRAME_FACTOR)
+    if not (FRAME_FACTOR <= nframes and nframes <= total_frames):
+        raise ValueError(
+            f"nframes should in interval [{FRAME_FACTOR}, {total_frames}], but got {nframes}."
+        )
+    return nframes
+
+
+# process video, qwen-specific
+async def preprocess_video(
+    vr,
+    image_factor: int = IMAGE_FACTOR,
+    video_config: dict = {},
+) -> torch.Tensor:
+    # preprocessed video
+    is_video_obj = isinstance(vr, VideoDecoderWrapper)
+    if not is_video_obj:
+        return vr, None
+    entry_time = time.perf_counter()
+
+    total_frames, video_fps = len(vr), vr.avg_fps
+
+    nframes = smart_nframes(
+        video_config, total_frames=total_frames, video_fps=video_fps
+    )
+    idx = np.linspace(0, total_frames - 1, num=nframes, dtype=np.int64)
+    idx = np.unique(idx)
+
+    video = vr.get_frames_as_tensor(idx.tolist())
+
+    video = video.permute(0, 3, 1, 2)  # NHWC -> TCHW
+
+    nframes, _, height, width = video.shape
+    min_pixels = video_config.get("min_pixels", VIDEO_MIN_PIXELS)
+    total_pixels = video_config.get("total_pixels", VIDEO_TOTAL_PIXELS)
+    max_pixels = max(
+        min(
+            video_config.get("max_pixels", VIDEO_MAX_PIXELS),
+            total_pixels / nframes * FRAME_FACTOR,
+        ),
+        int(min_pixels * 1.05),
+    )
+
+    get_batch_time = time.perf_counter()
+
+    max_pixels_supposed = video_config.get("max_pixels", max_pixels)
+
+    if max_pixels_supposed > max_pixels:
+        logger.warning(
+            f"The given max_pixels[{max_pixels_supposed}] exceeds limit[{max_pixels}]."
+        )
+    max_pixels = min(max_pixels_supposed, max_pixels)
+    if "resized_height" in video_config and "resized_width" in video_config:
+        resized_height, resized_width = smart_resize(
+            video_config["resized_height"],
+            video_config["resized_width"],
+            factor=image_factor,
+        )
+    else:
+        resized_height, resized_width = smart_resize(
+            height,
+            width,
+            factor=image_factor,
+            min_pixels=min_pixels,
+            max_pixels=max_pixels,
+        )
+    smart_resize_time = time.perf_counter()
+    video = torchvision.transforms.functional.resize(
+        video,
+        [resized_height, resized_width],
+        interpolation=InterpolationMode.BILINEAR,
+    )
+    if not is_cpu() and video.device.type == "cpu":
+        video = video.pin_memory()
+    video_metadata = {
+        "fps": video_fps,
+        "duration": total_frames / video_fps,
+        "total_num_frames": total_frames,
+        "frames_indices": idx,
+        "video_backend": "torchvision",
+    }
+    torchvision_resize_time = time.perf_counter()
+    logger.debug(
+        f"[preprocess_video Perf], "
+        f"get_batch_time: {(get_batch_time - entry_time) * 1000:.2f} ms, "
+        f"smart_resize_time: {(smart_resize_time - get_batch_time) * 1000:.2f} ms, "
+        f"torchvision_resize_time: {(torchvision_resize_time - smart_resize_time) * 1000:.2f} ms, "
+        f"total_time: {(torchvision_resize_time - entry_time) * 1000:.2f} ms"
+    )
+    return video, video_metadata
+
+
+# Compatible with Qwen-VL & Qwen-Omni Series
+class QwenVLImageProcessor(MediaArtifactCacheMixin, SGLangBaseProcessor):
+    supports_transformers_backend = True
+    generates_input_ids_from_raw_prompt = True
+    artifact_modality = Modality.IMAGE
+    models = [
+        Qwen2VLForConditionalGeneration,
+        Qwen2_5_VLForConditionalGeneration,
+        Qwen3VLForConditionalGeneration,
+        Qwen3VLMoeForConditionalGeneration,
+        Qwen3_5ForConditionalGeneration,
+        Qwen3_5MoeForConditionalGeneration,
+        SenseNovaLoopForConditionalGeneration,
+        Qwen3_5ForCausalLMMTP,
+        InternS2PreviewForConditionalGeneration,
+        InternS2MobiusForConditionalGeneration,
+        Qwen3OmniMoeForConditionalGeneration,
+        Cosmos3ForConditionalGeneration,
+        Qwen4ExpForConditionalGeneration,
+    ]
+
+    def __init__(self, hf_config, server_args, _processor, *args, **kwargs):
+        self.model_type = hf_config.model_type
+        self.uses_media_artifacts_without_cache = self.model_type in (
+            "qwen3_vl",
+            "qwen3_vl_moe",
+        )
+        if self.model_type in (
+            "qwen2_vl",
+            "qwen2_5_vl",
+            "qwen3_vl",
+            "qwen3_vl_moe",
+            "qwen3_5",
+            "qwen3_5_moe",
+            "sensenova_loop",
+            "qwen4_exp",
+            "intern_s2_preview",
+            "interns2_mobius",
+        ):
+            # Two workers overlap CPU preprocessing without over-fragmenting
+            # burst arrivals into smaller GPU prefill batches. Higher counts can
+            # improve short-output TTFT, but regress long-output throughput on
+            # Blackwell when requests reach the scheduler too far apart.
+            self.auto_mm_processor_worker_num = 2
+            self.auto_mm_io_worker_num = 16
+            self.supports_mm_processor_concurrency = True
+        if hf_config.model_type == "qwen3_omni_moe":
+            hf_config = hf_config.thinker_config
+
+        super().__init__(hf_config, server_args, _processor, *args, **kwargs)
+
+        self.IM_START_TOKEN_ID = hf_config.vision_start_token_id
+        self.IM_END_TOKEN_ID = hf_config.vision_end_token_id
+        self.IM_TOKEN_ID = hf_config.image_token_id
+        self.VIDEO_TOKEN_ID = hf_config.video_token_id
+
+        self.vision_start_token_id = hf_config.vision_start_token_id
+        self.vision_end_token_id = getattr(hf_config, "vision_end_token_id", None)
+
+        self.audio_start_token_id = getattr(hf_config, "audio_start_token_id", None)
+        self.audio_token_id = getattr(hf_config, "audio_token_id", None)
+
+        self._spatial_merge_size = self.hf_config.vision_config.spatial_merge_size
+        self._tokens_per_second = getattr(
+            self.hf_config.vision_config, "tokens_per_second", None
+        )
+
+        # Also match the legacy sglang <image> sentinel used by /generate,
+        # so the artifact fast path can normalize it before build_input_ids.
+        self.mm_tokens = MultimodalSpecialTokens(
+            image_token="<|vision_start|><|image_pad|><|vision_end|>",
+            image_token_id=hf_config.image_token_id,
+            image_token_regex=re.compile(
+                r"<\|vision_start\|>(?:<\|image_pad\|>)+<\|vision_end\|>|<image>"
+            ),
+            video_token_id=self.VIDEO_TOKEN_ID,
+            audio_token_id=self.audio_token_id,
+        ).build(_processor)
+
+    @property
+    def spatial_merge_size(self):
+        return self._spatial_merge_size
+
+    def build_input_ids_with_timestamps(
+        self, prompt, embeddings, img_grid_thw, video_grid_thw, video_timestamps
+    ):
+        """
+        Build input_ids with timestamps for qwen3_vl models.
+        """
+        if not isinstance(prompt, list):
+            prompt = self._processor.tokenizer.encode(prompt)
+
+        img_token_id = getattr(self, "IM_TOKEN_ID", None)
+        video_token_id = getattr(self, "VIDEO_TOKEN_ID", None)
+        spatial_merge_size = self.spatial_merge_size
+        vision_start_token_id = getattr(self, "vision_start_token_id", None)
+        vision_end_token_id = getattr(self, "vision_end_token_id", None)
+
+        input_ids = []
+        offsets = []
+        modality_list = []
+        cur_idx = 0
+
+        vision_start_indices = []
+        for i in range(len(prompt) - 1):
+            if img_token_id is not None and prompt[i + 1] == img_token_id:
+                vision_start_indices.append((i, Modality.IMAGE))
+            elif video_token_id is not None and prompt[i + 1] == video_token_id:
+                vision_start_indices.append((i, Modality.VIDEO))
+
+        img_idx = 0
+        video_idx = 0
+        for mm_start_idx, modality in vision_start_indices:
+            modality_list.append(modality)
+            video_tokens = None
+            if modality == Modality.IMAGE:
+                mm_token_num = img_grid_thw[img_idx].prod() // (spatial_merge_size**2)
+                mm_token_id = img_token_id
+                img_idx += 1
+            elif modality == Modality.VIDEO:
+                curr_timestamps = video_timestamps[video_idx]
+                num_frames = video_grid_thw[video_idx][0]
+                frame_seqlen = video_grid_thw[video_idx][1:].prod().item() // (
+                    spatial_merge_size**2
+                )
+                video_tokens = []
+                _current_offset = len(input_ids) + mm_start_idx + 1 - cur_idx
+                # take single frame as one mm_item
+                for frame_idx in range(num_frames):
+                    if frame_idx > 0:
+                        modality_list.append(Modality.VIDEO)
+                    curr_time = curr_timestamps[frame_idx]
+                    timestamp_text = f"<{curr_time:.1f} seconds>"
+                    timestamp_tokens = self._processor.tokenizer.encode(
+                        timestamp_text, add_special_tokens=False
+                    )
+                    video_tokens.extend(timestamp_tokens)
+                    _current_offset += len(timestamp_tokens)
+                    if vision_start_token_id is not None:
+                        video_tokens.append(vision_start_token_id)
+                        _current_offset += 1
+                    video_tokens.extend([video_token_id] * frame_seqlen)
+                    if vision_end_token_id is not None:
+                        video_tokens.append(vision_end_token_id)
+                    offsets.append(
+                        (_current_offset, _current_offset + frame_seqlen - 1)
+                    )
+                    _current_offset += (
+                        frame_seqlen + 1
+                        if vision_end_token_id is not None
+                        else frame_seqlen
+                    )  # for vision_end_token_id
+                mm_token_num = len(video_tokens)
+                mm_token_id = None
+                video_idx += 1
+            else:
+                logger.warning(
+                    f"{modality} modality is not supported for qwen3_vl models with timestamps."
+                )
+                continue
+            assert cur_idx <= mm_start_idx
+            input_ids.extend(prompt[cur_idx : mm_start_idx + 1])
+            if modality == Modality.VIDEO:
+                input_ids.extend(video_tokens)
+            else:
+                mm_offset_start = len(input_ids)
+                input_ids.extend([mm_token_id] * mm_token_num)
+                offsets.append((mm_offset_start, len(input_ids) - 1))
+            cur_idx = mm_start_idx + 2  # jump to vision_end_id
+        else:
+            input_ids.extend(prompt[cur_idx:])
+
+        return input_ids, offsets, modality_list
+
+    def compute_mrope_positions(self, input_ids, mm_items):
+        image_grid_thw = self._concat_mm_item_grid(
+            mm_items, "image_grid_thw", Modality.IMAGE
+        )
+        video_grid_thw = self._concat_mm_item_grid(
+            mm_items, "video_grid_thw", Modality.VIDEO
+        )
+
+        input_ids_tensor = torch.tensor(input_ids, dtype=torch.long).unsqueeze(0)
+        mrope_positions, mrope_position_delta = MRotaryEmbedding.get_rope_index(
+            spatial_merge_size=self._spatial_merge_size,
+            image_token_id=self.mm_tokens.image_token_id,
+            video_token_id=self.mm_tokens.video_token_id,
+            vision_start_token_id=self.vision_start_token_id,
+            model_type=self.model_type,
+            tokens_per_second=self._tokens_per_second,
+            input_ids=input_ids_tensor,
+            image_grid_thw=image_grid_thw,
+            video_grid_thw=video_grid_thw,
+        )
+        return mrope_positions.squeeze(1), mrope_position_delta
+
+    @staticmethod
+    def _get_processor_output_value(ret, key):
+        if ret is None:
+            return None
+        return ret.get(key) if hasattr(ret, "get") else getattr(ret, key, None)
+
+    def _get_precomputed_mrope_from_output(self, ret):
+        mrope_positions = self._get_processor_output_value(ret, "mrope_positions")
+        mrope_position_delta = self._get_processor_output_value(
+            ret, "mrope_position_delta"
+        )
+        if mrope_positions is None or mrope_position_delta is None:
+            return None
+
+        mrope_positions = torch.as_tensor(mrope_positions)
+        if mrope_positions.ndim == 3:
+            if mrope_positions.shape[1] != 1:
+                return None
+            mrope_positions = mrope_positions.squeeze(1)
+        if mrope_positions.ndim != 2 or mrope_positions.shape[0] != 3:
+            return None
+
+        mrope_position_delta = torch.as_tensor(mrope_position_delta)
+        if mrope_position_delta.ndim <= 1:
+            mrope_position_delta = mrope_position_delta.reshape(-1, 1)
+        return mrope_positions, mrope_position_delta
+
+    @staticmethod
+    def _as_grid_batch(value):
+        if value is None:
+            return None
+        if isinstance(value, torch.Tensor):
+            return value.unsqueeze(0) if value.ndim == 1 else value
+        tensor = torch.as_tensor(value, dtype=torch.long)
+        return tensor.unsqueeze(0) if tensor.ndim == 1 else tensor
+
+    def _compute_image_only_mrope_positions_from_offsets(
+        self,
+        input_len: int,
+        mm_items: List[MultimodalDataItem],
+        dtype: torch.dtype,
+        device: torch.device,
+    ) -> Optional[tuple[torch.Tensor, torch.Tensor]]:
+        """instead of calling get_rope_index, build mrope position from mm_items.offsets and image_grid_thw of each image
+        basically a simplified version of get_rope_index for image-only reqs
+        """
+        if self.model_type not in (
+            "qwen3_vl",
+            "qwen3_vl_moe",
+            "qwen3_5",
+            "qwen3_5_moe",
+            "sensenova_loop",
+            "qwen4_exp",
+            "intern_s2_preview",
+            "interns2_mobius",
+            "cosmos3_omni",
+        ):
+            return None
+
+        image_items = [item for item in mm_items if item.is_image()]
+        if not image_items or len(image_items) != len(mm_items):
+            return None
+
+        spatial_merge_size = self._spatial_merge_size
+        sorted_items = sorted(image_items, key=lambda item: item.offsets[0][0])
+        position_segments = []
+        st = 0
+        next_pos = 0
+
+        for item in sorted_items:
+            if item.offsets is None or len(item.offsets) != 1:
+                return None
+
+            start, end = item.offsets[0]
+            if start < st or end >= input_len:
+                return None
+
+            text_len = start - st
+            if text_len > 0:
+                position_segments.append(
+                    torch.arange(text_len, dtype=dtype, device=device)
+                    .view(1, -1)
+                    .expand(3, -1)
+                    + next_pos
+                )
+                next_pos += text_len
+
+            grid = self._as_grid_batch(item.model_specific_data.get("image_grid_thw"))
+            if grid is None or grid.shape[0] != 1:
+                return None
+            t, h, w = [int(x) for x in grid[0].tolist()]
+            llm_grid_t = t
+            llm_grid_h = h // spatial_merge_size
+            llm_grid_w = w // spatial_merge_size
+            num_image_tokens = llm_grid_t * llm_grid_h * llm_grid_w
+            if num_image_tokens != end - start + 1:
+                return None
+
+            t_index = (
+                torch.arange(llm_grid_t, dtype=dtype, device=device)
+                .view(-1, 1)
+                .expand(llm_grid_t, llm_grid_h * llm_grid_w)
+                .reshape(-1)
+            )
+            h_index = (
+                torch.arange(llm_grid_h, dtype=dtype, device=device)
+                .view(1, -1, 1)
+                .expand(llm_grid_t, llm_grid_h, llm_grid_w)
+                .reshape(-1)
+            )
+            w_index = (
+                torch.arange(llm_grid_w, dtype=dtype, device=device)
+                .view(1, 1, -1)
+                .expand(llm_grid_t, llm_grid_h, llm_grid_w)
+                .reshape(-1)
+            )
+            position_segments.append(
+                torch.stack([t_index, h_index, w_index]) + next_pos
+            )
+            next_pos += max(llm_grid_t, llm_grid_h, llm_grid_w)
+            st = end + 1
+
+        if st < input_len:
+            text_len = input_len - st
+            position_segments.append(
+                torch.arange(text_len, dtype=dtype, device=device)
+                .view(1, -1)
+                .expand(3, -1)
+                + next_pos
+            )
+
+        mrope_positions = torch.cat(position_segments, dim=1).unsqueeze(1)
+        mrope_position_delta = (mrope_positions.max() + 1 - input_len).reshape(1, 1)
+        return mrope_positions, mrope_position_delta
+
+    @classmethod
+    def _concat_mm_item_grid(cls, mm_items: list[MultimodalDataItem], key, modality):
+        grids = []
+        for item in mm_items:
+            if not item.is_modality(modality):
+                continue
+            grid = cls._as_grid_batch(item.model_specific_data.get(key))
+            if grid is not None:
+                grids.append(grid)
+        if not grids:
+            return None
+        if len(grids) == 1:
+            return grids[0]
+        return torch.cat(grids, dim=0)
+
+    @classmethod
+    def _get_grid_from_output_or_items(
+        cls, ret, mm_items, key, modality, input_data=None
+    ):
+        grid = cls._get_processor_output_value(ret, key)
+        if grid is None:
+            grid = cls._concat_mm_item_grid(mm_items, key, modality)
+        if grid is None and input_data and isinstance(input_data[0], dict):
+            grid = input_data[0].get(key)
+        return grid
+
+    def get_mm_data(self, prompt, embeddings, **kwargs):
+        img_grid_thw = kwargs.get("img_grid_thw", None)
+        video_grid_thw = kwargs.get("video_grid_thw", None)
+        audio_feature_lens = kwargs.get("audio_feature_lens", None)
+        video_timestamps = kwargs.get("video_timestamps", None)
+        second_per_grid_ts = kwargs.get("second_per_grid_ts", None)
+
+        audio_seq_lens = None
+        if audio_feature_lens is not None:
+            if self.model_type == "qwen3_omni_moe":
+                # apply _get_feat_extract_lengths to get seq_lens
+                input_lengths_leave = audio_feature_lens % 100
+                feat_lengths = (input_lengths_leave - 1) // 2 + 1
+                audio_seq_lens = (
+                    ((feat_lengths - 1) // 2 + 1 - 1) // 2
+                    + 1
+                    + (audio_feature_lens // 100) * 13
+                )
+            elif self.model_type == "qwen2_5_omni":
+                audio_seq_lens = (audio_feature_lens - 1) // 2 + 1
+                audio_seq_lens = (audio_seq_lens - 2) // 2 + 1
+
+        if (
+            self.model_type
+            in [
+                "qwen3_vl",
+                "qwen3_vl_moe",
+                "qwen3_5",
+                "qwen3_5_moe",
+                "sensenova_loop",
+                "qwen4_exp",
+                "intern_s2_preview",
+                "cosmos3_omni",
+            ]
+            and video_timestamps is not None
+        ):
+            input_ids, offsets, modality_list = self.build_input_ids_with_timestamps(
+                prompt, embeddings, img_grid_thw, video_grid_thw, video_timestamps
+            )
+        else:
+            input_ids, offsets, modality_list = self.build_input_ids(
+                prompt, img_grid_thw, video_grid_thw, audio_seq_lens=audio_seq_lens
+            )
+        assert all(isinstance(modality, Modality) for modality in modality_list)
+
+        mrope_positions, mrope_position_delta = MRotaryEmbedding.get_rope_index(
+            spatial_merge_size=self._spatial_merge_size,
+            image_token_id=self.mm_tokens.image_token_id,
+            video_token_id=self.mm_tokens.video_token_id,
+            vision_start_token_id=self.vision_start_token_id,
+            model_type=self.model_type,
+            input_ids=torch.tensor(input_ids, dtype=torch.long).unsqueeze(0),
+            image_grid_thw=img_grid_thw,
+            video_grid_thw=video_grid_thw,
+            second_per_grid_ts=second_per_grid_ts,
+            use_audio_in_video=False,
+            audio_seqlens=(
+                audio_feature_lens if self.model_type == "qwen3_omni_moe" else None
+            ),
+            audio_token_id=getattr(self.hf_config, "audio_token_id", None),
+            audio_start_token_id=self.audio_start_token_id,
+            position_id_per_seconds=getattr(
+                self.hf_config, "position_id_per_seconds", None
+            ),
+            tokens_per_second=self._tokens_per_second,
+        )
+        mrope_positions = mrope_positions.squeeze(1)
+
+        mm_items = []
+        consumed_per_modality = {}
+
+        for modality, offset in zip(modality_list, offsets):
+            num_tokens = offset[1] - offset[0] + 1
+            embedding_start = consumed_per_modality.get(modality, 0)
+            embedding_slice = embeddings[modality][
+                embedding_start : embedding_start + num_tokens
+            ]
+            consumed_per_modality[modality] = embedding_start + num_tokens
+            mm_items.append(
+                MultimodalDataItem(
+                    modality=modality,
+                    offsets=[offset],
+                    precomputed_embeddings=embedding_slice,
+                )
+            )
+
+        return MultimodalProcessorOutput(
+            input_ids=input_ids,
+            mm_items=mm_items,
+            im_start_id=self.IM_START_TOKEN_ID,
+            im_end_id=self.IM_END_TOKEN_ID,
+            im_token_id=self.mm_tokens.image_token_id,
+            video_token_id=self.mm_tokens.video_token_id,
+            audio_token_id=self.mm_tokens.audio_token_id,
+            mrope_positions=mrope_positions,
+            mrope_position_delta=mrope_position_delta,
+        )
+
+    def prepare_artifact_batch(
+        self,
+        entries: list[MediaArtifactInput],
+        *,
+        processor=None,
+    ) -> list[QwenVLImagePreprocessArtifact]:
+        """Preprocess image cache misses without retaining prompt-specific state."""
+        if not entries:
+            return []
+
+        processor, _ = self._resolve_processor(processor)
+        image_kwargs = dict(self.image_config or {})
+        processor_device = None
+        if (
+            isinstance(processor.image_processor, BaseImageProcessor)
+            and not self.disable_fast_image_processor
+        ):
+            processor_device = self._fast_image_processor_device(processor)
+            if processor_device is not None:
+                image_kwargs["device"] = processor_device
+
+        with self._temporary_fast_processor_cuda_pool(processor_device):
+            result = processor.image_processor(
+                images=[entry.media for entry in entries],
+                return_tensors="pt",
+                **image_kwargs,
+            )
+            features = self._get_processor_output_value(result, "pixel_values")
+            image_grid_thw = self._get_processor_output_value(result, "image_grid_thw")
+            if (
+                isinstance(features, torch.Tensor)
+                and (
+                    self.mm_preprocess_cache.enabled
+                    or not self.keep_mm_features_on_device
+                )
+                and not self.precompute_hash_before_cpu_transfer
+            ):
+                features = features.cpu()
+
+        if not isinstance(features, torch.Tensor):
+            raise TypeError("Qwen-VL image processor must return pixel_values")
+        if not isinstance(image_grid_thw, torch.Tensor):
+            image_grid_thw = torch.as_tensor(image_grid_thw, dtype=torch.long)
+        if image_grid_thw.ndim != 2 or image_grid_thw.shape != (len(entries), 3):
+            raise ValueError(
+                "Qwen-VL image processor returned an invalid image_grid_thw shape: "
+                f"{tuple(image_grid_thw.shape)}"
+            )
+        feature_lengths = image_grid_thw.prod(dim=1).tolist()
+        if sum(feature_lengths) != features.shape[0]:
+            raise ValueError(
+                "Qwen-VL image processor feature count does not match image grids: "
+                f"{features.shape[0]} != {sum(feature_lengths)}"
+            )
+
+        artifacts = []
+        feature_offset = 0
+        for entry, grid, feature_length in zip(
+            entries, image_grid_thw, feature_lengths
+        ):
+            feature = features[
+                feature_offset : feature_offset + feature_length
+            ].contiguous()
+            feature_offset += feature_length
+            # The artifact key already commits to the media content, processor
+            # fingerprint, and every preprocessing kwarg. Derive the downstream
+            # cache identity from it so independently preprocessed copies in
+            # different tokenizer workers share the same radix/VLM cache key.
+            feature_hash = resolve_multimodal_item_hash(
+                existing_hash=0, namespace=entry.artifact_key
+            )
+            artifacts.append(
+                QwenVLImagePreprocessArtifact(
+                    content_digest=entry.content_digest,
+                    artifact_key=entry.artifact_key,
+                    feature_hash=feature_hash,
+                    feature=feature,
+                    model_specific_data={"image_grid_thw": grid.unsqueeze(0)},
+                )
+            )
+        return artifacts
+
+    def compose_image_artifacts(
+        self,
+        input_text,
+        artifacts: list[QwenVLImagePreprocessArtifact],
+    ) -> Optional[MultimodalProcessorOutput]:
+        """Compose prompt tokens and request-owned items from cached images.
+
+        Returns None when the raw prompt cannot be aligned with the prepared
+        artifacts; the caller falls back to _process_mm_data_uncached.
+        """
+        image_grids = []
+        for artifact in artifacts:
+            grid = self._as_grid_batch(
+                artifact.model_specific_data.get("image_grid_thw")
+            )
+            if grid is None or grid.shape[0] != 1:
+                raise ValueError("Each Qwen-VL image artifact requires one image grid")
+            image_grids.append(grid)
+        image_grid_thw = torch.cat(image_grids, dim=0)
+
+        grid_key = tuple(
+            tuple(int(value) for value in row.tolist()) for row in image_grid_thw
+        )
+        normalized_text = self._normalize_prompt_for_fast_path(
+            input_text, expected_image_count=len(grid_key)
+        )
+        if normalized_text is None:
+            return None
+        try:
+            if isinstance(normalized_text, str):
+                (
+                    input_ids_tuple,
+                    offsets,
+                    mrope_positions,
+                    mrope_position_delta,
+                ) = self._cached_image_prompt_template(normalized_text, grid_key)
+            else:
+                (
+                    input_ids_tuple,
+                    offsets,
+                    mrope_positions,
+                    mrope_position_delta,
+                ) = self._build_image_prompt_template(normalized_text, grid_key)
+        except ValueError as exc:
+            if "prompt placeholders" not in str(exc):
+                raise
+            logger.debug(
+                "Qwen-VL fast path skipped after normalization (%s); "
+                "falling back to full preprocessing",
+                exc,
+            )
+            return None
+        input_ids_list = list(input_ids_tuple)
+
+        mm_items = []
+        for artifact, offset in zip(artifacts, offsets):
+            item = MultimodalDataItem(
+                modality=Modality.IMAGE,
+                feature=artifact.feature,
+                offsets=[offset],
+                model_specific_data=deepcopy(artifact.model_specific_data),
+            )
+            item.set_hash(artifact.feature_hash)
+            mm_items.append(item)
+
+        padded_input_ids = MultimodalProcessorOutput.build_padded_input_ids(
+            input_ids_list, mm_items
+        )
+        mm_items = self._prepare_mm_items_for_transport(mm_items)
+        self._mark_cuda_ipc_features_for_deferred_reconstruction(mm_items)
+        return MultimodalProcessorOutput(
+            input_ids=input_ids_list,
+            padded_input_ids=padded_input_ids,
+            mm_items=mm_items,
+            im_start_id=self.vision_start_token_id,
+            im_end_id=self.vision_end_token_id,
+            im_token_id=self.mm_tokens.image_token_id,
+            video_token_id=self.mm_tokens.video_token_id,
+            audio_token_id=self.mm_tokens.audio_token_id,
+            mrope_positions=mrope_positions.clone(),
+            mrope_position_delta=mrope_position_delta.clone(),
+        )
+
+    def _normalize_prompt_for_fast_path(self, input_text, expected_image_count: int):
+        # Non-string input (list of ints) is already in the pre-tokenized shape
+        # build_input_ids expects; only string prompts need placeholder rewriting.
+        if not isinstance(input_text, str):
+            return input_text
+        native = self.mm_tokens.image_token
+        normalized, count = self.mm_tokens.image_token_regex.subn(native, input_text)
+        if count != expected_image_count:
+            logger.debug(
+                "Qwen-VL fast path skipped: %d image placeholder(s) in prompt "
+                "vs %d prepared artifact(s); falling back to full preprocessing",
+                count,
+                expected_image_count,
+            )
+            return None
+        return normalized
+
+    @lru_cache(maxsize=256)
+    def _cached_image_prompt_template(self, input_text: str, grid_key: tuple):
+        """Cache prompt expansion and M-RoPE by prompt and image-grid shape."""
+        return self._build_image_prompt_template(input_text, grid_key)
+
+    def _build_image_prompt_template(self, input_text, grid_key: tuple):
+        image_grid_thw = torch.tensor(grid_key, dtype=torch.long)
+        input_ids_list, offsets, modalities = self.build_input_ids(
+            input_text, img_grid_thw=image_grid_thw
+        )
+        if modalities != [Modality.IMAGE] * len(grid_key):
+            raise ValueError("Qwen-VL image artifacts do not match prompt placeholders")
+
+        template_items = [
+            MultimodalDataItem(
+                modality=Modality.IMAGE,
+                feature=None,
+                offsets=[offset],
+                model_specific_data={"image_grid_thw": grid.unsqueeze(0)},
+            )
+            for grid, offset in zip(image_grid_thw, offsets)
+        ]
+        input_ids = torch.tensor(input_ids_list, dtype=torch.long)
+        mrope_result = self._compute_image_only_mrope_positions_from_offsets(
+            input_len=input_ids.numel(),
+            mm_items=template_items,
+            dtype=input_ids.dtype,
+            device=input_ids.device,
+        )
+        if mrope_result is None:
+            mrope_result = self.compute_mrope_positions(input_ids_list, template_items)
+        mrope_positions, mrope_position_delta = mrope_result
+        if mrope_positions is not None and mrope_positions.ndim == 3:
+            mrope_positions = mrope_positions.squeeze(1)
+        return (
+            tuple(input_ids_list),
+            tuple(offsets),
+            mrope_positions,
+            mrope_position_delta,
+        )
+
+    async def _process_mm_data_uncached(
+        self,
+        image_data: List[Union[str, bytes]],
+        input_text,
+        request_obj,
+        *args,
+        **kwargs,
+    ):
+        entry_time = time.perf_counter()
+        base_output = await self.load_mm_data(
+            prompt=input_text,
+            image_data=image_data,
+            video_data=request_obj.video_data,
+            audio_data=request_obj.audio_data,
+            multimodal_tokens=self.mm_tokens,
+        )
+        load_time = time.perf_counter()
+        rid = getattr(request_obj, "rid", "anonymous_rid")
+
+        video_metadata = None
+        if base_output.videos and not isinstance(base_output.videos[0], dict):
+            videos_processed = [
+                await preprocess_video(video, video_config=self.video_config)
+                for video in base_output.videos
+            ]
+            base_output.videos, video_metadata = map(list, zip(*videos_processed))
+
+        preprocess_time = time.perf_counter()
+
+        processor_kwargs = {}
+        processor_video_config = _get_processor_video_config(
+            self.video_config, video_metadata
+        )
+        if processor_video_config is not None:
+            processor_kwargs["processor_video_config"] = processor_video_config
+
+        # NOTE: for qwen3-vl, video_meta need to be passed in, since do_sample_frames is already done in preprocess_video
+        if self.hf_config.model_type in (
+            "qwen3_vl",
+            "qwen3_vl_moe",
+            "qwen3_5",
+            "qwen3_5_moe",
+            "sensenova_loop",
+            "qwen4_exp",
+            "intern_s2_preview",
+            "interns2_mobius",
+            "cosmos3_omni",
+        ):
+            processor_kwargs.update(
+                video_metadata=video_metadata,
+                do_sample_frames=False,
+            )
+
+        mm_items, input_ids, ret = await self.process_and_combine_mm_data_async(
+            base_output, self.mm_tokens, **processor_kwargs
+        )
+
+        self._mark_cuda_ipc_features_for_deferred_reconstruction(mm_items)
+
+        audio_feature_lengths = None
+
+        if self.model_type == "qwen3_omni_moe":
+            audio_item = next((mm for mm in mm_items if mm.is_audio()), None)
+            if audio_item:
+                audio_feature_lengths = torch.sum(
+                    audio_item.feature_attention_mask, dim=1
+                )
+
+        second_per_grid_ts = self._get_processor_output_value(ret, "second_per_grid_ts")
+        if second_per_grid_ts is None:
+            second_per_grid_ts = self._get_processor_output_value(
+                ret, "video_second_per_grid"
+            )
+
+        process_time = time.perf_counter()
+
+        input_ids = input_ids.flatten()
+        base_input_ids = getattr(base_output, "input_ids", None)
+        if (
+            isinstance(base_input_ids, list)
+            and len(base_input_ids) == input_ids.numel()
+        ):
+            # reuse preprocess input if it already carries list of input_ids
+            input_ids_list = base_input_ids
+        else:
+            input_ids_list = input_ids.tolist()
+
+        # look for if padded_input_ids already exists before computing
+        padded_input_ids = self._get_processor_output_value(ret, "padded_input_ids")
+        if padded_input_ids is None:
+            padded_input_ids = MultimodalProcessorOutput.build_padded_input_ids(
+                input_ids_list, mm_items
+            )
+        elif isinstance(padded_input_ids, torch.Tensor):
+            # reuse existing padded_input_ids
+            padded_input_ids = padded_input_ids.flatten().tolist()
+        else:
+            padded_input_ids = list(padded_input_ids)
+
+        image_grid_thw = self._get_grid_from_output_or_items(
+            ret, mm_items, "image_grid_thw", Modality.IMAGE, image_data
+        )
+        video_grid_thw = self._get_grid_from_output_or_items(
+            ret,
+            mm_items,
+            "video_grid_thw",
+            Modality.VIDEO,
+            request_obj.video_data,
+        )
+
+        mrope_result = self._get_precomputed_mrope_from_output(ret)
+        if mrope_result is None:
+            if (
+                video_grid_thw is None
+                and second_per_grid_ts is None
+                and audio_feature_lengths is None
+            ):
+                mrope_result = self._compute_image_only_mrope_positions_from_offsets(
+                    input_len=input_ids.numel(),
+                    mm_items=mm_items,
+                    dtype=input_ids.dtype,
+                    device=input_ids.device,
+                )
+        if mrope_result is None:
+            mrope_result = MRotaryEmbedding.get_rope_index(
+                spatial_merge_size=self._spatial_merge_size,
+                image_token_id=self.mm_tokens.image_token_id,
+                video_token_id=self.mm_tokens.video_token_id,
+                vision_start_token_id=self.vision_start_token_id,
+                model_type=self.model_type,
+                tokens_per_second=self._tokens_per_second,
+                # use the expanded token ids
+                input_ids=input_ids.unsqueeze(0),
+                image_grid_thw=image_grid_thw,
+                video_grid_thw=video_grid_thw,
+                second_per_grid_ts=second_per_grid_ts,
+                use_audio_in_video=False,
+                audio_seqlens=audio_feature_lengths,
+                audio_token_id=getattr(self.hf_config, "audio_token_id", None),
+                audio_start_token_id=self.audio_start_token_id,
+                position_id_per_seconds=getattr(
+                    self.hf_config, "position_id_per_seconds", None
+                ),
+            )
+
+        mrope_positions, mrope_position_delta = mrope_result
+        if mrope_positions.ndim == 3:
+            mrope_positions = mrope_positions.squeeze(1)
+        get_rope_index_time = time.perf_counter()
+        logger.debug(
+            f"[QwenVLProcessor Perf] {rid=}, "
+            f"load_time: {(load_time - entry_time) * 1000:.2f} ms, "
+            f"preprocess_time: {(preprocess_time - load_time) * 1000:.2f} ms, "
+            f"process_time: {(process_time - preprocess_time) * 1000:.2f} ms, "
+            f"get_rope_index_time: {(get_rope_index_time - process_time) * 1000:.2f} ms, "
+            f"total_time: {(get_rope_index_time - entry_time) * 1000:.2f} ms"
+        )
+
+        return MultimodalProcessorOutput(
+            input_ids=input_ids_list,
+            padded_input_ids=padded_input_ids,
+            mm_items=mm_items,
+            im_start_id=self.vision_start_token_id,
+            im_end_id=self.vision_end_token_id,
+            im_token_id=self.mm_tokens.image_token_id,
+            video_token_id=self.mm_tokens.video_token_id,
+            audio_token_id=self.mm_tokens.audio_token_id,
+            mrope_positions=mrope_positions,
+            mrope_position_delta=mrope_position_delta,
+        )
+
+    async def process_mm_data_async(
+        self,
+        image_data: List[Union[str, bytes]],
+        input_text,
+        request_obj,
+        *args,
+        **kwargs,
+    ):
+        if (
+            not image_data
+            or request_obj.video_data
+            or request_obj.audio_data
+            or any(self._is_preprocessed_input(item) for item in image_data)
+            or (
+                not self.mm_preprocess_cache.enabled
+                and not self.uses_media_artifacts_without_cache
+            )
+        ):
+            return await self._process_mm_data_uncached(
+                image_data, input_text, request_obj, *args, **kwargs
+            )
+
+        prepare_artifacts = (
+            self.prepare_media_artifacts
+            if self.mm_preprocess_cache.enabled
+            else self.prepare_media_artifacts_without_cache
+        )
+        artifacts = await prepare_artifacts(
+            image_data, content_hashes=getattr(request_obj, "mm_content_hashes", None)
+        )
+        composed = self.compose_image_artifacts(input_text, artifacts)
+        if composed is not None:
+            return composed
+        return await self._process_mm_data_uncached(
+            image_data, input_text, request_obj, *args, **kwargs
+        )
+
+    def _mark_cuda_ipc_features_for_deferred_reconstruction(self, mm_items):
+        supports_deferred_reconstruction = get_mm().mm_enable_dp_encoder or (
+            get_parallel().tp_size == 1
+            and self.model_type in ("qwen3_vl", "qwen3_vl_moe")
+        )
+        if not (
+            self.keep_mm_features_on_device
+            and supports_deferred_reconstruction
+            and self.model_type
+            in ("qwen3_vl", "qwen3_vl_moe", "qwen3_5", "qwen3_5_moe", "sensenova_loop")
+        ):
+            return
+        for item in mm_items:
+            if item.is_image() or item.is_video():
+                item.model_specific_data[DEFER_CUDA_IPC_FEATURE_RECONSTRUCTION_KEY] = (
+                    True
+                )

runtime-v2/overlay/python/sglang/srt/utils/hf_transformers/common.py

@@ -0,0 +1,785 @@
+# Copyright 2023-2024 SGLang Team
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+"""Shared helpers used by config, tokenizer, and processor modules."""
+
+import json
+import os
+from pathlib import Path
+from typing import Any, Dict, Optional, Type, Union
+
+import torch
+from huggingface_hub import snapshot_download
+
+from sglang.srt.configs import (
+    AfmoeConfig,
+    BailingHybridConfig,
+    BailingMM2Config,
+    BailingMoeV3VLConfig,
+    BeamConfig,
+    ChatGLMConfig,
+    Cosmos3Config,
+    Cosmos3EdgeConfig,
+    Cosmos3EdgeProjectorConfig,
+    Cosmos3EdgeTextConfig,
+    Cosmos3EdgeVisionConfig,
+    DbrxConfig,
+    DeepseekVL2Config,
+    Dots3Config,
+    DotsOCRConfig,
+    DotsVLMConfig,
+    ExaoneConfig,
+    FalconH1Config,
+    FalconMambaConfig,
+    FlintConfig,
+    GigaChat35Config,
+    Glm5NextConfig,
+    Glm5NextTextConfig,
+    GraniteMoeHybridConfig,
+    HYV4Config,
+    InklingAudioConfig,
+    InklingMMConfig,
+    InklingModelConfig,
+    InklingVisionConfig,
+    InternS2MobiusConfig,
+    InternS2MobiusTextConfig,
+    InternS2PreviewConfig,
+    IQuestQ1Config,
+    IQuestQ1MTPConfig,
+    JetNemotronConfig,
+    JetVLMConfig,
+    K2HorizonConfig,
+    KimiK3Config,
+    KimiK25Config,
+    KimiLinearConfig,
+    KimiVLConfig,
+    LagunaConfig,
+    LocateAnythingConfig,
+    LongcatFlashConfig,
+    Mamba2Config,
+    MambaConfig,
+    MiniCPMHybridConfig,
+    MiniCPMV4_6Config,
+    MiniCPMV4_6VisionConfig,
+    MiniCPMV4_7Config,
+    MiniMaxM3VLConfig,
+    MultiModalityConfig,
+    MuseGlimmerAssistantConfig,
+    MuseGlimmerConfig,
+    NanbeigeConfig,
+    NemotronH_Nano_Omni_Reasoning_V3_Config,
+    NemotronH_Nano_VL_V2_Config,
+    NemotronH_Omni_Reasoning_V3_Config,
+    NemotronHConfig,
+    NemotronHPuzzleConfig,
+    Olmo3Config,
+    Qwen3_5Config,
+    Qwen3_5MoeConfig,
+    Qwen3_5MoeTextConfig,
+    Qwen3_5TextConfig,
+    Qwen3NextConfig,
+    Qwen4ExpConfig,
+    Qwen4ExpTextConfig,
+    SenseNovaLoopConfig,
+    Spark2_5Config,
+    Step3p5Config,
+    Step3p7Config,
+    Step3VLConfig,
+    XllmConfig,
+)
+from sglang.srt.configs.deepseek_ocr import DeepseekVLV2Config
+from sglang.srt.configs.deepseek_v41 import DEEPSEEK_V41_CONFIG_CLASSES
+from sglang.srt.configs.internvl import InternVLChatConfig
+from sglang.srt.utils import get_bool_env_var, logger, lru_cache_frozenset
+from sglang.srt.utils.runai_utils import ObjectStorageModel, is_runai_obj_uri
+
+from ..hf_transformers_patches import normalize_rope_scaling_compat
+
+if get_bool_env_var("SGLANG_USE_MODELSCOPE"):
+    from modelscope import AutoConfig, GenerationConfig
+else:
+    from transformers import AutoConfig, GenerationConfig
+
+from transformers import PretrainedConfig
+
+# ---------------------------------------------------------------------------
+# Config registry
+# ---------------------------------------------------------------------------
+
+_CONFIG_REGISTRY: Dict[str, Type[PretrainedConfig]] = {
+    cls.model_type: cls
+    for cls in [
+        AfmoeConfig,
+        BailingHybridConfig,
+        BailingMM2Config,
+        BailingMoeV3VLConfig,
+        BeamConfig,
+        ChatGLMConfig,
+        DbrxConfig,
+        ExaoneConfig,
+        DeepseekVL2Config,
+        MultiModalityConfig,
+        KimiVLConfig,
+        K2HorizonConfig,
+        LocateAnythingConfig,
+        InternVLChatConfig,
+        LagunaConfig,
+        Spark2_5Config,
+        Step3VLConfig,
+        LongcatFlashConfig,
+        Olmo3Config,
+        MuseGlimmerConfig,
+        MuseGlimmerAssistantConfig,
+        KimiK3Config,
+        Glm5NextConfig,
+        Glm5NextTextConfig,
+        KimiLinearConfig,
+        Qwen3NextConfig,
+        Qwen4ExpConfig,
+        Qwen4ExpTextConfig,
+        FalconH1Config,
+        FalconMambaConfig,
+        Mamba2Config,
+        MambaConfig,
+        GigaChat35Config,
+        GraniteMoeHybridConfig,
+        HYV4Config,
+        DotsVLMConfig,
+        DotsOCRConfig,
+        Dots3Config,
+        NemotronH_Nano_VL_V2_Config,
+        NemotronH_Nano_Omni_Reasoning_V3_Config,
+        NemotronH_Omni_Reasoning_V3_Config,
+        NemotronHConfig,
+        NemotronHPuzzleConfig,
+        NanbeigeConfig,
+        DeepseekVLV2Config,
+        Qwen3_5Config,
+        Qwen3_5MoeConfig,
+        Qwen3_5TextConfig,
+        Qwen3_5MoeTextConfig,
+        SenseNovaLoopConfig,
+        InternS2PreviewConfig,
+        IQuestQ1Config,
+        IQuestQ1MTPConfig,
+        FlintConfig,
+        InternS2MobiusConfig,
+        InternS2MobiusTextConfig,
+        JetNemotronConfig,
+        JetVLMConfig,
+        KimiK25Config,
+        Step3p5Config,
+        Step3p7Config,
+        MiniCPMHybridConfig,
+        MiniCPMV4_6Config,
+        MiniCPMV4_6VisionConfig,
+        MiniCPMV4_7Config,
+        InklingModelConfig,
+        InklingAudioConfig,
+        InklingVisionConfig,
+        InklingMMConfig,
+        MiniMaxM3VLConfig,
+        XllmConfig,
+    ]
+}
+
+
+# DeepSeek V3.2 / V4 reuse the V3 config schema. Subclass the upstream
+# transformers class with each model_type so AutoConfig.register passes its
+# consistency check (which requires class.model_type == registered key).
+# Default-value divergences (e.g. V4's topk_group) are handled in
+# model_config.py post-load.
+try:
+    from transformers import DeepseekV3Config as _HFDeepseekV3Config
+
+    class _DeepseekV32ConfigAlias(_HFDeepseekV3Config):
+        model_type = "deepseek_v32"
+
+    class _DeepseekV4ConfigAlias(_HFDeepseekV3Config):
+        model_type = "deepseek_v4"
+        hc_pre_from_prev_sublayer = False
+        # V4 normalizes each attention query head (weightless rmsnorm) before RoPE.
+        q_head_norm = True
+        kv_source_layer_ids = ()
+        index_source_layer_ids = ()
+        candidate_source_layer_id = -1
+        candidate_topk_blocks = 0
+        candidate_block_size = 0
+        engram_layer_ids = ()
+        engram_num_embeddings = ()
+        engram_max_ngram_size = 1
+        engram_vocab_size = 0
+        engram_n_heads = 0
+        engram_head_dim = 0
+        engram_pad_token_id = 2
+        engram_compressed_vocab_size = 0
+
+    _CONFIG_REGISTRY["deepseek_v32"] = _DeepseekV32ConfigAlias
+    _CONFIG_REGISTRY["deepseek_v4"] = _DeepseekV4ConfigAlias
+    _CONFIG_REGISTRY.update(
+        {cls.model_type: cls for cls in DEEPSEEK_V41_CONFIG_CLASSES}
+    )
+
+    # For kimi_k25_eagle3
+    class _KimiK2ConfigAlias(_HFDeepseekV3Config):
+        model_type = "kimi_k2"
+
+    _CONFIG_REGISTRY["kimi_k2"] = _KimiK2ConfigAlias
+except ImportError:
+    pass
+
+# Newer transformers versions (>=5.10.2) expose MellumConfig directly,
+# but fallback to Qwen3MoeConfig for older versions.
+try:
+    import transformers as _hf_transformers
+
+    _HFMellumConfig = getattr(_hf_transformers, "MellumConfig", None)
+
+    if _HFMellumConfig is not None:
+        _CONFIG_REGISTRY["mellum"] = _HFMellumConfig
+    else:
+        from transformers import Qwen3MoeConfig as _HFQwen3MoeConfig
+
+        class _MellumConfigAlias(_HFQwen3MoeConfig):
+            model_type = "mellum"
+
+            def __post_init__(self, **kwargs):
+                # Qwen3MoeConfig.__post_init__ wipes sliding_window unless
+                # use_sliding_window=True. Mellum gates sliding attention
+                # per-layer via layer_types, so preserve sliding_window
+                # regardless of the legacy use_sliding_window flag.
+                sliding_window = getattr(self, "sliding_window", None)
+                super().__post_init__(**kwargs)
+                self.sliding_window = sliding_window
+
+        _CONFIG_REGISTRY["mellum"] = _MellumConfigAlias
+
+except ImportError:
+    pass
+
+
+try:
+    from transformers import Gemma4Config as _HFGemma4Config
+
+    class _Gemma4UnifiedConfigAlias(_HFGemma4Config):
+        model_type = "gemma4_unified"
+
+    _CONFIG_REGISTRY["gemma4_unified"] = _Gemma4UnifiedConfigAlias
+except ImportError:
+    pass
+
+# Bare register: `get_config` re-parses every registry entry through its own
+# class, so AutoConfig need not resolve to it.
+#
+# `_LazyAutoMapping` keys on the config class `__name__`, so a differently-named
+# shadow drops out of PROCESSOR/TOKENIZER/MODEL_MAPPING.
+for name, cls in _CONFIG_REGISTRY.items():
+    try:
+        AutoConfig.register(name, cls)
+    except ValueError as e:
+        err = str(e).lower()
+        if "already registered" not in err and "already used" not in err:
+            logger.warning("Failed to register config %s: %s", name, e)
+
+# Cosmos3 (understanding tower) reuses the Qwen3-VL config schema. Register it
+# with AutoConfig only (not `_CONFIG_REGISTRY`), so the nested `text_config` is
+# flattened onto the top-level config in `get_config` — the same path the base
+# Qwen3-VL config relies on. Adding it to `_CONFIG_REGISTRY` would trigger a
+# `from_pretrained` reload that drops that flattening.
+#
+# transformers owns `cosmos3_omni` as `Cosmos3OmniConfig`, so the name
+# constraint above means this registration must lose to it.
+try:
+    AutoConfig.register(Cosmos3Config.model_type, Cosmos3Config)
+except ValueError as e:
+    err = str(e).lower()
+    if "already registered" not in err and "already used" not in err:
+        logger.warning("Failed to register config %s: %s", Cosmos3Config.model_type, e)
+
+# Cosmos3-Edge native text support starts from the checkpoint root config, then
+# consumes ``text_config`` in ``sglang.srt.models.cosmos3_edge``. Keep it out of
+# `_CONFIG_REGISTRY` so the generic parser can flatten text attributes onto the
+# root config after `AutoConfig.from_pretrained`, matching other multimodal
+# configs that use a text sub-config.
+#
+# `exist_ok=True`: no registry re-parse backs these up, so AutoConfig itself
+# must resolve to SGLang's. Safe because they reuse the native class names.
+for _cosmos3_edge_config_cls in (
+    Cosmos3EdgeTextConfig,
+    Cosmos3EdgeVisionConfig,
+    Cosmos3EdgeProjectorConfig,
+    Cosmos3EdgeConfig,
+):
+    try:
+        AutoConfig.register(
+            _cosmos3_edge_config_cls.model_type,
+            _cosmos3_edge_config_cls,
+            exist_ok=True,
+        )
+    except ValueError as e:
+        logger.warning(
+            "Failed to register config %s: %s",
+            _cosmos3_edge_config_cls.model_type,
+            e,
+        )
+
+
+# ---------------------------------------------------------------------------
+# Download / path helpers
+# ---------------------------------------------------------------------------
+
+
+def download_from_hf(
+    model_path: str,
+    allow_patterns: Optional[Union[str, list]] = None,
+):
+    if os.path.exists(model_path):
+        return model_path
+
+    if not allow_patterns:
+        allow_patterns = ["*.json", "*.bin", "*.model"]
+
+    return snapshot_download(model_path, allow_patterns=allow_patterns)
+
+
+def resolve_runai_obj_uri(model_name_or_path: str) -> str:
+    if is_runai_obj_uri(model_name_or_path):
+        return ObjectStorageModel.get_path(model_name_or_path)
+    return model_name_or_path
+
+
+def _resolve_local_or_cached_file(model_name_or_path, filename, revision=None):
+    """Resolve a file from a local directory or HF hub cache (no network)."""
+    local_path = Path(model_name_or_path) / filename
+    if local_path.is_file():
+        return str(local_path)
+    from huggingface_hub import hf_hub_download
+
+    return hf_hub_download(
+        model_name_or_path, filename, revision=revision, local_files_only=True
+    )
+
+
+def _cached_file_exists(model_name_or_path, filename, revision=None) -> bool:
+    """Whether *filename* is available locally or in the HF cache (no network)."""
+    try:
+        _resolve_local_or_cached_file(model_name_or_path, filename, revision)
+        return True
+    except Exception:
+        return False
+
+
+def _remote_file_exists(repo_id, filename, revision=None) -> bool:
+    """Whether *filename* exists on the HF hub (HEAD request only, no download).
+
+    Returns False on any error (offline, gated, network, invalid id) so callers
+    fall back to their default path instead of crashing.
+    """
+    from huggingface_hub.constants import HF_HUB_OFFLINE
+
+    if HF_HUB_OFFLINE:
+        return False
+    try:
+        from huggingface_hub import HfApi
+
+        return HfApi().file_exists(repo_id, filename, revision=revision)
+    except Exception:
+        return False
+
+
+def check_gguf_file(model: Union[str, os.PathLike]) -> bool:
+    model = Path(model)
+    if not model.is_file():
+        return False
+    elif model.suffix == ".gguf":
+        return True
+
+    with open(model, "rb") as f:
+        header = f.read(4)
+    return header == b"GGUF"
+
+
+def resolve_hf_gguf_reference(
+    model: str, revision: Optional[str] = None
+) -> Optional[str]:
+    """Download a .gguf named by Hub reference and return its local path.
+
+    owner/repo/path/inside/repo.gguf   -> exactly that file
+    owner/repo:QUANT_TYPE              -> the only matching quantization
+    owner/repo                         -> the only .gguf in the repo
+    """
+    from sglang.srt.utils import is_remote_url
+
+    if not model or os.path.exists(model) or is_remote_url(model):
+        return None
+
+    from huggingface_hub import hf_hub_download
+
+    if ":" in model:
+        repo_id, _, quant_type = model.rpartition(":")
+        if repo_id.count("/") != 1 or not quant_type:
+            return None
+
+        from huggingface_hub import HfApi
+
+        files = [
+            sibling.rfilename
+            for sibling in HfApi().repo_info(repo_id, revision=revision).siblings
+        ]
+        suffix = f"-{quant_type}.gguf"
+        candidates = [filename for filename in files if filename.endswith(suffix)]
+        if not candidates:
+            available = sorted(
+                filename for filename in files if filename.endswith(".gguf")
+            )
+            raise ValueError(
+                f"No file matching quant type {quant_type!r} in {repo_id}. "
+                f"Available GGUF files: {available}"
+            )
+        if len(candidates) > 1:
+            raise ValueError(
+                f"Quant type {quant_type!r} is ambiguous in {repo_id}: "
+                f"{sorted(candidates)}. Pass the full owner/repo/path/file.gguf "
+                "reference instead."
+            )
+        return hf_hub_download(repo_id, candidates[0], revision=revision)
+
+    parts = model.strip("/").split("/")
+    if len(parts) < 2:
+        return None
+
+    if len(parts) > 2 and model.endswith(".gguf"):
+        repo_id = "/".join(parts[:2])
+        filename = "/".join(parts[2:])
+        return hf_hub_download(repo_id, filename, revision=revision)
+
+    if len(parts) != 2:
+        return None
+
+    from huggingface_hub import HfApi
+
+    try:
+        files = [
+            s.rfilename for s in HfApi().repo_info(model, revision=revision).siblings
+        ]
+    except Exception:
+        return None
+    if any(f == "config.json" for f in files):
+        return None
+
+    candidates = [f for f in files if f.endswith(".gguf")]
+    if not candidates:
+        return None
+    if len(candidates) > 1:
+        listing = "\n  ".join(f"{model}/{f}" for f in sorted(candidates))
+        raise ValueError(
+            f"{model} contains {len(candidates)} .gguf files; name the one to "
+            f"serve:\n  {listing}"
+        )
+    return hf_hub_download(model, candidates[0], revision=revision)
+
+
+def gguf_sidecar_dir(
+    gguf_path: Union[str, os.PathLike], sentinel: str
+) -> Optional[Path]:
+    """Directory containing *sentinel* next to a .gguf file, if there is one."""
+    directory = Path(gguf_path).parent
+    return directory if (directory / sentinel).is_file() else None
+
+
+# ---------------------------------------------------------------------------
+# Rope / text config helpers
+# ---------------------------------------------------------------------------
+
+
+def get_rope_config(config):
+    """Get (rope_theta, rope_params) from config, supporting both v4 and v5.
+
+    Trust-remote-code configs or parent configs passed to sub-models may not
+    have the v5 ``rope_parameters`` property, so we fall back to the v4-style
+    ``config.rope_theta`` / ``config.rope_scaling`` attributes.
+
+    Returns:
+        (rope_theta, rope_params): In v5, rope_params is the full
+        rope_parameters dict (which subsumes rope_scaling and includes
+        rope_theta). In v4, rope_params is the rope_scaling dict or None.
+    """
+    rope_params = getattr(config, "rope_parameters", None)
+    if rope_params is not None:
+        rope_theta = rope_params.get("rope_theta", getattr(config, "rope_theta", 10000))
+        return rope_theta, rope_params
+    return getattr(config, "rope_theta", 10000), getattr(config, "rope_scaling", None)
+
+
+def _patch_text_config(parent_config: PretrainedConfig, text_config):
+    """Synchronize standard attributes between parent config and text sub-config.
+
+    In transformers v5, the "untangle config" refactor removed automatic
+    inheritance of top-level PretrainedConfig attributes (pad_token_id,
+    tie_word_embeddings, etc.) from sub-configs. Downstream code expects
+    these attributes to be present on both configs (some models pass the
+    parent directly to the language model, others pass the text sub-config),
+    so we propagate in both directions when an attribute is missing.
+    (See https://github.com/huggingface/transformers/pull/41541)
+    """
+    _ATTRS_TO_PROPAGATE = [
+        "pad_token_id",
+        "bos_token_id",
+        "eos_token_id",
+        "tie_word_embeddings",
+    ]
+    for attr in _ATTRS_TO_PROPAGATE:
+        parent_has = hasattr(parent_config, attr)
+        text_has = hasattr(text_config, attr)
+        if parent_has and not text_has:
+            setattr(text_config, attr, getattr(parent_config, attr))
+        elif text_has and not parent_has:
+            setattr(parent_config, attr, getattr(text_config, attr))
+    return text_config
+
+
+def get_hf_text_config(config: PretrainedConfig):
+    """Get the "sub" config relevant to llm for multi modal models.
+    No op for pure text models.
+    """
+    if config.architectures is not None:
+        class_name = config.architectures[0]
+        if class_name.startswith("Llava") and class_name.endswith("ForCausalLM"):
+            # We support non-hf version of llava models, so we do not want to
+            # read the wrong values from the unused default text_config.
+            # NOTE(HandH1998): We set `torch_dtype` of config to `torch.float16` for the weights, as
+            # `torch.float16` is default used for image features in `python/sglang/srt/models/llava.py`.
+            setattr(config, "dtype", torch.float16)
+            return config
+
+    text_config = None
+
+    # Some models (e.g. DeepSeek-OCR) store sub-configs as plain dicts.
+    # Convert to PretrainedConfig early so hasattr() checks and asserts work.
+    parent_dtype = getattr(config, "dtype", None)
+    for _attr in ("text_config", "llm_config", "language_config", "thinker_config"):
+        _sub = getattr(config, _attr, None)
+        if isinstance(_sub, dict):
+            _converted = PretrainedConfig(**_sub)
+            if getattr(_converted, "dtype", None) is None and parent_dtype is not None:
+                _converted.dtype = parent_dtype
+            setattr(config, _attr, _converted)
+        elif _sub is not None and parent_dtype is not None:
+            # transformers v5 multimodal configs (e.g. Mistral3Config) carry
+            # `dtype` only on the top-level config, leaving the sub-configs at
+            # None. Without this, _get_and_verify_dtype falls back to float32
+            # and then "auto" downcasts to float16, which overflows the Pixtral
+            # vision tower on real images and produces NaN features.
+            if getattr(_sub, "dtype", None) is None:
+                _sub.dtype = parent_dtype
+
+    # Priority: thinker_config > llm_config > language_config > text_config
+    if hasattr(config, "thinker_config"):
+        # qwen2.5 omni
+        thinker_config = config.thinker_config
+        if hasattr(thinker_config, "text_config"):
+            setattr(
+                thinker_config.text_config,
+                "dtype",
+                getattr(thinker_config, "dtype", None),
+            )
+            text_config = thinker_config.text_config
+        else:
+            text_config = thinker_config
+    elif hasattr(config, "llm_config"):
+        # PointsV1.5 Chat Model
+        assert hasattr(config.llm_config, "num_attention_heads")
+        text_config = config.llm_config
+    elif hasattr(config, "language_config"):
+        text_config = config.language_config
+    elif hasattr(config, "text_config"):
+        # The code operates under the assumption that text_config should have
+        # `num_attention_heads` (among others). Assert here to fail early
+        # if transformers config doesn't align with this assumption.
+        assert hasattr(config.text_config, "num_attention_heads")
+        text_config = config.text_config
+
+    # Ensure rope_scaling dicts have "type" for remote-code compat (v5).
+    normalize_rope_scaling_compat(config)
+
+    if text_config is not None:
+        return _patch_text_config(config, text_config)
+    return config
+
+
+# ---------------------------------------------------------------------------
+# Model-specific helpers
+# ---------------------------------------------------------------------------
+
+
+def _ensure_sub_configs(config: PretrainedConfig, *attr_names: str) -> None:
+    """Convert dict-valued sub-configs to proper AutoConfig objects in-place."""
+    for attr in attr_names:
+        sub = getattr(config, attr, None)
+        if sub is not None and isinstance(sub, dict):
+            setattr(config, attr, AutoConfig.for_model(**sub))
+
+
+def _is_deepseek_ocr_model(config: PretrainedConfig) -> bool:
+    # TODO: Remove this workaround once AutoConfig correctly identifies deepseek-ocr.
+    # Hugging Face's AutoConfig currently misidentifies it as deepseekvl2.
+    auto_map = getattr(config, "auto_map", None) or {}
+    return auto_map.get("AutoModel") == "modeling_deepseekocr.DeepseekOCRForCausalLM"
+
+
+def _is_deepseek_ocr2_model(config: PretrainedConfig) -> bool:
+    auto_map = getattr(config, "auto_map", None) or {}
+    return auto_map.get("AutoModel") == "modeling_deepseekocr2.DeepseekOCR2ForCausalLM"
+
+
+def _override_v_head_dim_if_zero(config: PretrainedConfig, patch: int = 128) -> None:
+    patched = False
+    for attr in ("text_config", "language_config"):
+        sub = getattr(config, attr, None)
+        if sub is None:
+            continue
+        if isinstance(sub, dict):
+            if sub.get("v_head_dim") == 0:
+                sub["v_head_dim"] = patch
+                patched = True
+        elif getattr(sub, "v_head_dim", None) == 0:
+            sub.v_head_dim = patch
+            patched = True
+    if patched:
+        logger.warning(
+            f"Overriding v_head_dim from 0 to {patch} to avoid potential issues."
+        )
+
+
+# ---------------------------------------------------------------------------
+# Context length / generation config / sparse attention
+# ---------------------------------------------------------------------------
+
+# Models don't use the same configuration key for determining the maximum
+# context length.  Store them here so we can sanely check them.
+# NOTE: The ordering here is important. Some models have two of these and we
+# have a preference for which value gets used.
+CONTEXT_LENGTH_KEYS = [
+    "max_sequence_length",
+    "seq_length",
+    "max_seq_len",
+    "model_max_length",
+    "max_position_embeddings",
+]
+
+
+def get_context_length(config):
+    """Get the context length of a model from a huggingface model configs."""
+    text_config = config
+    rope_scaling = getattr(text_config, "rope_scaling", None)
+    if rope_scaling:
+        rope_scaling_factor = rope_scaling.get("factor", 1)
+        if "original_max_position_embeddings" in rope_scaling:
+            rope_scaling_factor = 1
+        if rope_scaling.get("rope_type", None) == "llama3":
+            rope_scaling_factor = 1
+    else:
+        rope_scaling_factor = 1
+
+    for key in CONTEXT_LENGTH_KEYS:
+        val = getattr(text_config, key, None)
+        if val is not None:
+            return int(rope_scaling_factor * val)
+    return 2048
+
+
+@lru_cache_frozenset(maxsize=32)
+def get_generation_config(
+    model: str,
+    trust_remote_code: bool,
+    revision: Optional[str] = None,
+    **kwargs,
+):
+    if check_gguf_file(model):
+        sidecar = gguf_sidecar_dir(model, "generation_config.json")
+        if sidecar is not None:
+            model = str(sidecar)
+        else:
+            from .gguf_native import (
+                build_gguf_generation_config,
+                has_native_gguf_support,
+            )
+
+            if has_native_gguf_support(model):
+                return build_gguf_generation_config(model)
+
+    try:
+        return GenerationConfig.from_pretrained(
+            model, trust_remote_code=trust_remote_code, revision=revision, **kwargs
+        )
+    except (FileNotFoundError, OSError) as e:
+        # A missing generation_config.json is normal for many checkpoints and
+        # is surfaced by HF as a generic OSError (not FileNotFoundError). Treat
+        # it as benign — proceed without a generation config, at DEBUG level so
+        # normal startup logs stay quiet.
+        logger.debug(
+            "No generation config for %s: %s. Proceeding without it.",
+            model,
+            e,
+        )
+        return None
+
+
+# Qwen-1M related
+def get_sparse_attention_config(
+    model: str,
+    sparse_attention_config_filename: str = "sparse_attention_config.json",
+) -> Dict[str, Any]:
+    is_local = os.path.isdir(model)
+    if not is_local:
+        model = download_from_hf(model, allow_patterns=["*.json"])
+
+    config_file = os.path.join(model, sparse_attention_config_filename)
+    if not os.path.exists(config_file):
+        return {}
+
+    with open(config_file) as f:
+        config = json.load(f)
+    return config
+
+
+# ---------------------------------------------------------------------------
+# Tokenizer / processor helpers
+# ---------------------------------------------------------------------------
+
+
+# Some models don't have an available processor, e.g.: InternVL
+def get_tokenizer_from_processor(processor):
+    from transformers import PreTrainedTokenizerBase
+
+    if isinstance(processor, PreTrainedTokenizerBase):
+        return processor
+    return processor.tokenizer
+
+
+# Turn-final markers that some checkpoints ship without EOS metadata:
+# <|eom_id|> (Llama-3 tool use), <|content_model_end_sampling|> (Inkling,
+# whose bundled tokenizer config leaves eos_token unset), and
+# <|ifm|im_end|> (some K2 Horizon checkpoints, notably 0.9B, name only
+# <|endoftext|> as EOS).
+_ADDITIONAL_STOP_TOKEN_TEXTS = (
+    "<|eom_id|>",
+    "<|content_model_end_sampling|>",
+    "<|ifm|im_end|>",
+)
+
+
+def attach_additional_stop_token_ids(tokenizer):
+    added = tokenizer.get_added_vocab()
+    stop_ids = {added[text] for text in _ADDITIONAL_STOP_TOKEN_TEXTS if text in added}
+    tokenizer.additional_stop_token_ids = stop_ids or None

runtime-v2/patches/model_config_mtp_draft.patch

@@ -0,0 +1,29 @@
+--- src/sglang-pr/python/sglang/srt/configs/model_config.py	2026-10-30 14:12:31.930788808 -0700
++++ overlay/python/sglang/srt/configs/model_config.py	2026-10-30 14:31:57.333705000 -0700
+@@ -1166,6 +1166,26 @@
+             self.hf_config.architectures[0] = "Qwen3MoeForCausalLMMTP"
+             self.hf_config.num_nextn_predict_layers = 1
+ 
++        if (
++            is_draft_model
++            and self.hf_config.architectures[0]
++            == "SenseNovaLoopForConditionalGeneration"
++        ):
++            # MTP patch: the bundled mtp.* head is a plain Qwen3.5 MTP layer (one
++            # full-attention layer, BF16). Shrink the looped execution layout to
++            # that single layer; deep-copy so the target keeps its 56-slot plan.
++            self.hf_config = copy.deepcopy(self.hf_config)
++            self.hf_text_config = get_hf_text_config(self.hf_config)
++            text_config = self.hf_text_config
++            text_config.num_hidden_layers = 1
++            text_config.layer_types = ["full_attention"]
++            text_config.full_attention_interval = 1
++            text_config.execution_plan = (0,)
++            text_config.mhc = None
++            text_config.num_nextn_predict_layers = 1
++            self.hf_config.num_nextn_predict_layers = 1
++            self.hf_config.architectures[0] = "Qwen3_5ForCausalLMMTP"
++
+         if is_draft_model and self.hf_config.architectures[0] in [
+             "Qwen3_5ForConditionalGeneration",
+             "Qwen3_5MoeForConditionalGeneration",

runtime-v2/patches/sensenova_loop_quant.patch

@@ -0,0 +1,22 @@
+--- ../src/sglang-pr/python/sglang/srt/models/sensenova_loop.py	2026-10-30 14:12:32.015788819 -0700
++++ python/sglang/srt/models/sensenova_loop.py	2026-10-30 14:18:45.936764463 -0700
+@@ -121,11 +121,18 @@
+             )
+             if physical_layer_id not in physical_layers:
+                 layer_class = self.decoder_layer_types[layer_type]
++                # NVFP4 patch: (1) pass quant_config so ModelOpt FP4 linears are
++                # built; (2) name modules by PHYSICAL layer id so the checkpoint's
++                # exclude_modules (physical names, e.g. layers.30.linear_attn.in_proj_a)
++                # match layers 28..31, which run at execution ids 52..55.
++                physical_prefix = f"{prefix.rsplit('.', 1)[0]}.{physical_layer_id}"
+                 layer = layer_class(
+                     config=config,
+                     layer_id=execution_layer_id,
++                    quant_config=quant_config,
+                     prefix=add_prefix(
+-                        "linear_attn" if is_linear_attention else "self_attn", prefix
++                        "linear_attn" if is_linear_attention else "self_attn",
++                        physical_prefix,
+                     ),
+                     alt_stream=alt_stream,
+                     build_stages=False,

runtime-v2/patches/v2_mhc_fused_router.patch

@@ -0,0 +1,583 @@
+diff --git a/overlay/python/sglang/srt/layers/sensenova_mhc_fused.py b/overlay/python/sglang/srt/layers/sensenova_mhc_fused.py
+new file mode 100644
+index 0000000..0338c12
+--- /dev/null
++++ b/overlay/python/sglang/srt/layers/sensenova_mhc_fused.py
+@@ -0,0 +1,462 @@
++"""Fused Triton kernels for the SenseNova Loop mHC router (streams == 2).
++
++The reference router (SenseNovaMHCRouter._forward_reference) turns its FP32
++logits [T, 8] into
++
++    pre  = sigmoid(l[:, 0:2])
++    post = 2 * sigmoid(l[:, 2:4])
++    mix  = exp(Sinkhorn^iterations(l[:, 4:8].view(T, 2, 2)))   (log space)
++
++with torch.logsumexp, i.e. ~300 tiny CUDA kernels per call. ``mhc_coeffs``
++does the same in one kernel, replicating the eager op order exactly:
++
++  * torch.logsumexp(x, dim): m = amax(x) (NaN-propagating), m := 0 where
++    |m| == inf, r = log(sum(exp(x - m))) + m.  A 2-element fp32 sum is a
++    single correctly rounded add, so its order cannot differ.
++  * torch.sigmoid (CUDA, fp32): 1 / (1 + exp(-x)) with IEEE division.
++  * exp / log / division use the correctly rounded libdevice routines that
++    nvcc uses for expf / logf / '/' in ATen (no ex2.approx / lg2.approx /
++    div.full), so results are bitwise identical to the eager path.
++
++The iteration count is a constexpr taken from the model config.
++"""
++
++import torch
++import triton
++import triton.language as tl
++from triton.language.extra import libdevice
++
++
++@triton.jit
++def _lse2(x, y):
++    # torch.logsumexp over two elements.
++    m = tl.maximum(x, y, propagate_nan=tl.PropagateNan.ALL)
++    m = tl.where(tl.abs(m) == float("inf"), 0.0, m)
++    s = libdevice.exp(x - m) + libdevice.exp(y - m)
++    return libdevice.log(s) + m
++
++
++@triton.jit
++def _sigmoid(x):
++    # ATen sigmoid_kernel_cuda: one / (one + std::exp(-a)) in opmath (fp32).
++    return libdevice.div_rn(1.0, 1.0 + libdevice.exp(-x))
++
++
++@triton.jit
++def _mhc_coeffs_kernel(
++    logits_ptr,
++    pre_ptr,
++    post_ptr,
++    mix_ptr,
++    T,
++    stride_t,
++    stride_k,
++    ITERS: tl.constexpr,
++    BLOCK_T: tl.constexpr,
++):
++    pid = tl.program_id(0)
++    rows = pid * BLOCK_T + tl.arange(0, BLOCK_T)
++    mask = rows < T
++    base = logits_ptr + rows.to(tl.int64) * stride_t
++    l0 = tl.load(base + 0 * stride_k, mask=mask, other=0.0)
++    l1 = tl.load(base + 1 * stride_k, mask=mask, other=0.0)
++    l2 = tl.load(base + 2 * stride_k, mask=mask, other=0.0)
++    l3 = tl.load(base + 3 * stride_k, mask=mask, other=0.0)
++    # mix[t] = [[a, b], [c, d]]; dim -1 = rows (a,b),(c,d); dim -2 = cols (a,c),(b,d)
++    a = tl.load(base + 4 * stride_k, mask=mask, other=0.0)
++    b = tl.load(base + 5 * stride_k, mask=mask, other=0.0)
++    c = tl.load(base + 6 * stride_k, mask=mask, other=0.0)
++    d = tl.load(base + 7 * stride_k, mask=mask, other=0.0)
++
++    for _ in tl.static_range(ITERS):
++        r0 = _lse2(a, b)
++        r1 = _lse2(c, d)
++        a = a - r0
++        b = b - r0
++        c = c - r1
++        d = d - r1
++        c0 = _lse2(a, c)
++        c1 = _lse2(b, d)
++        a = a - c0
++        c = c - c0
++        b = b - c1
++        d = d - c1
++
++    out2 = rows.to(tl.int64) * 2
++    tl.store(pre_ptr + out2, _sigmoid(l0), mask=mask)
++    tl.store(pre_ptr + out2 + 1, _sigmoid(l1), mask=mask)
++    tl.store(post_ptr + out2, 2.0 * _sigmoid(l2), mask=mask)
++    tl.store(post_ptr + out2 + 1, 2.0 * _sigmoid(l3), mask=mask)
++    out4 = rows.to(tl.int64) * 4
++    tl.store(mix_ptr + out4 + 0, libdevice.exp(a), mask=mask)
++    tl.store(mix_ptr + out4 + 1, libdevice.exp(b), mask=mask)
++    tl.store(mix_ptr + out4 + 2, libdevice.exp(c), mask=mask)
++    tl.store(mix_ptr + out4 + 3, libdevice.exp(d), mask=mask)
++
++
++def supported(streams: int, fixed_identity: bool) -> bool:
++    return streams == 2 and not fixed_identity
++
++
++def mhc_coeffs(logits: torch.Tensor, iterations: int):
++    """FP32 logits [..., 8] -> (pre [..., 2], post [..., 2], mix [..., 2, 2]),
++    all FP32 and contiguous, bitwise equal to the reference router."""
++    assert logits.dtype == torch.float32 and logits.shape[-1] == 8, logits.shape
++    lead = logits.shape[:-1]
++    flat = logits if logits.dim() == 2 else logits.reshape(-1, 8)
++    T = flat.shape[0]
++    pre = torch.empty((*lead, 2), device=logits.device, dtype=torch.float32)
++    post = torch.empty((*lead, 2), device=logits.device, dtype=torch.float32)
++    mix = torch.empty((*lead, 2, 2), device=logits.device, dtype=torch.float32)
++    if T == 0:
++        return pre, post, mix
++    BLOCK_T = 128 if T >= 128 else max(32, triton.next_power_of_2(T))
++    grid = (triton.cdiv(T, BLOCK_T),)
++    _mhc_coeffs_kernel[grid](
++        flat,
++        pre,
++        post,
++        mix,
++        T,
++        flat.stride(0),
++        flat.stride(1),
++        ITERS=int(iterations),
++        BLOCK_T=BLOCK_T,
++        num_warps=max(1, min(4, BLOCK_T // 32)),
++        # ATen keeps denormals; Triton's default links libdevice in FTZ mode,
++        # which would flush exp() / sigmoid() results below 2**-126 to zero.
++        enable_reflect_ftz=False,
++    )
++    return pre, post, mix
++
++
++# ---------------------------------------------------------------------------
++# Router tail + pre-branch in one kernel, and the merge in one kernel.
++#
++# bf16 bmm with K == 2 (cuBLAS, fp32 accumulate): both bf16 x bf16 products are
++# exact in fp32, so the result is round_bf16(round_fp32(p0 + p1)) whatever the
++# accumulation order or FMA use. Elementwise bf16 ops in ATen compute in fp32
++# and round once to bf16. The kernels below reproduce exactly that.
++# ---------------------------------------------------------------------------
++
++
++@triton.jit
++def _bf16(x):
++    # torch .to(torch.bfloat16) (round to nearest even), read back as fp32.
++    return x.to(tl.bfloat16).to(tl.float32)
++
++
++@triton.jit
++def _mhc_pre_kernel(
++    logits_ptr,
++    hs_ptr,
++    branch_ptr,
++    pre_ptr,
++    post_ptr,
++    mix_ptr,
++    H,
++    stride_lt,
++    stride_lk,
++    stride_ht,
++    stride_bt,
++    ITERS: tl.constexpr,
++    BLOCK_H: tl.constexpr,
++):
++    t = tl.program_id(0).to(tl.int64)
++    base = logits_ptr + t * stride_lt
++    l0 = tl.load(base + 0 * stride_lk)
++    l1 = tl.load(base + 1 * stride_lk)
++    l2 = tl.load(base + 2 * stride_lk)
++    l3 = tl.load(base + 3 * stride_lk)
++    a = tl.load(base + 4 * stride_lk)
++    b = tl.load(base + 5 * stride_lk)
++    c = tl.load(base + 6 * stride_lk)
++    d = tl.load(base + 7 * stride_lk)
++    for _ in tl.static_range(ITERS):
++        r0 = _lse2(a, b)
++        r1 = _lse2(c, d)
++        a = a - r0
++        b = b - r0
++        c = c - r1
++        d = d - r1
++        c0 = _lse2(a, c)
++        c1 = _lse2(b, d)
++        a = a - c0
++        c = c - c0
++        b = b - c1
++        d = d - c1
++    p0 = _sigmoid(l0)
++    p1 = _sigmoid(l1)
++    tl.store(pre_ptr + t * 2, p0)
++    tl.store(pre_ptr + t * 2 + 1, p1)
++    tl.store(post_ptr + t * 2, 2.0 * _sigmoid(l2))
++    tl.store(post_ptr + t * 2 + 1, 2.0 * _sigmoid(l3))
++    tl.store(mix_ptr + t * 4 + 0, libdevice.exp(a))
++    tl.store(mix_ptr + t * 4 + 1, libdevice.exp(b))
++    tl.store(mix_ptr + t * 4 + 2, libdevice.exp(c))
++    tl.store(mix_ptr + t * 4 + 3, libdevice.exp(d))
++
++    # branch = bmm(pre.to(bf16)[t, None, :], hs[t].view(2, H))
++    w0 = _bf16(p0)
++    w1 = _bf16(p1)
++    offs = tl.arange(0, BLOCK_H)
++    row = hs_ptr + t * stride_ht
++    out = branch_ptr + t * stride_bt
++    for h in range(0, H, BLOCK_H):
++        m = h + offs < H
++        x0 = tl.load(row + h + offs, mask=m, other=0.0).to(tl.float32)
++        x1 = tl.load(row + H + h + offs, mask=m, other=0.0).to(tl.float32)
++        tl.store(out + h + offs, (w0 * x0 + w1 * x1).to(tl.bfloat16), mask=m)
++
++
++@triton.jit
++def _mhc_merge_kernel(
++    branch_ptr,
++    res_ptr,
++    mix_ptr,
++    post_ptr,
++    out_ptr,
++    H,
++    stride_bt,
++    stride_rt,
++    stride_ot,
++    stride_mt,
++    stride_mi,
++    stride_mj,
++    stride_pt,
++    stride_pk,
++    BLOCK_H: tl.constexpr,
++):
++    t = tl.program_id(0).to(tl.int64)
++    hb = tl.program_id(1)
++    mb = mix_ptr + t * stride_mt
++    m00 = _bf16(tl.load(mb))
++    m01 = _bf16(tl.load(mb + stride_mj))
++    m10 = _bf16(tl.load(mb + stride_mi))
++    m11 = _bf16(tl.load(mb + stride_mi + stride_mj))
++    q0 = _bf16(tl.load(post_ptr + t * stride_pt))
++    q1 = _bf16(tl.load(post_ptr + t * stride_pt + stride_pk))
++    offs = hb * BLOCK_H + tl.arange(0, BLOCK_H)
++    m = offs < H
++    row = res_ptr + t * stride_rt
++    r0 = tl.load(row + offs, mask=m, other=0.0).to(tl.float32)
++    r1 = tl.load(row + H + offs, mask=m, other=0.0).to(tl.float32)
++    br = tl.load(branch_ptr + t * stride_bt + offs, mask=m, other=0.0).to(tl.float32)
++    # mixed = bmm(mix.bf16, residual); update = post.bf16[:, :, None] * branch[:, None]
++    o0 = _bf16(m00 * r0 + m01 * r1) + _bf16(q0 * br)
++    o1 = _bf16(m10 * r0 + m11 * r1) + _bf16(q1 * br)
++    out = out_ptr + t * stride_ot
++    tl.store(out + offs, o0.to(tl.bfloat16), mask=m)
++    tl.store(out + H + offs, o1.to(tl.bfloat16), mask=m)
++
++
++def mhc_pre(logits: torch.Tensor, hidden_states: torch.Tensor, iterations: int):
++    """Router tail for streams == 2: FP32 logits [T, 8] and BF16 hidden_states
++    [T, 2H] -> (branch [T, H] bf16, pre [T, 2], post [T, 2], mix [T, 2, 2])."""
++    assert logits.dim() == 2 and logits.shape[1] == 8 and logits.dtype == torch.float32
++    assert hidden_states.dim() == 2 and hidden_states.dtype == torch.bfloat16
++    T, H2 = hidden_states.shape
++    H = H2 // 2
++    if hidden_states.stride(1) != 1:
++        hidden_states = hidden_states.contiguous()
++    dev = logits.device
++    branch = torch.empty((T, H), device=dev, dtype=hidden_states.dtype)
++    pre = torch.empty((T, 2), device=dev, dtype=torch.float32)
++    post = torch.empty((T, 2), device=dev, dtype=torch.float32)
++    mix = torch.empty((T, 2, 2), device=dev, dtype=torch.float32)
++    if T == 0:
++        return branch, pre, post, mix
++    _mhc_pre_kernel[(T,)](
++        logits,
++        hidden_states,
++        branch,
++        pre,
++        post,
++        mix,
++        H,
++        logits.stride(0),
++        logits.stride(1),
++        hidden_states.stride(0),
++        branch.stride(0),
++        ITERS=int(iterations),
++        BLOCK_H=1024,
++        num_warps=4,
++        enable_reflect_ftz=False,
++    )
++    return branch, pre, post, mix
++
++
++def mhc_merge(branch, residual, mix, post):
++    """SenseNovaMHCRouter.merge for streams == 2 in one kernel (bitwise equal)."""
++    T, H2 = residual.shape
++    H = H2 // 2
++    if residual.stride(1) != 1:
++        residual = residual.contiguous()
++    if branch.stride(-1) != 1:
++        branch = branch.contiguous()
++    out = torch.empty((T, H2), device=residual.device, dtype=residual.dtype)
++    if T == 0:
++        return out
++    BLOCK_H = 1024
++    _mhc_merge_kernel[(T, triton.cdiv(H, BLOCK_H))](
++        branch,
++        residual,
++        mix,
++        post,
++        out,
++        H,
++        branch.stride(0),
++        residual.stride(0),
++        out.stride(0),
++        mix.stride(0),
++        mix.stride(1),
++        mix.stride(2),
++        post.stride(0),
++        post.stride(1),
++        BLOCK_H=BLOCK_H,
++        num_warps=4,
++        enable_reflect_ftz=False,
++        # Otherwise the bf16-rounded update product is contracted into an fma
++        # with the following add and its rounding is lost (not bitwise).
++        enable_fp_fusion=False,
++    )
++    return out
++
++
++# ---------------------------------------------------------------------------
++# Whole router forward in one kernel: FP32 RMS-normalize + 8 FP32 dot products
++# + bias + sigmoid / Sinkhorn + pre-branch. One program per token row.
++#
++# NOT bitwise equal to the eager path: the sum of squares and the 8192-long dot
++# products are FP32 but summed in a different order than torch.mean / cuBLAS
++# (which itself changes order with M). Everything after the logits is exact.
++# ---------------------------------------------------------------------------
++
++
++@triton.jit
++def _pick(v, idx, j: tl.constexpr):
++    return tl.sum(tl.where(idx == j, v, 0.0), axis=0)
++
++
++@triton.jit
++def _mhc_router_kernel(
++    hs_ptr,
++    w_ptr,
++    bias_ptr,
++    branch_ptr,
++    pre_ptr,
++    post_ptr,
++    mix_ptr,
++    stride_ht,
++    stride_bt,
++    H: tl.constexpr,
++    EPS: tl.constexpr,
++    ITERS: tl.constexpr,
++    BLOCK_K: tl.constexpr,
++):
++    K: tl.constexpr = 2 * H
++    t = tl.program_id(0).to(tl.int64)
++    row = hs_ptr + t * stride_ht
++    offs = tl.arange(0, BLOCK_K)
++
++    # mean(x^2): squares of bf16 values are exact in fp32.
++    sq = tl.zeros([BLOCK_K], dtype=tl.float32)
++    for k in range(0, K, BLOCK_K):
++        x = tl.load(row + k + offs).to(tl.float32)
++        sq += x * x
++    ms = libdevice.div_rn(tl.sum(sq, axis=0), K * 1.0)
++    r = libdevice.rsqrt(ms + EPS)
++
++    # logits[j] = sum_k (x_k * r) * W[j, k] + b[j]   (normalized rounded as in eager)
++    jj = tl.arange(0, 8)
++    acc = tl.zeros([8, BLOCK_K], dtype=tl.float32)
++    for k in range(0, K, BLOCK_K):
++        n = tl.load(row + k + offs).to(tl.float32) * r
++        w = tl.load(w_ptr + jj[:, None] * K + k + offs[None, :])
++        acc += n[None, :] * w
++    logits = tl.sum(acc, axis=1) + tl.load(bias_ptr + jj)
++    l0 = _pick(logits, jj, 0)
++    l1 = _pick(logits, jj, 1)
++    l2 = _pick(logits, jj, 2)
++    l3 = _pick(logits, jj, 3)
++    a = _pick(logits, jj, 4)
++    b = _pick(logits, jj, 5)
++    c = _pick(logits, jj, 6)
++    d = _pick(logits, jj, 7)
++
++    for _ in tl.static_range(ITERS):
++        r0 = _lse2(a, b)
++        r1 = _lse2(c, d)
++        a = a - r0
++        b = b - r0
++        c = c - r1
++        d = d - r1
++        c0 = _lse2(a, c)
++        c1 = _lse2(b, d)
++        a = a - c0
++        c = c - c0
++        b = b - c1
++        d = d - c1
++    p0 = _sigmoid(l0)
++    p1 = _sigmoid(l1)
++    tl.store(pre_ptr + t * 2, p0)
++    tl.store(pre_ptr + t * 2 + 1, p1)
++    tl.store(post_ptr + t * 2, 2.0 * _sigmoid(l2))
++    tl.store(post_ptr + t * 2 + 1, 2.0 * _sigmoid(l3))
++    tl.store(mix_ptr + t * 4 + 0, libdevice.exp(a))
++    tl.store(mix_ptr + t * 4 + 1, libdevice.exp(b))
++    tl.store(mix_ptr + t * 4 + 2, libdevice.exp(c))
++    tl.store(mix_ptr + t * 4 + 3, libdevice.exp(d))
++
++    w0 = _bf16(p0)
++    w1 = _bf16(p1)
++    out = branch_ptr + t * stride_bt
++    for h in range(0, H, BLOCK_K):
++        x0 = tl.load(row + h + offs).to(tl.float32)
++        x1 = tl.load(row + H + h + offs).to(tl.float32)
++        tl.store(out + h + offs, (w0 * x0 + w1 * x1).to(tl.bfloat16))
++
++
++def mhc_router(
++    hidden_states: torch.Tensor,
++    weight: torch.Tensor,
++    bias: torch.Tensor,
++    iterations: int,
++    eps: float = 1e-6,
++):
++    """Whole streams == 2 router forward: BF16 hidden_states [T, 2H] ->
++    (branch [T, H] bf16, pre [T, 2], post [T, 2], mix [T, 2, 2])."""
++    T, K = hidden_states.shape
++    H = K // 2
++    assert weight.shape == (8, K) and weight.dtype == torch.float32
++    assert hidden_states.dtype == torch.bfloat16
++    BLOCK_K = 1024
++    assert H % BLOCK_K == 0, H
++    if hidden_states.stride(1) != 1:
++        hidden_states = hidden_states.contiguous()
++    weight = weight.contiguous()
++    dev = hidden_states.device
++    branch = torch.empty((T, H), device=dev, dtype=hidden_states.dtype)
++    pre = torch.empty((T, 2), device=dev, dtype=torch.float32)
++    post = torch.empty((T, 2), device=dev, dtype=torch.float32)
++    mix = torch.empty((T, 2, 2), device=dev, dtype=torch.float32)
++    if T == 0:
++        return branch, pre, post, mix
++    _mhc_router_kernel[(T,)](
++        hidden_states,
++        weight,
++        bias,
++        branch,
++        pre,
++        post,
++        mix,
++        hidden_states.stride(0),
++        branch.stride(0),
++        H=H,
++        EPS=eps,
++        ITERS=int(iterations),
++        BLOCK_K=BLOCK_K,
++        num_warps=8,
++        enable_reflect_ftz=False,
++    )
++    return branch, pre, post, mix
+diff --git a/overlay/python/sglang/srt/models/sensenova_loop.py b/overlay/python/sglang/srt/models/sensenova_loop.py
+index e813eb2..786e05a 100644
+--- a/overlay/python/sglang/srt/models/sensenova_loop.py
++++ b/overlay/python/sglang/srt/models/sensenova_loop.py
+@@ -8,6 +8,7 @@
+ """Shared-weight SenseNova Loop with independent attention state per visit."""
+ 
+ import copy
++import os
+ from collections.abc import Iterable
+ 
+ import torch
+@@ -27,12 +28,24 @@ from sglang.srt.layers.layer_boundary import (
+ )
+ from sglang.srt.layers.layer_boundary.residual import batch as residual_batch
+ from sglang.srt.layers.layernorm import GemmaRMSNorm
++from sglang.srt.layers import sensenova_mhc_fused
+ from sglang.srt.layers.quantization.base_config import QuantizationConfig
+ from sglang.srt.model_executor.forward_batch_info import ForwardBatch, PPProxyTensors
+ from sglang.srt.models import qwen3_5
+ from sglang.srt.runtime_context import get_parallel, get_stream
+ from sglang.srt.utils import add_prefix, make_pp_layers
+ 
++_MHC_FUSED = os.environ.get("SENSENOVA_MHC_FUSED", "1") != "0"
++_MHC_FUSED_LINEAR = os.environ.get("SENSENOVA_MHC_FUSED_LINEAR", "1") != "0"
++# The single-kernel router beat torch-norm + cuBLAS at every T measured (1..4096).
++_MHC_FUSED_LINEAR_MAX_T = int(
++    os.environ.get("SENSENOVA_MHC_FUSED_LINEAR_MAX_T", str(1 << 62))
++)
++# Fused merge rounds the K=2 bf16 bmm exactly once (IEEE RN); bitwise equal to
++# torch for T <= 32, ~1e-6 of elements differ by 1-2 bf16 ulp at T >= 64 where
++# cuBLAS picks tensor-core kernels with non-RN fp32 accumulation.
++_MHC_FUSED_MERGE = _MHC_FUSED and os.environ.get("SENSENOVA_MHC_FUSED_MERGE", "1") != "0"
++
+ 
+ class SenseNovaMHCRouter(nn.Module):
+     """Canonical FP32 router with log-space Sinkhorn and BF16 residual math."""
+@@ -50,6 +63,15 @@ class SenseNovaMHCRouter(nn.Module):
+             torch.empty(output_size, self.streams * hidden_size, dtype=torch.float32)
+         )
+         self.bias = nn.Parameter(torch.empty(output_size, dtype=torch.float32))
++        # SENSENOVA_MHC_FUSED=1 (default): Triton router kernels; 0: eager reference.
++        self.use_fused = _MHC_FUSED and sensenova_mhc_fused.supported(
++            self.streams, self.fixed_identity
++        )
++        # SENSENOVA_MHC_FUSED_LINEAR=1 (default): for T <= ..._MAX_T the whole
++        # forward (FP32 RMS-normalize + FP32 linear + coefficients + branch) is
++        # one kernel; FP32 but not cuBLAS's summation order, so not bitwise.
++        self.use_fused_linear = self.use_fused and _MHC_FUSED_LINEAR
++        self.fused_linear_max_t = _MHC_FUSED_LINEAR_MAX_T
+ 
+     def forward(
+         self,
+@@ -57,6 +79,42 @@ class SenseNovaMHCRouter(nn.Module):
+         out_norm_weight: torch.Tensor | None = None,
+         out_norm_eps: float | None = None,
+     ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, bool]:
++        if self.use_fused:
++            return self._forward_fused(hidden_states, out_norm_weight, out_norm_eps)
++        return self._forward_reference(hidden_states, out_norm_weight, out_norm_eps)
++
++    def _forward_fused(
++        self,
++        hidden_states: torch.Tensor,
++        out_norm_weight: torch.Tensor | None = None,
++        out_norm_eps: float | None = None,
++    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, bool]:
++        """Same math and dtype boundaries as _forward_reference."""
++        if self.use_fused_linear and hidden_states.shape[0] <= self.fused_linear_max_t:
++            branch, pre, post, mix = sensenova_mhc_fused.mhc_router(
++                hidden_states, self.weight, self.bias, self.iterations
++            )
++        else:
++            flat = hidden_states.float()
++            normalized = flat * torch.rsqrt(
++                flat.square().mean(-1, keepdim=True) + 1e-6
++            )
++            logits = F.linear(normalized, self.weight, self.bias)
++            # Bitwise equal to the eager sigmoid / Sinkhorn / exp / bf16 bmm.
++            branch, pre, post, mix = sensenova_mhc_fused.mhc_pre(
++                logits, hidden_states, self.iterations
++            )
++        if self.pre_scale != 1.0:
++            branch = branch * self.pre_scale
++        return branch, mix, post, False
++
++    def _forward_reference(
++        self,
++        hidden_states: torch.Tensor,
++        out_norm_weight: torch.Tensor | None = None,
++        out_norm_eps: float | None = None,
++    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, bool]:
++        """Original eager implementation (SENSENOVA_MHC_FUSED=0), kept verbatim."""
+         residual = hidden_states.unflatten(-1, (self.streams, -1))
+         flat = hidden_states.float()
+         normalized = flat * torch.rsqrt(flat.square().mean(-1, keepdim=True) + 1e-6)
+@@ -86,6 +144,18 @@ class SenseNovaMHCRouter(nn.Module):
+         residual: torch.Tensor,
+         mix: torch.Tensor,
+         post: torch.Tensor,
++    ) -> torch.Tensor:
++        if _MHC_FUSED_MERGE and post.shape[-1] == 2 and residual.dim() == 2:
++            # One kernel (see _MHC_FUSED_MERGE for exactness).
++            return sensenova_mhc_fused.mhc_merge(branch, residual, mix, post)
++        return SenseNovaMHCRouter._merge_reference(branch, residual, mix, post)
++
++    @staticmethod
++    def _merge_reference(
++        branch: torch.Tensor,
++        residual: torch.Tensor,
++        mix: torch.Tensor,
++        post: torch.Tensor,
+     ) -> torch.Tensor:
+         streams = post.shape[-1]
+         residual = residual.unflatten(-1, (streams, -1))

runtime-v2/runs/chain1.log

@@ -0,0 +1,18 @@
+ready after 80s
+exact_tm64k_cutlass greedy identical 10/12
+exact_tm64k_cutlass synthetic c 1 agg 26.328 req 29.233 ttft 469.897 acc 1.957
+exact_tm64k_cutlass synthetic c 4 agg 80.849 req 22.721 ttft 1116.713 acc 1.997
+exact_tm64k_cutlass code c 1 agg 33.249 req 33.467 ttft 159.326 acc 2.349
+exact_tm64k_cutlass code c 4 agg 108.574 req 28.349 ttft 232.512 acc 2.35
+ready after 70s
+full_tm64k greedy identical 2/12
+full_tm64k synthetic c 1 agg 30.963 req 31.389 ttft 320.257 acc 2.086
+full_tm64k synthetic c 4 agg 90.598 req 25.263 ttft 724.477 acc 1.968
+full_tm64k code c 1 agg 37.483 req 37.946 ttft 109.651 acc 2.456
+full_tm64k code c 4 agg 117.494 req 31.783 ttft 243.143 acc 2.371
+ready after 160s
+exact_tm64k_marlin greedy identical 3/12
+exact_tm64k_marlin synthetic c 1 agg 27.942 req 30.266 ttft 659.059 acc 2.057
+exact_tm64k_marlin synthetic c 4 agg 77.439 req 23.136 ttft 2213.987 acc 2.086
+exact_tm64k_marlin code c 1 agg 33.833 req 34.436 ttft 159.247 acc 2.351
+exact_tm64k_marlin code c 4 agg 107.749 req 28.065 ttft 326.778 acc 2.329

runtime-v2/runs/chain2.log

@@ -0,0 +1,10 @@
+ready after 70s
+full_mtp synthetic c 8 agg 130.2 req 19.7 ttft 1293 acc 1.9536215068338272
+full_mtp synthetic c 16 agg 171.4 req 14.9 ttft 5097 acc 1.8842565959781015
+full_mtp code c 8 agg 198.4 req 26.1 ttft 351 acc 2.402667027337142
+full_mtp code c 16 agg 295.0 req 20.0 ttft 425 acc 2.3972806387936223
+ready after 65s
+full_nomtp synthetic c 8 agg 129.4 req 18.6 ttft 2080 acc None
+full_nomtp synthetic c 16 agg 194.3 req 15.6 ttft 4685 acc None
+full_nomtp code c 8 agg 150.6 req 19.2 ttft 320 acc None
+full_nomtp code c 16 agg 258.0 req 16.6 ttft 466 acc None

runtime-v2/runs/chain3.log

@@ -0,0 +1,12 @@
+ifeval     n= 60 acc=0.750 CI95=[0.628,0.842] err=0 trunc=3 mean_tok=245.3 94.7s
+gsm8k      n= 80 acc=0.975 CI95=[0.913,0.993] err=0 trunc=0 mean_tok=185.025 52.1s
+mmlu_pro   n= 60 acc=0.683 CI95=[0.558,0.787] err=0 trunc=6 mean_tok=558.7 126.9s
+humaneval  n= 40 acc=0.800 CI95=[0.652,0.895] err=0 trunc=0 mean_tok=193.5 27.7s
+tools      n= 20 acc=0.900 CI95=[0.699,0.972] err=0 trunc=0 mean_tok=61.6 10.7s
+vqa        n= 40 acc=0.875 CI95=[0.739,0.945] err=0 trunc=0 mean_tok=4.15 22.8s
+v2_exact eval done
+v2_fast rep1 mmlu_pro   n= 60 acc=0.717 CI95=[0.592,0.815] err=0 trunc=10 mean_tok=601.8833333333333 119.8s
+v2_fast rep2 mmlu_pro   n= 60 acc=0.650 CI95=[0.524,0.758] err=0 trunc=9 mean_tok=599.4333333333333 119.6s
+frozen rep1 mmlu_pro   n= 60 acc=0.700 CI95=[0.575,0.801] err=0 trunc=7 mean_tok=571.8 227.6s
+frozen rep2 mmlu_pro   n= 60 acc=0.650 CI95=[0.524,0.758] err=0 trunc=10 mean_tok=594.25 214.7s
+chain3 done

runtime-v2/runs/final_chain.log

@@ -0,0 +1,11 @@
+frozen startup: ready after 91s
+frozen launch->ready 92s
+done runs/final_frozen_v1
+frozen done
+v2fast startup: ready after 75s
+v2fast launch->ready 77s
+done runs/final_v2_fast
+v2_fast done
+v2exact startup: ready after 76s
+done runs/final_v2_exact
+v2_exact done

runtime-v2/runs/final_frozen_v1/eval.log

@@ -0,0 +1,7 @@
+ifeval     n= 60 acc=0.767 CI95=[0.646,0.856] err=0 trunc=4 mean_tok=268.68333333333334 143.2s
+gsm8k      n= 80 acc=0.950 CI95=[0.878,0.980] err=0 trunc=1 mean_tok=187.55 95.8s
+mmlu_pro   n= 60 acc=0.700 CI95=[0.575,0.801] err=0 trunc=9 mean_tok=577.3333333333334 209.0s
+humaneval  n= 40 acc=0.825 CI95=[0.681,0.913] err=0 trunc=0 mean_tok=208.9 60.0s
+tools      n= 20 acc=0.900 CI95=[0.699,0.972] err=0 trunc=0 mean_tok=60.2 16.7s
+vqa        n= 40 acc=0.900 CI95=[0.769,0.960] err=0 trunc=0 mean_tok=4.075 24.5s
+wrote runs/final_frozen_v1/eval/summary.json

runtime-v2/runs/final_frozen_v1/eval/compare_vs_ref.json

@@ -0,0 +1,78 @@
+{
+  "ref": "../eval/runs/sglang_nvfp4_v1_mtp",
+  "cand": "runs/final_frozen_v1/eval",
+  "tasks": {
+    "gsm8k": {
+      "n": 80,
+      "ref_acc": 0.9625,
+      "cand_acc": 0.95,
+      "delta": -0.0125,
+      "delta_boot95": [
+        -0.0375,
+        0.0
+      ],
+      "ref_only_correct": 1,
+      "cand_only_correct": 0,
+      "mcnemar_p": 1.0,
+      "ref_agreement": 0.9875,
+      "identical_output": 0.1125,
+      "mean_tokens_ref": 190.4,
+      "mean_tokens_cand": 187.6,
+      "truncated_ref": 1,
+      "truncated_cand": 1,
+      "errors_cand": 0
+    },
+    "humaneval": {
+      "n": 40,
+      "ref_acc": 0.8,
+      "cand_acc": 0.825,
+      "delta": 0.025,
+      "delta_boot95": [
+        -0.05,
+        0.1
+      ],
+      "ref_only_correct": 1,
+      "cand_only_correct": 2,
+      "mcnemar_p": 1.0,
+      "ref_agreement": 0.5,
+      "identical_output": 0.5,
+      "mean_tokens_ref": 191.3,
+      "mean_tokens_cand": 208.9,
+      "truncated_ref": 0,
+      "truncated_cand": 0,
+      "errors_cand": 0
+    },
+    "tools": {
+      "n": 20,
+      "ref_acc": 0.9,
+      "cand_acc": 0.9,
+      "delta": 0.0,
+      "delta_boot95": [
+        0.0,
+        0.0
+      ],
+      "ref_only_correct": 0,
+      "cand_only_correct": 0,
+      "mcnemar_p": 1.0,
+      "ref_agreement": 1.0,
+      "identical_output": 0.25,
+      "mean_tokens_ref": 60.2,
+      "mean_tokens_cand": 60.2,
+      "truncated_ref": 0,
+      "truncated_cand": 0,
+      "errors_cand": 0
+    }
+  },
+  "overall_micro": {
+    "n": 140,
+    "ref_acc": 0.9071428571428571,
+    "cand_acc": 0.9071428571428571,
+    "delta": 0.0,
+    "delta_boot95": [
+      -0.02857142857142857,
+      0.02857142857142857
+    ],
+    "ref_agreement": 0.85,
+    "identical_output": 0.24285714285714285
+  }
+}

runtime-v2/runs/final_frozen_v1/eval/raw_gsm8k.jsonl

@@ -0,0 +1,80 @@
+{"id": "gsm8k_test_48", "task": "gsm8k", "prompt_sha256": "36714d4330abecb7", "content": "To solve this problem, let's break it down step by step.\n\n1. **Convert the length of the wire to inches:**\n   - The wire is 4 feet long.\n   - Since 1 foot = 12 inches, we convert the length to inches:\n     \\[\n     4 \\text{ feet} \\times 12 \\text{ inches/foot} = 48 \\text{ inches}\n     \\]\n\n2. **Determine how many 6-inch pieces can be cut from the wire:**\n   - Each piece is 6 inches long.\n   - To find the number of pieces, divide the total length of the wire by the length of each piece:\n     \\[\n     \\frac{48 \\text{ inches}}{6 \\text{ inches/piece}} = 8 \\text{ pieces}\n     \\]\n\n**Final Answer:**\nThe final answer is 8.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 83, "total_tokens": 284, "completion_tokens": 201, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 201, "latency_s": 17.5, "error": null, "agree_key": "8", "pred": "8", "correct": true}
+{"id": "gsm8k_test_85", "task": "gsm8k", "prompt_sha256": "a7d140ba3629be55", "content": "Let me break this problem down step by step.\n\n**Step 1: Determine how many years Gene has been vacationing.**\n- He started vacationing at age 23\n- He is now 34\n- Number of years = 34 - 23 = 11 years\n\n**Step 2: Determine how many vacations he goes on per year.**\n- He goes on 4 vacations per year\n\n**Step 3: Calculate the total number of quilt blocks.**\n- Each vacation = 1 shirt = 1 quilt block\n- Total quilt blocks = 11 years × 4 vacations per year = 44\n\nThe final answer is 44.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 129, "total_tokens": 275, "completion_tokens": 146, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 146, "latency_s": 12.938, "error": null, "agree_key": "44", "pred": "44", "correct": true}
+{"id": "gsm8k_test_127", "task": "gsm8k", "prompt_sha256": "23e1fd9aa22b1ff7", "content": "Let me solve this step by step.\n\n**Given information:**\n- Lily made 50 friends\n- Amy made 20 more friends than Lily\n\n**Step 1: Find how many friends Amy made.**\nAmy made 20 more friends than Lily.\nAmy's friends = Lily's friends + 20\nAmy's friends = 50 + 20 = 70\n\n**Step 2: Find the total number of friends Lily and Amy have together.**\nTotal = Lily's friends + Amy's friends\nTotal = 50 + 70 = 120\n\nThe final answer is 120", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 81, "total_tokens": 218, "completion_tokens": 137, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 137, "latency_s": 11.856, "error": null, "agree_key": "120", "pred": "120", "correct": true}
+{"id": "gsm8k_test_195", "task": "gsm8k", "prompt_sha256": "e1dbcf820b17542a", "content": "Step 1: Find the cost of a pencil and eraser combined.\n- Pencil = $1.20\n- Eraser = $0.30\n- Combined = $1.20 + $0.30 = $1.50\n\nStep 2: Determine the cost of one pen.\n- A pen costs as much as a pencil and eraser combined\n- So, one pen = $1.50\n\nStep 3: Calculate the cost of 8 pens.\n- 8 pens × $1.50 = $12.00\n\nThe final answer is 12.00", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 86, "total_tokens": 223, "completion_tokens": 137, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 137, "latency_s": 10.152, "error": null, "agree_key": "12", "pred": "12", "correct": true}
+{"id": "gsm8k_test_199", "task": "gsm8k", "prompt_sha256": "868918576db22bb7", "content": "Let me work through this step by step.\n\n**Given information:**\n- Mark edits an equal number of sentences for each publisher\n- Total sentences per week: 1000\n- Publisher A pays 5 cents per sentence\n- Publisher B pays twice what Publisher A pays\n\n**Step 1: Find how many sentences Mark edits for each publisher.**\nSince he edits an equal number for each publisher:\n- Sentences for Publisher A = 1000 ÷ 2 = 500 sentences\n- Sentences for Publisher B = 1000 ÷ 2 = 500 sentences\n\n**Step 2: Find Publisher B's rate per sentence.**\nPublisher B pays twice what Publisher A pays:\n- Publisher B's rate = 2 × 5 cents = 10 cents per sentence\n\n**Step 3: Calculate earnings from each publisher.**\n- Earnings from Publisher A = 500 × 5 = 2500 cents\n- Earnings from Publisher B = 500 × 10 = 5000 cents\n\n**Step 4: Calculate total weekly earnings.**\nTotal = 2500 + 5000 = 7500 cents\n\nThe final answer is 7500", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 124, "total_tokens": 399, "completion_tokens": 275, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 275, "latency_s": 23.547, "error": null, "agree_key": "7500", "pred": "7500", "correct": true}
+{"id": "gsm8k_test_209", "task": "gsm8k", "prompt_sha256": "f5965010ae837d62", "content": "Let me solve this step by step.\n\n**Given information:**\n- Twenty dozen cups cost $1200 less than the total cost of half a dozen plates sold at $6000 each\n- I need to find the cost of each cup\n\n**Step 1: Find the total cost of half a dozen plates.**\n\nHalf a dozen = 6 plates\nEach plate costs $6000\nTotal cost of plates = 6 × $6000 = $36,000\n\n**Step 2: Find the total cost of twenty dozen cups.**\n\nTwenty dozen cups cost $1200 less than the total cost of the plates.\nTotal cost of cups = $36,000 - $1,200 = $34,800\n\n**Step 3: Find the cost of each cup.**\n\nTwenty dozen cups = 20 × 12 = 240 cups\nCost per cup = $34,800 ÷ 240 = $145\n\nThe final answer is 145", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 83, "total_tokens": 317, "completion_tokens": 234, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 234, "latency_s": 21.669, "error": null, "agree_key": "145", "pred": "145", "correct": true}
+{"id": "gsm8k_test_217", "task": "gsm8k", "prompt_sha256": "6ae06a1253cf40b5", "content": "Step 1: Find the number of roses in the vase.\nThere are 4 roses in the vase.\n\nStep 2: Find the number of dahlias in the vase.\nThere are 7 more dahlias than roses, so:\n7 + 4 = 11 dahlias\n\nStep 3: Find the total number of flowers.\nTotal flowers = roses + dahlias\nTotal flowers = 4 + 11 = 15\n\nThe final answer is 15", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 79, "total_tokens": 184, "completion_tokens": 105, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 105, "latency_s": 8.046, "error": null, "agree_key": "15", "pred": "15", "correct": true}
+{"id": "gsm8k_test_295", "task": "gsm8k", "prompt_sha256": "44ebc745086512e1", "content": "Let me solve this step by step.\n\n**Day 1:**\n- Pushups: 100\n- Squats: 50\n- Dumbbell presses: 20\n\n**Day 2:**\n- Pushups: 100 + 20 = 120\n- Squats: 50 - 10 = 40\n- Dumbbell presses: 20 × 2 = 40\n\n**Total for both days:**\n- Pushups: 100 + 120 = 220\n- Squats: 50 + 40 = 90\n- Dumbbell presses: 20 + 40 = 60\n\n**Grand total:** 220 + 90 + 60 = 370\n\nThe final answer is 370", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 134, "total_tokens": 324, "completion_tokens": 190, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 190, "latency_s": 14.22, "error": null, "agree_key": "370", "pred": "370", "correct": true}
+{"id": "gsm8k_test_298", "task": "gsm8k", "prompt_sha256": "4bb525049fdbc040", "content": "Let me work through this step by step.\n\n**Setting up the problem:**\n- Opening bid: $200\n- Bids rise by $50 each time\n- 3 other people each bid once\n- Carmen bids after each of the 3 other people\n- Carmen eventually wins\n\n**Sequence of bids:**\n\n1. Opening bid: $200\n2. Person 1 bids: $200 + $50 = $250\n3. Carmen bids: $250 + $50 = $300\n4. Person 2 bids: $300 + $50 = $350\n5. Carmen bids: $350 + $50 = $400\n6. Person 3 bids: $400 + $50 = $450\n7. Carmen bids: $450 + $50 = $500\n\nCarmen wins at $500.\n\nThe final answer is 500", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 124, "total_tokens": 343, "completion_tokens": 219, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 219, "latency_s": 18.41, "error": null, "agree_key": "500", "pred": "500", "correct": true}
+{"id": "gsm8k_test_307", "task": "gsm8k", "prompt_sha256": "57be1a5920582601", "content": "Let me work through this step by step.\n\n**Given information:**\n- Joey loses 8 pounds in 4 weeks\n- Sandy needs 4 weeks to lose the same amount of weight that Joey loses in a single week\n\n**Step 1: Find how much weight Joey loses per week.**\nJoey loses 8 pounds in 4 weeks.\nWeight per week for Joey = 8 ÷ 4 = 2 pounds per week\n\n**Step 2: Find how much weight Sandy loses per week.**\nSandy needs 4 weeks to lose the same amount Joey loses in 1 week.\nSo Sandy loses 2 pounds in 4 weeks.\nWeight per week for Sandy = 2 ÷ 4 = 0.5 pounds per week\n\n**Step 3: Find how many weeks it takes Sandy to lose the same amount Joey loses in 4 weeks (8 pounds).**\nSandy loses 0.5 pounds per week.\nWeeks needed = 8 ÷ 0.5 = 16 weeks\n\nThe final answer is 16", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 103, "total_tokens": 329, "completion_tokens": 226, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 226, "latency_s": 19.293, "error": null, "agree_key": "16", "pred": "16", "correct": true}
+{"id": "gsm8k_test_340", "task": "gsm8k", "prompt_sha256": "2cc1e6445af2dd49", "content": "Let me work through this step by step.\n\n**Step 1: Determine how many balls Josh starts with.**\nJosh starts by juggling 3 balls.\n\n**Step 2: Determine how many balls he adds over 4 weeks.**\nHe adds 1 ball each week for 4 weeks:\n- Week 1: 3 + 1 = 4 balls\n- Week 2: 4 + 1 = 5 balls\n- Week 3: 5 + 1 = 6 balls\n- Week 4: 6 + 1 = 7 balls\n\nSo after 4 weeks, Josh is juggling 7 balls.\n\n**Step 3: Determine how many balls he has when he drops them.**\nHe drops 3 balls on stage.\n7 - 3 = 4 balls remaining\n\n**Step 4: Account for the dropped balls.**\n- 2 of the dropped balls are caught by people in the crowd\n- 1 gets lost completely\n\nThe 2 caught balls are recovered, so Josh has them back. The 1 lost ball is gone forever.\n\nSo Josh has: 4 (remaining) + 2 (caught) = 6 balls\n\n**Step 5: Determine how many balls Josh juggles.**\nJosh juggles with 6 balls.\n\nThe final answer is 6", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 169, "total_tokens": 452, "completion_tokens": 283, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 283, "latency_s": 27.61, "error": null, "agree_key": "6", "pred": "6", "correct": false}
+{"id": "gsm8k_test_349", "task": "gsm8k", "prompt_sha256": "8c0eb966ad7141e9", "content": "Let me solve this step by step.\n\n**Given information:**\n- Food costs $25 per week\n- Treats cost $20 per month\n- Medicine costs $100 per month\n- There are 4 weeks in a month\n- We need to find the yearly cost\n\n**Step 1: Calculate monthly food costs**\nFood = $25 per week × 4 weeks per month = $100 per month\n\n**Step 2: Calculate total monthly costs**\nMonthly total = Food + Treats + Medicine\nMonthly total = $100 + $20 + $100 = $220 per month\n\n**Step 3: Calculate yearly costs**\nYearly total = $220 per month × 12 months = $2,640 per year\n\nThe final answer is 2640", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 112, "total_tokens": 296, "completion_tokens": 184, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 184, "latency_s": 14.837, "error": null, "agree_key": "2640", "pred": "2640", "correct": true}
+{"id": "gsm8k_test_353", "task": "gsm8k", "prompt_sha256": "2e227ad2f58a1f57", "content": "Let me work through this step by step.\n\n**Activities John did:**\n- Boating: 6 hours\n- Swimming: half of 6 hours = 3 hours\n- Shows: 3 shows × 2 hours each = 6 hours\n\n**Total time for these activities:**\n6 + 3 + 6 = 15 hours\n\n**Finding total vacation time:**\nThe problem states that the boating, swimming, and shows (15 hours) was 30% of his total time.\n\nSo: 30% of total time = 15 hours\nTotal time = 15 / 0.30 = 50 hours\n\n**Time spent sightseeing:**\nHe spent 40% of his time sightseeing.\n40% × 50 hours = 20 hours\n\nThe final answer is 20", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 122, "total_tokens": 306, "completion_tokens": 184, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 184, "latency_s": 16.41, "error": null, "agree_key": "20", "pred": "20", "correct": true}
+{"id": "gsm8k_test_360", "task": "gsm8k", "prompt_sha256": "e5b08846dfdc3adf", "content": "Let me solve this step by step.\n\n**Given information:**\n- Carver is 45 years old\n- Carver's age is 5 years less than twice the age of his son\n\n**Setting up the equation:**\n\nLet s = son's age\n\n\"5 years less than twice the age of his son\" means: 2s - 5\n\nSo: 45 = 2s - 5\n\n**Solving:**\n45 = 2s - 5\n45 + 5 = 2s\n50 = 2s\ns = 25\n\n**Verification:**\nTwice the son's age: 2 × 25 = 50\n5 less than that: 50 - 5 = 45 ✓\n\nThe final answer is 25", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 81, "total_tokens": 256, "completion_tokens": 175, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 175, "latency_s": 14.423, "error": null, "agree_key": "25", "pred": "25", "correct": true}
+{"id": "gsm8k_test_415", "task": "gsm8k", "prompt_sha256": "d0f6afa9bb8ed4a9", "content": "Let me work through this step by step.\n\n**Joanne's apples:**\n- Tallest trees: 30 apples\n- Shortest trees: half of 30 = 15 apples\n- Average trees: unknown (let's call this x)\n\n**Sister's apples:**\n- Tallest trees: twice as many as Joanne = 2 × 30 = 60 apples\n- Shortest trees: 3 times as many as Joanne = 3 × 15 = 45 apples\n- Average trees: 0 apples\n\n**Combined total:**\nJoanne's total + Sister's total = 500\n\nJoanne's total = 30 + 15 + x = 45 + x\nSister's total = 60 + 45 + 0 = 105\n\nCombined: (45 + x) + 105 = 500\n150 + x = 500\nx = 350\n\nThe final answer is 350", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 160, "total_tokens": 388, "completion_tokens": 228, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 228, "latency_s": 20.172, "error": null, "agree_key": "350", "pred": "350", "correct": true}
+{"id": "gsm8k_test_425", "task": "gsm8k", "prompt_sha256": "16ca8003af2685c1", "content": "Let me break this problem down step by step.\n\n**Step 1: Determine how many bird feeders Lillian has**\n- She builds 3 bird feeders\n- She buys 3 bird feeders\n- Total: 3 + 3 = 6 bird feeders\n\n**Step 2: Determine how many birds each type of feeder attracts**\n- Store-bought feeders attract 20 birds each\n- Self-built feeders attract 10 more birds each than store-bought ones\n- So self-built feeders attract: 20 + 10 = 30 birds each\n\n**Step 3: Calculate total birds**\n- 3 self-built feeders × 30 birds = 90 birds\n- 3 store-bought feeders × 20 birds = 60 birds\n- Total: 90 + 60 = 150 birds\n\nThe final answer is 150", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 146, "total_tokens": 348, "completion_tokens": 202, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 202, "latency_s": 17.029, "error": null, "agree_key": "150", "pred": "150", "correct": true}
+{"id": "gsm8k_test_429", "task": "gsm8k", "prompt_sha256": "b206c79c03d3b8ca", "content": "Let me solve this step by step.\n\n**Step 1: Find the number of letters in Indras' name.**\nIndras has 6 letters in her name.\n\n**Step 2: Find the number of letters in her sister's name.**\nHer sister's name has 4 more letters than half of the letters in Indras' name.\n\nHalf of Indras' letters = 6 ÷ 2 = 3\nSister's letters = 3 + 4 = 7\n\n**Step 3: Find the total number of letters in both names.**\nTotal = Indras' letters + Sister's letters\nTotal = 6 + 7 = 13\n\nThe final answer is 13", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 89, "total_tokens": 242, "completion_tokens": 153, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 153, "latency_s": 13.622, "error": null, "agree_key": "13", "pred": "13", "correct": true}
+{"id": "gsm8k_test_434", "task": "gsm8k", "prompt_sha256": "eb9d23f7786e55bb", "content": "Let me work through this step by step.\n\n**Given information:**\n- Chips cost 40 cents each\n- Candy bars cost 75 cents each\n- George spent $5 (500 cents)\n- He bought 3 bags of chips\n- He had 1% of his money left\n\n**Step 1: Find how much money George started with.**\n\nHe had 1% of his money left, meaning he spent 99% of his money.\n\nTotal spent = 99% of total money\n$5 = 0.99 × total money\nTotal money = $5 / 0.99 = $505.05...\n\nWait, let me reconsider. Let me re-read the problem.\n\n\"George spent $5 and got 3 bags of chips and had 1% of his money left.\"\n\nSo he spent $5 total, and after spending, he had 1% of his original money left.\n\nLet me set up the equation:\n- Let T = total money George started with\n- He spent $5\n- After spending, he has 1% of T left\n\nSo: T - 5 = 0.01T\nT - 0.01T = 5\n0.99T = 5\nT = 5/0.99 = $5.0505...\n\nHmm, that doesn't give a clean number. Let me reconsider.\n\nWait, maybe \"had 1% of his money left\" means he had 1% of $5 left? No, that doesn't make sense either.\n\nLet me re-read: \"George spent $5 and got 3 bags of chips and had 1% of his money left.\"\n\nI think the interpretation is:\n- George started with some amount of money\n- He spent $5 total (on chips and candy bars)\n- After spending, he had 1% of his original money left\n\nSo: T - 5 = 0.01T\n0.99T = 5\nT = 5/0.99 ≈ $5.05\n\nThat's not a clean number. Let me reconsider the problem.\n\nActually, wait. Maybe \"had 1% of his money left\" means he had 1% of the $5 left, which is $0.05.\n\nSo George started with $5 + $0.05 = $5.05? No, that doesn't work either because he spent $5.\n\nLet me re-read more carefully. \"George spent $5 and got 3 bags of chips and had 1% of his money left.\"\n\nI think the most natural reading is:\n- George had some total amount of money\n- He spent $5 (on chips and candy bars)\n- After spending, he had 1% of his original money remaining\n\nT - 5 = 0.01T\nT - 0.01T = 5\n0.99T = 5\nT = 5/0.99 = 500/99 ≈ $5.05\n\nThis doesn't give a clean answer. Let me try another interpretation.\n\nMaybe \"had 1% of his money left\" means he had 1% of $5 left = $0.05.\n\nSo George started with $5 + $0.05 = $5.05? No, that doesn't make sense.\n\nWait, maybe the problem means he spent $5 and had 1% of that $5 left, meaning he actually spent $4.95?\n\nNo, let me re-read: \"George spent $5 and got 3 bags of chips and had 1% of his money left.\"\n\nI think the most logical interpretation is:\n- George had some total amount of money T\n- He spent $5 (on chips and candy bars)\n- After spending, he had 1% of T left\n\nT - 5 = 0.01T\n0.99T = 5\nT = 5/0.99\n\nHmm, but this gives a non-integer. Let me try:\n\nActually, wait. Maybe the problem is saying he spent $5 and had 1% of his money left, meaning the $5 IS 99% of his money.\n\nSo 99% of T = $5\nT = $5/0.99 = $5.0505...\n\nThis still doesn't give a clean answer.\n\nLet me try yet another interpretation. Maybe \"had 1% of his money left\" means he had 1% of the total amount he had left, and the total amount he had was $5.\n\nSo he had $5", "reasoning_content": "", "tool_calls": [], "finish_reason": "length", "usage": {"prompt_tokens": 94, "total_tokens": 1118, "completion_tokens": 1024, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 1024, "latency_s": 85.601, "error": null, "agree_key": "5", "pred": "5", "correct": true}
+{"id": "gsm8k_test_441", "task": "gsm8k", "prompt_sha256": "b8e51655058bacf4", "content": "Let me solve this step by step.\n\n**Elizabeth's beads:**\n- 1 pack of red beads = 1 × 20 = 20 beads\n- 2 packs of clear beads = 2 × 20 = 40 beads\n- Total: 20 + 40 = 60 beads\n\n**Margareth's beads:**\n- 3 packs of blue beads = 3 × 20 = 60 beads\n- 4 packs of red beads = 4 × 20 = 80 beads\n- Total: 60 + 80 = 140 beads\n\n**Difference:**\n140 - 60 = 80 beads\n\nMargareth has 80 more beads than Elizabeth.\n\nThe final answer is 80", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 114, "total_tokens": 286, "completion_tokens": 172, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 172, "latency_s": 14.889, "error": null, "agree_key": "80", "pred": "80", "correct": true}
+{"id": "gsm8k_test_458", "task": "gsm8k", "prompt_sha256": "a23a391c53cba06f", "content": "To find the percentage of all the puppies that have spots, let's follow these steps:\n\n1. **Calculate the total number of puppies:**\n   - Jennifer's dog has 8 puppies.\n   - Brandon's dog has 12 puppies.\n   - Total puppies = 8 + 12 = 20.\n\n2. **Calculate the total number of puppies with spots:**\n   - Jennifer's dog has 3 spotted puppies.\n   - Brandon's dog has 4 spotted puppies.\n   - Total spotted puppies = 3 + 4 = 7.\n\n3. **Calculate the percentage of spotted puppies:**\n   - Percentage = (Total spotted puppies / Total puppies) × 100\n   - Percentage = (7 / 20) × 100 = 35%.\n\nThe final answer is 35.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 86, "total_tokens": 269, "completion_tokens": 183, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 183, "latency_s": 16.816, "error": null, "agree_key": "35", "pred": "35", "correct": true}
+{"id": "gsm8k_test_464", "task": "gsm8k", "prompt_sha256": "6b4745921eb73c34", "content": "Let me track Finley's position step by step.\n\nStarting position: 1st place\n\n1. Fell back 5 spots: 1 + 5 = 6th place\n2. Moved ahead 2 spots: 6 - 2 = 4th place\n3. Fell behind 3 spots: 4 + 3 = 7th place\n4. Jumped ahead 1 spot: 7 - 1 = 6th place\n\nThe final answer is 6", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 111, "total_tokens": 215, "completion_tokens": 104, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 104, "latency_s": 10.198, "error": null, "agree_key": "6", "pred": "6", "correct": true}
+{"id": "gsm8k_test_471", "task": "gsm8k", "prompt_sha256": "4067d67c5450d022", "content": "Let me solve this step by step.\n\n**Step 1: Find the total number of pencils at the start of the year.**\n- 30 students × 10 pencils each = 300 pencils\n\n**Step 2: Find how many pencils were used after two months.**\n- 1/5 of 300 = 300 × (1/5) = 60 pencils were used\n- Remaining pencils: 300 - 60 = 240 pencils\n\n**Step 3: Find how many pencils were left at the end of the year.**\n- 1/3 of the remaining 240 pencils were left\n- 240 × (1/3) = 80 pencils\n\nThe final answer is 80.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 111, "total_tokens": 282, "completion_tokens": 171, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 171, "latency_s": 15.935, "error": null, "agree_key": "80", "pred": "80", "correct": true}
+{"id": "gsm8k_test_478", "task": "gsm8k", "prompt_sha256": "8a080a9ef176c3f7", "content": "Let's solve this step by step.\n\n**Step 1: Calculate the total number of employees.**\n- There are 3 groups.\n- Each group has 200 employees.\n- Total employees = 3 × 200 = 600\n\n**Step 2: Calculate the total number of tour guides.**\n- Each group is assigned 7 tour guides.\n- Total tour guides = 3 × 7 = 21\n\n**Step 3: Calculate the total number of people on the tour.**\n- Total people = Total employees + Total tour guides\n- Total people = 600 + 21 = 621\n\nThe final answer is 621", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 99, "total_tokens": 251, "completion_tokens": 152, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 152, "latency_s": 12.773, "error": null, "agree_key": "621", "pred": "621", "correct": true}
+{"id": "gsm8k_test_498", "task": "gsm8k", "prompt_sha256": "bbee94433693b474", "content": "Let me solve this step by step.\n\n**Step 1: Find how many oranges Ashley brought.**\nAshley brought 5 boxes with 10 oranges in each box.\n5 × 10 = 50 oranges\n\n**Step 2: Find how many oranges Brianne brought.**\nBrianne brought 20 more oranges than Ashley.\n50 + 20 = 70 oranges\n\n**Step 3: Find the total number of oranges.**\n50 + 70 = 120 oranges\n\n**Step 4: Find how many pies they can make.**\nEach pie needs 3 oranges.\n120 ÷ 3 = 40 pies\n\nThe final answer is 40", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 99, "total_tokens": 255, "completion_tokens": 156, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 156, "latency_s": 13.529, "error": null, "agree_key": "40", "pred": "40", "correct": true}
+{"id": "gsm8k_test_506", "task": "gsm8k", "prompt_sha256": "898be87d443a78f1", "content": "Let me solve this step by step.\n\n**Step 1:** Find the number of legs that need snowshoes.\n- Mario has 6 sled dogs\n- Each dog has 4 legs\n- Total legs = 6 × 4 = 24 legs\n\n**Step 2:** Find how many pairs of snowshoes are needed.\n- A pair of snowshoes covers 2 legs\n- Number of pairs needed = 24 ÷ 2 = 12 pairs\n\n**Step 3:** Calculate the total cost.\n- Each pair costs $12.00\n- Total cost = 12 × $12.00 = $144.00\n\nThe final answer is $144.00", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 100, "total_tokens": 262, "completion_tokens": 162, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 162, "latency_s": 14.922, "error": null, "agree_key": "144", "pred": "144", "correct": true}
+{"id": "gsm8k_test_515", "task": "gsm8k", "prompt_sha256": "6e94df12666e68d6", "content": "Let me solve this step by step.\n\n**Step 1: Find the total number of canvas bags**\nTara bought 8 packs of 5 canvas bags each.\nTotal bags = 8 × 5 = 40 bags\n\n**Step 2: Find the total cost**\nEach pack cost $4.\nTotal cost = 8 × $4 = $32\n\n**Step 3: Find the total revenue from selling**\nShe sold each bag for $8.\nTotal revenue = 40 × $8 = $320\n\n**Step 4: Calculate the profit**\nProfit = Revenue - Cost\nProfit = $320 - $32 = $288\n\nThe final answer is 288", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 87, "total_tokens": 247, "completion_tokens": 160, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 160, "latency_s": 14.67, "error": null, "agree_key": "288", "pred": "288", "correct": false}
+{"id": "gsm8k_test_525", "task": "gsm8k", "prompt_sha256": "de7ccd7446f1ef68", "content": "Let me solve this step by step.\n\n**Step 1: Find how many cups of raisins go to each category.**\n\nHeather has 27 cups of raisins total and wants to divide them equally among 3 categories (oatmeal cookies, granola, and snack mix).\n\n27 cups ÷ 3 categories = 9 cups per category\n\n**Step 2: Find how many batches of oatmeal cookies she can make.**\n\nEach batch of oatmeal cookies takes 3/4 of a cup of raisins.\n\nNumber of batches = 9 cups ÷ (3/4 cup per batch)\n\n9 ÷ (3/4) = 9 × (4/3) = 36/3 = 12\n\nThe final answer is 12", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 134, "total_tokens": 302, "completion_tokens": 168, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 168, "latency_s": 15.966, "error": null, "agree_key": "12", "pred": "12", "correct": true}
+{"id": "gsm8k_test_534", "task": "gsm8k", "prompt_sha256": "fe2a72d70e4c3a12", "content": "Let me work through this problem step by step.\n\n**Given information:**\n- 20 kids total, divided into two equal groups of girls and boys\n- So there are 10 boys and 10 girls\n- Each boy brought back 60 shells\n- Each girl brought an amount equal to the number of shells brought by the boys PLUS four times as many seashells as the boys\n\n**Setting up the equation:**\n\nEach boy brought 60 shells.\n\nEach girl brought: (shells brought by boys) + 4 × (shells brought by boys)\n= 60 + 4(60)\n= 60 + 240\n= 300\n\nWait, let me re-read the problem. \"If the girls brought an amount equal to the number of shells brought by the boys plus four times as many seashells as the boys\"\n\nThis means each girl brought: 60 + 4(60) = 60 + 240 = 300\n\nThe final answer is 300.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 145, "total_tokens": 377, "completion_tokens": 232, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 232, "latency_s": 21.424, "error": null, "agree_key": "300", "pred": "300", "correct": true}
+{"id": "gsm8k_test_543", "task": "gsm8k", "prompt_sha256": "9c6b4c10b9c42d2b", "content": "Let me solve this step by step.\n\n**Given information:**\n- Jairus gets $0.80 per task\n- Jenny gets $0.50 per task\n- Each of them finished 20 tasks\n\n**Step 1: Calculate how much Jairus earns**\nJairus: 20 tasks × $0.80 = $16.00\n\n**Step 2: Calculate how much Jenny earns**\nJenny: 20 tasks × $0.50 = $10.00\n\n**Step 3: Find the difference**\n$16.00 - $10.00 = $6.00\n\nThe final answer is 6", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 87, "total_tokens": 239, "completion_tokens": 152, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 152, "latency_s": 13.329, "error": null, "agree_key": "6", "pred": "6", "correct": true}
+{"id": "gsm8k_test_548", "task": "gsm8k", "prompt_sha256": "cba0f1d50152dcdb", "content": "Let me solve this step by step.\n\n**Step 1: Determine the total weight lost.**\nMark lost 10 pounds per month for 3 months.\nTotal weight lost = 10 × 3 = 30 pounds\n\n**Step 2: Calculate the initial weight.**\nHis final weight was 70 pounds, and he lost 30 pounds during this period.\nInitial weight = Final weight + Weight lost\nInitial weight = 70 + 30 = 100 pounds\n\nThe final answer is 100", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 82, "total_tokens": 200, "completion_tokens": 118, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 118, "latency_s": 11.839, "error": null, "agree_key": "100", "pred": "100", "correct": true}
+{"id": "gsm8k_test_591", "task": "gsm8k", "prompt_sha256": "7a80dc1e04d89d13", "content": "Let me solve this step by step.\n\n**Step 1:** The Affenpinscher weighed 10 pounds.\n\n**Step 2:** The Dachshund weighed twice as much as the Affenpinscher.\n- Dachshund = 2 × 10 = 20 pounds\n\n**Step 3:** The Papillon weighed one-fourth as much as the Dachshund.\n- Papillon = 20 ÷ 4 = 5 pounds\n\n**Step 4:** The Mastiff weighed 44 times the weight of the Papillon.\n- Mastiff = 44 × 5 = 220 pounds\n\nThe final answer is 220", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 163, "total_tokens": 311, "completion_tokens": 148, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 148, "latency_s": 12.71, "error": null, "agree_key": "220", "pred": "220", "correct": true}
+{"id": "gsm8k_test_627", "task": "gsm8k", "prompt_sha256": "2cdf0fcef5c972a8", "content": "Let me solve this step by step.\n\n**First three days:**\n- The tow truck pulled 10 cars each day for 3 days\n- Total for first three days: 10 × 3 = 30 cars\n\n**Remaining days:**\n- A week has 7 days, so remaining days = 7 - 3 = 4 days\n- On each of these days, the tow truck pulled 4 fewer cars than the first three days\n- Cars per remaining day: 10 - 4 = 6 cars\n- Total for remaining days: 6 × 4 = 24 cars\n\n**Total for the week:**\n- 30 + 24 = 54 cars\n\nThe final answer is 54", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 91, "total_tokens": 252, "completion_tokens": 161, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 161, "latency_s": 14.186, "error": null, "agree_key": "54", "pred": "54", "correct": true}
+{"id": "gsm8k_test_643", "task": "gsm8k", "prompt_sha256": "729604fea01edf0e", "content": "To solve this problem, let's break it down step by step:\n\n1. **Determine the number of articles published by the news website per day:**\n   - The news website publishes an average of **20** political and weather news articles every day.\n\n2. **Determine the number of articles published by the sister company per day:**\n   - The sister company publishes an average of **10** business news articles daily.\n\n3. **Calculate the total number of articles published by both websites per day:**\n   - Total articles per day = 20 (news website) + 10 (sister company) = **30** articles per day.\n\n4. **Calculate the total number of articles published in February:**\n   - February has **28** days.\n   - Total articles in February = 30 articles/day × 28 days = **840** articles.\n\nThe final answer is 840.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 103, "total_tokens": 306, "completion_tokens": 203, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 203, "latency_s": 18.074, "error": null, "agree_key": "840", "pred": "840", "correct": true}
+{"id": "gsm8k_test_645", "task": "gsm8k", "prompt_sha256": "b2aed100d54e16da", "content": "Let me solve this step by step.\n\n**Step 1: Find the amount of flour in one bag.**\n\nOne bag is divided into 8 portions of 2 kilograms each.\n\nFlour in one bag = 8 × 2 = 16 kilograms\n\n**Step 2: Find the total amount of flour in three bags.**\n\nTotal flour in three bags = 3 × 16 = 48 kilograms\n\nThe final answer is 48", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 83, "total_tokens": 181, "completion_tokens": 98, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 98, "latency_s": 9.14, "error": null, "agree_key": "48", "pred": "48", "correct": true}
+{"id": "gsm8k_test_656", "task": "gsm8k", "prompt_sha256": "29cdcf0e8dd6ab44", "content": "To determine how many days before the deadline Mike will finish his reading, let's break down the problem step by step.\n\n1. **Total pages in the book:** 200 pages.\n2. **Mike's reading rate:** 10 pages per day.\n3. **Total days available to complete the assignment:** 30 days.\n\n**Step 1: Calculate the number of days Mike needs to finish the book.**\n\\[\n\\text{Days needed} = \\frac{\\text{Total pages}}{\\text{Pages per day}} = \\frac{200}{10} = 20 \\text{ days}\n\\]\n\n**Step 2: Determine how many days before the deadline Mike will finish.**\n\\[\n\\text{Days before deadline} = \\text{Total days available} - \\text{Days needed} = 30 - 20 = 10 \\text{ days}\n\\]\n\n**Final Answer:**\nThe final answer is 10.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 99, "total_tokens": 315, "completion_tokens": 216, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 216, "latency_s": 19.808, "error": null, "agree_key": "10", "pred": "10", "correct": true}
+{"id": "gsm8k_test_663", "task": "gsm8k", "prompt_sha256": "6ab38caceafc3aa8", "content": "To determine how many newspapers James delivers to District B, we can follow these steps:\n\n1. **Total newspapers delivered in a day:** 600\n2. **Newspapers delivered to District A:** 198\n3. **Newspapers delivered to District C:** 209\n4. **Newspapers delivered to District B:** Let's call this \\( x \\).\n\nThe sum of newspapers delivered to all districts should equal the total:\n\\[\n198 + x + 209 = 600\n\\]\n\nCombine the known values:\n\\[\n407 + x = 600\n\\]\n\nSolve for \\( x \\):\n\\[\nx = 600 - 407\n\\]\n\\[\nx = 193\n\\]\n\n**The final answer is 193**", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 92, "total_tokens": 282, "completion_tokens": 190, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 190, "latency_s": 18.285, "error": null, "agree_key": "193", "pred": "193", "correct": true}
+{"id": "gsm8k_test_670", "task": "gsm8k", "prompt_sha256": "517ffd08acaf40ba", "content": "Step 1: Find the total amount of candy.\nRobert: 3 pounds\nCindy: 5 pounds\nAaron: 4 pounds\nTotal = 3 + 5 + 4 = 12 pounds\n\nStep 2: Divide the total equally among the 3 people.\n12 ÷ 3 = 4 pounds each\n\nThe final answer is 4", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 96, "total_tokens": 177, "completion_tokens": 81, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 81, "latency_s": 8.187, "error": null, "agree_key": "4", "pred": "4", "correct": true}
+{"id": "gsm8k_test_677", "task": "gsm8k", "prompt_sha256": "edca6f1ccb46ef51", "content": "Let me solve this step by step.\n\n**Step 1: Find the total number of signatures they currently have.**\n- Carol has 20 signatures\n- Jennifer has 44 signatures\n- Total current signatures = 20 + 44 = 64\n\n**Step 2: Find how many more signatures they need to reach their goal.**\n- Their goal is 100 signatures total\n- They currently have 64 signatures\n- Signatures needed = 100 - 64 = 36\n\nThe final answer is 36", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 162, "total_tokens": 284, "completion_tokens": 122, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 122, "latency_s": 11.672, "error": null, "agree_key": "36", "pred": "36", "correct": true}
+{"id": "gsm8k_test_679", "task": "gsm8k", "prompt_sha256": "8f34802d0d71140e", "content": "To solve this problem, let's break it down step by step.\n\n1. **Understand the given ratio:**\n   - 6 potatoes make 36 hash browns.\n   - This means the ratio of potatoes to hash browns is 6:36, which simplifies to 1:6.\n\n2. **Find the number of hash browns per potato:**\n   - If 1 potato makes 6 hash browns, then:\n   - 96 potatoes will make \\( 96 \\times 6 \\) hash browns.\n\n3. **Calculate the total number of hash browns:**\n   - \\( 96 \\times 6 = 576 \\).\n\nSo, 96 potatoes can make 576 hash browns.\n\nThe final answer is 576.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 73, "total_tokens": 252, "completion_tokens": 179, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 179, "latency_s": 16.427, "error": null, "agree_key": "576", "pred": "576", "correct": true}
+{"id": "gsm8k_test_720", "task": "gsm8k", "prompt_sha256": "65ebd30a351362db", "content": "Let me solve this step by step.\n\n**This week:**\n- Coco Crunch: 3 boxes\n- Fruit Loops: 5 boxes\n- Total this week: 3 + 5 = 8 boxes\n\n**Last week:**\n- Cereal: 4 boxes\n\n**Difference:**\n8 - 4 = 4\n\nThe final answer is 4", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 90, "total_tokens": 168, "completion_tokens": 78, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 78, "latency_s": 8.14, "error": null, "agree_key": "4", "pred": "4", "correct": true}
+{"id": "gsm8k_test_751", "task": "gsm8k", "prompt_sha256": "ac5af3bf3c215691", "content": "Let me solve this step by step.\n\n**Starting amount:** Gunther had 48 bananas.\n\n**Step 1:** Arnold stole half of the bananas.\n- Half of 48 = 24\n- Bananas remaining: 48 - 24 = 24\n\n**Step 2:** Gunther added 25 bananas.\n- 24 + 25 = 49\n\n**Step 3:** Arnold stole 12 bananas.\n- 49 - 12 = 37\n\n**Step 4:** Gunther added 6 bananas.\n- 37 + 6 = 43\n\nThe final answer is 43.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 153, "total_tokens": 303, "completion_tokens": 150, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 150, "latency_s": 12.612, "error": null, "agree_key": "43", "pred": "43", "correct": true}
+{"id": "gsm8k_test_761", "task": "gsm8k", "prompt_sha256": "460f1a89a12578b8", "content": "Let me work through this step by step.\n\n**Given information:**\n- Two thirds of Jana's puppies are Pomeranians\n- One third of the Pomeranians are girls\n- There are 6 Pomeranian girls\n\n**Step 1: Find the total number of Pomeranians.**\n\nIf one third of the Pomeranians are girls, and there are 6 Pomeranian girls:\n$$\\frac{1}{3} \\times \\text{Pomeranians} = 6$$\n$$\\text{Pomeranians} = 6 \\times 3 = 18$$\n\n**Step 2: Find the total number of puppies.**\n\nIf two thirds of all puppies are Pomeranians, and there are 18 Pomeranians:\n$$\\frac{2}{3} \\times \\text{Total puppies} = 18$$\n$$\\text{Total puppies} = 18 \\times \\frac{3}{2} = 27$$\n\nThe final answer is 27", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 86, "total_tokens": 313, "completion_tokens": 227, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 227, "latency_s": 18.589, "error": null, "agree_key": "27", "pred": "27", "correct": true}
+{"id": "gsm8k_test_765", "task": "gsm8k", "prompt_sha256": "0a541a9cdf5cd8af", "content": "Step 1: Find the number of pears.\nGeorge has 45% more pears than bananas.\nPears = 200 + 45% of 200\nPears = 200 + 0.45 × 200\nPears = 200 + 90\nPears = 290\n\nStep 2: Find the total number of fruits.\nTotal fruits = bananas + pears\nTotal fruits = 200 + 290\nTotal fruits = 490\n\nThe final answer is 490", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 74, "total_tokens": 206, "completion_tokens": 132, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 132, "latency_s": 11.081, "error": null, "agree_key": "490", "pred": "490", "correct": true}
+{"id": "gsm8k_test_768", "task": "gsm8k", "prompt_sha256": "21264a8d224a486f", "content": "Let me solve this step by step.\n\n**Given information:**\n- In one hour, Ezra read twice as many books as Ahmed\n- Ezra read 300 books this hour\n- Ezra decided to read 150 more books\n\n**Step 1: Find how many books Ahmed read.**\nSince Ezra read twice as many books as Ahmed:\n- Ezra's books = 2 × Ahmed's books\n- 300 = 2 × Ahmed's books\n- Ahmed's books = 300 ÷ 2 = 150 books\n\n**Step 2: Find Ezra's total books.**\nEzra read 300 books and decided to read 150 more:\n- Ezra's total = 300 + 150 = 450 books\n\n**Step 3: Find the total books read by both.**\n- Total = Ezra's total + Ahmed's books\n- Total = 450 + 150 = 600 books\n\nThe final answer is 600", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 87, "total_tokens": 313, "completion_tokens": 226, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 226, "latency_s": 19.959, "error": null, "agree_key": "600", "pred": "600", "correct": false}
+{"id": "gsm8k_test_833", "task": "gsm8k", "prompt_sha256": "e343d6270536142f", "content": "Let me solve this step by step.\n\n**Given information:**\n- Naruto can lift 10 times higher than Kagiyami\n- Kagiyami can lift 4 times higher than Saskay\n- Saskay can lift 12 times higher than Pompei\n- Pompei can lift 1 inch\n\n**Step 1: Find how high Saskay can lift**\nSaskay = 12 × Pompei = 12 × 1 inch = 12 inches\n\n**Step 2: Find how high Kagiyami can lift**\nKagiyami = 4 × Saskay = 4 × 12 inches = 48 inches\n\n**Step 3: Find how high Naruto can lift**\nNaruto = 10 × Kagiyami = 10 × 48 inches = 480 inches\n\n**Step 4: Convert inches to feet**\n480 inches ÷ 12 inches/foot = 40 feet\n\nThe final answer is 40", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 121, "total_tokens": 340, "completion_tokens": 219, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 219, "latency_s": 18.341, "error": null, "agree_key": "40", "pred": "40", "correct": true}
+{"id": "gsm8k_test_861", "task": "gsm8k", "prompt_sha256": "3a9285de6ce7bb12", "content": "Let me solve this step by step.\n\n**Claire's badges:**\n- Claire earns 1 badge per month\n- In 1 year (12 months): 1 × 12 = 12 badges\n\n**Amber's badges:**\n- It takes Amber twice as long to earn a badge as Claire\n- So Amber earns 1/2 badge per month\n- In 1 year: (1/2) × 12 = 6 badges\n\n**Wendy's badges:**\n- Wendy earns three times the amount of badges as Claire in the same time frame\n- In 1 year: 3 × 12 = 36 badges\n\n**Difference between Wendy and Amber:**\n- 36 - 6 = 30 badges\n\nThe final answer is 30", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 107, "total_tokens": 279, "completion_tokens": 172, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 172, "latency_s": 15.351, "error": null, "agree_key": "30", "pred": "30", "correct": true}
+{"id": "gsm8k_test_878", "task": "gsm8k", "prompt_sha256": "5dcbf779ecdfb89e", "content": "Let me solve this step by step.\n\n**Given information:**\n- Julie, Micah, and Mitchell sold 32 glasses of lemonade total\n- Julie sold 14 glasses\n- The boys (Micah and Mitchell) sold an equal number of glasses\n\n**Step 1: Find how many glasses the boys sold together.**\nTotal glasses - Julie's glasses = Boys' glasses\n32 - 14 = 18 glasses\n\n**Step 2: Find how many glasses Micah sold.**\nSince the boys sold an equal number of glasses:\n18 ÷ 2 = 9 glasses each\n\nSo Micah sold 9 glasses.\n\n**Step 3: Find how many more glasses Julie sold than Micah.**\nJulie's glasses - Micah's glasses = 14 - 9 = 5\n\nThe final answer is 5", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 95, "total_tokens": 279, "completion_tokens": 184, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 184, "latency_s": 17.89, "error": null, "agree_key": "5", "pred": "5", "correct": true}
+{"id": "gsm8k_test_907", "task": "gsm8k", "prompt_sha256": "71902e0726c66225", "content": "Step 1: Determine how many books Joseph originally had.\nJoseph had twice the number of Sarah's books.\nSarah has 9 books, so Joseph originally had 2 × 9 = 18 books.\n\nStep 2: Account for the books Joseph lost.\nJoseph lost 2 books, so he now has 18 - 2 = 16 books.\n\nThe final answer is 16.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 78, "total_tokens": 168, "completion_tokens": 90, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 90, "latency_s": 7.961, "error": null, "agree_key": "16", "pred": "16", "correct": true}
+{"id": "gsm8k_test_908", "task": "gsm8k", "prompt_sha256": "baf3645910c2a4fd", "content": "Let me solve this step by step.\n\n**Step 1:** Start with Dean's total marbles.\nDean has 30 marbles.\n\n**Step 2:** Calculate how many marbles Dean gives to Jamie.\nDean gives 1/5 of his marbles to Jamie.\n1/5 × 30 = 6 marbles\n\n**Step 3:** Calculate how many marbles Dean gives to Donald.\nDean gives 10 marbles to Donald.\n\n**Step 4:** Calculate how many marbles are left for Dean.\nMarbles left = 30 - 6 - 10 = 14 marbles\n\nThe final answer is 14", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 82, "total_tokens": 229, "completion_tokens": 147, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 147, "latency_s": 13.082, "error": null, "agree_key": "14", "pred": "14", "correct": true}
+{"id": "gsm8k_test_911", "task": "gsm8k", "prompt_sha256": "54e6f8b97731bf85", "content": "Let me solve this step by step.\n\n**Step 1: Find the number of packages completed per hour.**\n\nColby completes 10 less than 50 packages per hour.\n50 - 10 = 40 packages per hour\n\n**Step 2: Find the total packages completed in an eight-hour workday.**\n\n40 packages/hour × 8 hours = 320 packages\n\n**Step 3: Calculate his earnings.**\n\nHe gets paid $0.20 per package.\n320 × $0.20 = $64.00\n\nThe final answer is 64", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 106, "total_tokens": 241, "completion_tokens": 135, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 135, "latency_s": 12.918, "error": null, "agree_key": "64", "pred": "64", "correct": true}
+{"id": "gsm8k_test_941", "task": "gsm8k", "prompt_sha256": "78f7ab3f15f45efd", "content": "Let me solve this step by step.\n\n**Step 1: Calculate the total bottles of apple juice from coupons.**\n- The teacher has 9 coupons\n- Each coupon redeems for 100 bottles\n- Total bottles from coupons: 9 × 100 = 900 bottles\n\n**Step 2: Calculate the bottles given to students.**\n- There are 29 pupils\n- Each student gets 2 bottles\n- Total bottles given to students: 29 × 2 = 58 bottles\n\n**Step 3: Calculate the bottles remaining for the teacher.**\n- Bottles for teacher = Total bottles - Bottles given to students\n- Bottles for teacher = 900 - 58 = 842 bottles\n\nThe final answer is 842", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 122, "total_tokens": 294, "completion_tokens": 172, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 172, "latency_s": 17.396, "error": null, "agree_key": "842", "pred": "842", "correct": true}
+{"id": "gsm8k_test_945", "task": "gsm8k", "prompt_sha256": "e3d613a7b2bd68c4", "content": "Let me break this problem down step by step.\n\n**Given information:**\n- Total distance: 20 miles\n- Swimming speed: 2 miles per hour\n- He swims 60% of the distance first\n- He rests for half as long as the swimming time\n- He finishes the remaining distance at half the speed\n\n**Step 1: Calculate the first swimming distance**\n60% of 20 miles = 0.60 × 20 = 12 miles\n\n**Step 2: Calculate the time for the first swim**\nTime = Distance ÷ Speed\nTime = 12 miles ÷ 2 mph = 6 hours\n\n**Step 3: Calculate the rest time**\nRest time = half the swimming time = 6 ÷ 2 = 3 hours\n\n**Step 4: Calculate the remaining distance**\nRemaining distance = 20 - 12 = 8 miles\n\n**Step 5: Calculate the speed for the remaining distance**\nHalf the speed = 2 ÷ 2 = 1 mile per hour\n\n**Step 6: Calculate the time for the remaining distance**\nTime = 8 miles ÷ 1 mph = 8 hours\n\n**Step 7: Calculate the total time**\nTotal time = First swim + Rest + Remaining swim\nTotal time = 6 + 3 + 8 = 17 hours\n\nThe final answer is 17", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 134, "total_tokens": 440, "completion_tokens": 306, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 306, "latency_s": 26.126, "error": null, "agree_key": "17", "pred": "17", "correct": true}
+{"id": "gsm8k_test_947", "task": "gsm8k", "prompt_sha256": "bb4f3e47d03b6090", "content": "Let's solve this step by step:\n\n1. **Start with the initial stickers:**\n   Jasmine was given 15 stickers for participating in class.\n   - Stickers: 15\n\n2. **Subtract the lost stickers:**\n   She lost 7 stickers during playtime.\n   - Stickers: 15 - 7 = 8\n\n3. **Add the additional stickers:**\n   Her teacher gave her 5 more stickers for helping her classmates.\n   - Stickers: 8 + 5 = 13\n\n**Final Answer:**\nThe final answer is 13.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 102, "total_tokens": 233, "completion_tokens": 131, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 131, "latency_s": 12.747, "error": null, "agree_key": "13", "pred": "13", "correct": true}
+{"id": "gsm8k_test_955", "task": "gsm8k", "prompt_sha256": "6e7f914aba67a66a", "content": "Let me solve this step by step.\n\n**Step 1: Find the initial number of occupied beds.**\n- Total capacity: 1000 beds\n- Occupancy rate: 1/5\n- Occupied beds = 1000 × 1/5 = 200 beds\n\n**Step 2: Find the initial number of unoccupied beds.**\n- Unoccupied beds = 1000 - 200 = 800 beds\n\n**Step 3: Calculate the number of new patients admitted over 2 weeks.**\n- 2 weeks = 14 days\n- 50 patients admitted per day\n- Total new patients = 50 × 14 = 700 patients\n\n**Step 4: Calculate the total number of occupied beds after 2 weeks.**\n- Total occupied beds = 200 + 700 = 900 beds\n\n**Step 5: Calculate the number of unoccupied beds after 2 weeks.**\n- Unoccupied beds = 1000 - 900 = 100 beds\n\nThe final answer is 100", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 99, "total_tokens": 346, "completion_tokens": 247, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 247, "latency_s": 20.999, "error": null, "agree_key": "100", "pred": "100", "correct": true}
+{"id": "gsm8k_test_958", "task": "gsm8k", "prompt_sha256": "71b5a74eb0b8b421", "content": "Let me work through this step by step.\n\n**Given information:**\n- Courtney reported the audience was 48 people\n- Kelly said Courtney overstated the number by 20%\n\n**Setting up the equation:**\n\nIf Courtney overstated the number by 20%, that means the actual number of people is 20% less than what Courtney reported.\n\nLet the actual number of people = x\n\nCourtney's reported number = x + 20% of x = x + 0.2x = 1.2x\n\nSo: 1.2x = 48\n\n**Solving for x:**\n\nx = 48 ÷ 1.2\nx = 40\n\n**Verification:**\n- Actual attendance: 40 people\n- 20% of 40 = 8\n- 40 + 8 = 48 ✓ (This matches Courtney's reported number)\n\nThe final answer is 40", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 106, "total_tokens": 313, "completion_tokens": 207, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 207, "latency_s": 18.379, "error": null, "agree_key": "40", "pred": "40", "correct": true}
+{"id": "gsm8k_test_971", "task": "gsm8k", "prompt_sha256": "9b2a3468d9b7eb1c", "content": "To determine how many wooden slats Robert will need, let's break down the problem step by step.\n\n### Step 1: Calculate the Perimeter of the Fence\nThe fence is rectangular with a length of 15 feet and a width of 10 feet. The perimeter \\( P \\) of a rectangle is calculated as:\n\\[\nP = 2 \\times (\\text{length} + \\text{width})\n\\]\nSubstituting the given dimensions:\n\\[\nP = 2 \\times (15 + 10) = 2 \\times 25 = 50 \\text{ feet}\n\\]\n\n### Step 2: Determine the Number of Slats Needed\nRobert needs 2 wood slats for every foot of fencing. Therefore, the total number of slats \\( S \\) is:\n\\[\nS = \\text{Perimeter} \\times \\text{Slats per foot}\n\\]\nSubstituting the values:\n\\[\nS = 50 \\times 2 = 100 \\text{ slats}\n\\]\n\n### Final Answer\nThe final answer is 100.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 95, "total_tokens": 345, "completion_tokens": 250, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 250, "latency_s": 22.72, "error": null, "agree_key": "100", "pred": "100", "correct": true}
+{"id": "gsm8k_test_998", "task": "gsm8k", "prompt_sha256": "207d556a898298ed", "content": "Let me solve this step by step.\n\n**Step 1: Find the total number of cookies.**\n- There were 100 guests at the wedding.\n- Each guest brought a plate of 15 cookies.\n- Total cookies = 100 × 15 = 1,500 cookies\n\n**Step 2: Find how many cookies were given to the church.**\n- The bride gave 1/2 of the cookies to the church.\n- Cookies given to church = 1,500 × 1/2 = 750 cookies\n\n**Step 3: Find how many people were in the church.**\n- Each person in the church got 15 cookies.\n- Number of people = 750 ÷ 15 = 50 people\n\nThe final answer is 50", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 126, "total_tokens": 307, "completion_tokens": 181, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 181, "latency_s": 15.157, "error": null, "agree_key": "50", "pred": "50", "correct": true}
+{"id": "gsm8k_test_1004", "task": "gsm8k", "prompt_sha256": "d4b6841e33cbe617", "content": "Step 1: Find how many chocolate bars Peter has in each box.\nPeter has 4 boxes with a total of 64 chocolate bars.\nChocolate bars per box for Peter = 64 ÷ 4 = 16\n\nStep 2: Find how many chocolate bars Martha has in each box.\nMartha has 7 boxes with a total of 56 chocolate bars.\nChocolate bars per box for Martha = 56 ÷ 7 = 8\n\nStep 3: Find the difference.\nDifference = 16 - 8 = 8\n\nThe final answer is 8", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 109, "total_tokens": 239, "completion_tokens": 130, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 130, "latency_s": 9.915, "error": null, "agree_key": "8", "pred": "8", "correct": true}
+{"id": "gsm8k_test_1008", "task": "gsm8k", "prompt_sha256": "d24e3b8859cb868f", "content": "To determine how many miles farther Shandy drove than Matteo, we'll calculate the distance each person traveled and then find the difference.\n\n**Step 1: Calculate Matteo's distance**\n- Speed: 55 miles per hour\n- Time: 4 hours\n- Distance = Speed × Time\n- Distance = 55 × 4 = 220 miles\n\n**Step 2: Calculate Shandy's distance**\n- Speed: 45 miles per hour\n- Time: 10 hours\n- Distance = Speed × Time\n- Distance = 45 × 10 = 450 miles\n\n**Step 3: Find the difference in distance**\n- Difference = Shandy's distance - Matteo's distance\n- Difference = 450 - 220 = 230 miles\n\nThe final answer is 230.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 88, "total_tokens": 273, "completion_tokens": 185, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 185, "latency_s": 16.761, "error": null, "agree_key": "230", "pred": "230", "correct": true}
+{"id": "gsm8k_test_1013", "task": "gsm8k", "prompt_sha256": "67f298169590227c", "content": "Let me solve this step by step.\n\n**Step 1: Calculate Janet's total hourly earnings**\n- Wages: $10/hour\n- Tips: $15/hour\n- Total per hour: $10 + $15 = $25/hour\n\n**Step 2: Calculate her weekly earnings**\n- Hours per week: 40\n- Weekly earnings: $25 × 40 = $1,000/week\n\n**Step 3: Calculate the down payment amount**\n- Car cost: $10,000\n- Down payment: 20% of $10,000 = 0.20 × $10,000 = $2,000\n\n**Step 4: Calculate how many weeks she needs to work**\n- Weeks needed: $2,000 ÷ $1,000/week = 2 weeks\n\nThe final answer is 2", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 126, "total_tokens": 336, "completion_tokens": 210, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 210, "latency_s": 18.745, "error": null, "agree_key": "2", "pred": "2", "correct": true}
+{"id": "gsm8k_test_1031", "task": "gsm8k", "prompt_sha256": "84e657715c7a5417", "content": "To determine how many tickets each graduate would receive to give to their friends and family, let's break down the problem step by step.\n\n1. **Total capacity of the Fine Arts Center**: 6000 people.\n2. **Seats reserved for graduates**: 950.\n3. **Seats reserved for faculty**: 300.\n\nFirst, subtract the seats reserved for graduates and faculty from the total capacity to find the remaining seats available for friends and family:\n\\[\n6000 - 950 - 300 = 4750 \\text{ seats}\n\\]\n\nNext, divide the remaining seats equally among the 950 graduates to find out how many tickets each graduate would receive:\n\\[\n\\frac{4750}{950} = 5\n\\]\n\nSo, each graduate would receive 5 tickets to give to their friends and family.\n\nThe final answer is 5.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 112, "total_tokens": 317, "completion_tokens": 205, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 205, "latency_s": 20.304, "error": null, "agree_key": "5", "pred": "5", "correct": true}
+{"id": "gsm8k_test_1033", "task": "gsm8k", "prompt_sha256": "74ab7f526b51acfb", "content": "Let me solve this step by step.\n\n**Step 1: Calculate total hours spent**\n- 6 hours per day × 3 days = 18 hours total\n\n**Step 2: Calculate total pay**\n- $15 per day × 3 days = $45 total pay\n\n**Step 3: Calculate total parking costs**\n- $3 per day × 3 days = $9 total parking costs\n\n**Step 4: Calculate net pay after expenses**\n- $45 - $9 = $36 net pay\n\n**Step 5: Calculate pay per hour**\n- $36 ÷ 18 hours = $2 per hour\n\nThe final answer is $2", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 104, "total_tokens": 255, "completion_tokens": 151, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 151, "latency_s": 13.06, "error": null, "agree_key": "2", "pred": "2", "correct": true}
+{"id": "gsm8k_test_1041", "task": "gsm8k", "prompt_sha256": "4315b6cfefd824fc", "content": "Let me solve this step by step.\n\n**Given information:**\n- 10 people want chicken salad at $6.50 per person\n- 6 people want pasta salad at $6 per person\n\n**Step 1: Calculate the cost for chicken salad**\n10 people × $6.50 = $65.00\n\n**Step 2: Calculate the cost for pasta salad**\n6 people × $6.00 = $36.00\n\n**Step 3: Calculate the total amount**\n$65.00 + $36.00 = $101.00\n\nThe final answer is 101", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 106, "total_tokens": 251, "completion_tokens": 145, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 145, "latency_s": 11.508, "error": null, "agree_key": "101", "pred": "101", "correct": true}
+{"id": "gsm8k_test_1052", "task": "gsm8k", "prompt_sha256": "709435e82006f688", "content": "Let me solve this step by step.\n\n**Initial boarding:**\nRomeo boards a train with 120 people.\nTotal = 120 people\n\n**First stop:**\n20 more people board the train.\nTotal = 120 + 20 = 140 people\n\n**Second stop:**\n- 50 people descend from the train\n- Twice that number (2 × 50 = 100) board the train\n\nTotal = 140 - 50 + 100 = 190 people\n\n**Third stop:**\n80 more people descend.\nTotal = 190 - 80 = 110 people\n\nThe final answer is 110", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 120, "total_tokens": 282, "completion_tokens": 162, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 162, "latency_s": 14.772, "error": null, "agree_key": "110", "pred": "110", "correct": true}
+{"id": "gsm8k_test_1071", "task": "gsm8k", "prompt_sha256": "8bb686f57c69e559", "content": "Let me solve this step by step.\n\n**Step 1: Calculate the total number of invited guests.**\n- 25 classmates\n- 7 girls in dance class\n- 13 family members\n\nTotal invited = 25 + 7 + 13 = 45 people\n\n**Step 2: Subtract those who couldn't come.**\n45 - 4 = 41 people attending\n\n**Step 3: Calculate the cost.**\n- Base fee for 20 people: $125\n- Additional guests beyond 20: 41 - 20 = 21 additional guests\n- Cost for additional guests: 21 × $6 = $126\n\n**Step 4: Total cost.**\n$125 + $126 = $251\n\nThe final answer is 251", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 129, "total_tokens": 314, "completion_tokens": 185, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 185, "latency_s": 17.403, "error": null, "agree_key": "251", "pred": "251", "correct": true}
+{"id": "gsm8k_test_1087", "task": "gsm8k", "prompt_sha256": "d9a1deaa5480cfeb", "content": "Let me solve this step by step.\n\n**Given information:**\n- The farmer makes 5 bales per hour\n- The truck picks up 3 bales per hour\n- They work for 6 hours\n\n**Step 1: Calculate how many bales the farmer makes in 6 hours.**\n5 bales/hour × 6 hours = 30 bales\n\n**Step 2: Calculate how many bales the truck picks up in 6 hours.**\n3 bales/hour × 6 hours = 18 bales\n\n**Step 3: Calculate the net number of bales left in the field.**\n30 bales - 18 bales = 12 bales\n\nThe final answer is 12", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 121, "total_tokens": 280, "completion_tokens": 159, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 159, "latency_s": 14.138, "error": null, "agree_key": "12", "pred": "12", "correct": true}
+{"id": "gsm8k_test_1089", "task": "gsm8k", "prompt_sha256": "e4a7cf437940c8ba", "content": "Let me solve this step by step.\n\n**Step 1: Find the starting height.**\nThe ball is dropped from the third-floor balcony, where each story is 24 feet high.\nStarting height = 3 × 24 = 72 feet\n\n**Step 2: Find the height after the first bounce.**\nThe ball bounces to 2/3 of its starting height.\nAfter first bounce: 72 × (2/3) = 48 feet\n\n**Step 3: Find the height after the second bounce.**\nThe ball bounces to 2/3 of its previous height.\nAfter second bounce: 48 × (2/3) = 32 feet\n\nThe final answer is 32.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 105, "total_tokens": 266, "completion_tokens": 161, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 161, "latency_s": 14.588, "error": null, "agree_key": "32", "pred": "32", "correct": true}
+{"id": "gsm8k_test_1094", "task": "gsm8k", "prompt_sha256": "2daf707f0d979fa9", "content": "Let me work through this step by step.\n\n**Initial Setup:**\n- 12 sodas total\n- 3 shaken sodas\n- 9 unshaken sodas\n\n**Step 1: Ellen takes 1 unshaken soda**\n- Remaining: 3 shaken, 8 unshaken (11 total)\n\n**Step 2: Ellen's brother takes 1 shaken and 2 unshaken sodas**\n- Remaining: 2 shaken, 6 unshaken (8 total)\n\n**Step 3: Ellen's friend takes 1 soda from the remaining 8**\n- Probability of getting a shaken soda = 2/8 = 1/4 = 25%\n\nThe final answer is 25%", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 144, "total_tokens": 306, "completion_tokens": 162, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 162, "latency_s": 15.363, "error": null, "agree_key": "25", "pred": "25", "correct": true}
+{"id": "gsm8k_test_1109", "task": "gsm8k", "prompt_sha256": "e3de44efbb758d5c", "content": "To find the average height of the heels at the cocktail party, let's follow these steps:\n\n1. **Identify the number of women and their heel heights:**\n   - 3 women are wearing 4-inch heels.\n   - 3 women are wearing 2-inch heels.\n\n2. **Calculate the total height of all heels:**\n   - For the 4-inch heels: \\(3 \\times 4 = 12\\) inches.\n   - For the 2-inch heels: \\(3 \\times 2 = 6\\) inches.\n   - Total height: \\(12 + 6 = 18\\) inches.\n\n3. **Calculate the total number of women:**\n   - \\(3 + 3 = 6\\) women.\n\n4. **Compute the average height:**\n   - Average height = Total height / Total number of women\n   - Average height = \\(18 / 6 = 3\\) inches.\n\nThe final answer is 3.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 80, "total_tokens": 290, "completion_tokens": 210, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 210, "latency_s": 17.264, "error": null, "agree_key": "3", "pred": "3", "correct": true}
+{"id": "gsm8k_test_1142", "task": "gsm8k", "prompt_sha256": "dd333c70d99b016a", "content": "Let me solve this step by step.\n\n**Given information:**\n- Green jellybeans: 17\n- Red jellybeans: twice as many as green = 2 × 17 = 34\n- Total jellybeans: 60\n- The rest are blue\n\n**Step 1: Find the total number of green and red jellybeans**\nGreen + Red = 17 + 34 = 51\n\n**Step 2: Find the number of blue jellybeans**\nBlue = Total - (Green + Red)\nBlue = 60 - 51 = 9\n\nThe final answer is 9", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 114, "total_tokens": 251, "completion_tokens": 137, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 137, "latency_s": 12.467, "error": null, "agree_key": "9", "pred": "9", "correct": true}
+{"id": "gsm8k_test_1183", "task": "gsm8k", "prompt_sha256": "5b743ca7554a77c7", "content": "Let me trace through this step by step.\n\n**Step 1:** Two apples fell out of the tree.\n- Apples on ground: 2\n- Newton picks up both apples.\n\n**Step 2:** Newton puts one in his pocket and throws the other into the tree.\n- Apples in pocket: 1\n- Throwing one apple into the tree causes 5 more apples to fall out.\n- Apples on ground: 5\n\n**Step 3:** Newton picks up the 5 apples, puts all but one in his pocket, and throws the remaining one at the tree.\n- Apples in pocket: 1 + 4 = 5\n- Throwing one apple at the tree causes 6 more apples to fall out.\n- Apples on ground: 6\n\n**Step 4:** Newton picks up all of the remaining apples (6) and goes home to make a pie.\n- Apples brought home: 6\n\nThe final answer is 6", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 161, "total_tokens": 372, "completion_tokens": 211, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 211, "latency_s": 16.469, "error": null, "agree_key": "6", "pred": "6", "correct": false}
+{"id": "gsm8k_test_1208", "task": "gsm8k", "prompt_sha256": "debf777bb07521b2", "content": "Let me solve this step by step.\n\n**Step 1: Find the selling price**\n\nTrinity sells magazines at 11/8 of the price she bought them at.\n\nSelling price = (11/8) × $72\n\nSelling price = (11 × 72) / 8\n\nSelling price = 792 / 8\n\nSelling price = $99\n\n**Step 2: Calculate the profit**\n\nProfit = Selling price - Cost price\n\nProfit = $99 - $72\n\nProfit = $27\n\nThe final answer is 27", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 79, "total_tokens": 211, "completion_tokens": 132, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 132, "latency_s": 11.356, "error": null, "agree_key": "27", "pred": "27", "correct": true}
+{"id": "gsm8k_test_1215", "task": "gsm8k", "prompt_sha256": "e30644d8de25e2b8", "content": "Let me solve this step by step.\n\n**Step 1: Calculate the volume of the jar.**\nThe jar has a square base that is 6 inches by 6 inches and is 15 inches tall.\nVolume = 6 × 6 × 15 = 540 cubic inches\n\n**Step 2: Calculate the effective volume for jelly beans.**\nWith 80% packing efficiency:\nEffective volume = 540 × 0.80 = 432 cubic inches\n\n**Step 3: Calculate the number of jelly beans.**\nEach jelly bean is 0.15 cubic inches.\nNumber of jelly beans = 432 ÷ 0.15 = 2,880 jelly beans\n\n**Step 4: Calculate the estimated number of red jelly beans.**\n30% are red:\nEstimated red jelly beans = 2,880 × 0.30 = 864\n\n**Step 5: Calculate how far off the guess was.**\nTrue number of red jelly beans = 900\nDifference = 900 - 864 = 36\n\nThe final answer is 36", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 158, "total_tokens": 414, "completion_tokens": 256, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 256, "latency_s": 18.285, "error": null, "agree_key": "36", "pred": "36", "correct": true}
+{"id": "gsm8k_test_1227", "task": "gsm8k", "prompt_sha256": "cd5b8ce93a58cd17", "content": "Let's break down the problem step by step:\n\n1. **First part:** Susan orders 3 magazines that send 12 issues a year each.\n   - Total issues from these magazines: \\(3 \\times 12 = 36\\) issues per year.\n\n2. **Second part:** She has 1 magazine that sends 6 issues a year.\n   - Total issues from this magazine: \\(6\\) issues per year.\n\n3. **Third part:** Her last magazine sends 4 times the amount of the 6-issue magazine.\n   - Issues from this magazine: \\(4 \\times 6 = 24\\) issues per year.\n\n4. **Total issues per year:**\n   - Add up all the issues: \\(36 + 6 + 24 = 66\\) issues per year.\n\nThe final answer is 66.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 102, "total_tokens": 288, "completion_tokens": 186, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 186, "latency_s": 14.802, "error": null, "agree_key": "66", "pred": "66", "correct": true}
+{"id": "gsm8k_test_1246", "task": "gsm8k", "prompt_sha256": "3abcf88d70a3934f", "content": "To calculate the total number of points Ara has scored over four years, let's break it down step by step:\n\n1. **Number of games per year**: Ara plays 40 games every year.\n2. **Number of years**: She has been playing for 4 years.\n3. **Total number of games**:  \n   \\( 40 \\text{ games/year} \\times 4 \\text{ years} = 160 \\text{ games} \\)\n4. **Points per game**: She scores 21 points in each game.\n5. **Total points scored**:  \n   \\( 160 \\text{ games} \\times 21 \\text{ points/game} = 3360 \\text{ points} \\)\n\nThe final answer is 3360.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 94, "total_tokens": 269, "completion_tokens": 175, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 175, "latency_s": 13.35, "error": null, "agree_key": "3360", "pred": "3360", "correct": true}
+{"id": "gsm8k_test_1249", "task": "gsm8k", "prompt_sha256": "6335134b069cc007", "content": "Let me break this problem down step by step.\n\n**Flashlights:**\n- There are 4 people, and each person gets 1 flashlight: 4 × 1 = 4 flashlights\n- There are 8 rooms, and each room gets 2 flashlights: 8 × 2 = 16 flashlights\n- Total flashlights: 4 + 16 = 20 flashlights\n\n**Candles:**\n- There are 8 rooms total, so half the rooms = 4 rooms\n- 4 small candles for each of the first 4 rooms: 4 × 4 = 16 candles\n- 5 medium candles for each of the other 4 rooms: 4 × 5 = 20 candles\n- Total candles: 16 + 20 = 36 candles\n\n**Total items:**\n- Flashlights + Candles = 20 + 36 = 56\n\nThe final answer is 56", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 157, "total_tokens": 366, "completion_tokens": 209, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 209, "latency_s": 14.502, "error": null, "agree_key": "56", "pred": "56", "correct": true}
+{"id": "gsm8k_test_1259", "task": "gsm8k", "prompt_sha256": "a7c04cbb8a942ebe", "content": "Let me solve this step by step.\n\n**Step 1: Find the number of doughnuts made per day.**\n- 10 dozen doughnuts per day\n- 1 dozen = 12 doughnuts\n- 10 × 12 = 120 doughnuts per day\n\n**Step 2: Find the revenue per day.**\n- Selling price: $2 per doughnut\n- 120 × $2 = $240 per day\n\n**Step 3: Find the number of days in June.**\n- June has 30 days\n\n**Step 4: Calculate total revenue for June.**\n- $240 × 30 = $7,200\n\nThe final answer is 7200", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 86, "total_tokens": 249, "completion_tokens": 163, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 163, "latency_s": 11.752, "error": null, "agree_key": "7200", "pred": "7200", "correct": true}
+{"id": "gsm8k_test_1272", "task": "gsm8k", "prompt_sha256": "d324c83dc34c3754", "content": "Let me solve this step by step.\n\n**Step 1: Calculate the hotel cost**\n- Helen stayed for 3 nights\n- Each night costs $80\n- Hotel cost = 3 × $80 = $240\n\n**Step 2: Calculate the cost of one bus trip**\n- Each bus trip costs 10% of one night at the hotel\n- Bus trip cost = 10% × $80 = $8\n\n**Step 3: Calculate the total bus cost**\n- Helen rode the bus 7 times\n- Total bus cost = 7 × $8 = $56\n\n**Step 4: Calculate the total cost**\n- Total = Hotel cost + Bus cost\n- Total = $240 + $56 = $296\n\nThe final answer is 296", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 121, "total_tokens": 302, "completion_tokens": 181, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 181, "latency_s": 12.759, "error": null, "agree_key": "296", "pred": "296", "correct": true}
+{"id": "gsm8k_test_1298", "task": "gsm8k", "prompt_sha256": "6e69630c819427da", "content": "To find the total dollar amount in Justin's jar, we'll calculate the value of each type of coin and then sum them up.\n\n1. **Quarters:**\n   - Each quarter is worth \\$0.25.\n   - 32 quarters: \\( 32 \\times 0.25 = 8 \\) dollars.\n\n2. **Dimes:**\n   - Each dime is worth \\$0.10.\n   - 95 dimes: \\( 95 \\times 0.10 = 9.50 \\) dollars.\n\n3. **Nickels:**\n   - Each nickel is worth \\$0.05.\n   - 120 nickels: \\( 120 \\times 0.05 = 6 \\) dollars.\n\n4. **Pennies:**\n   - Each penny is worth \\$0.01.\n   - 750 pennies: \\( 750 \\times 0.01 = 7.50 \\) dollars.\n\nNow, add all the amounts together:\n\\[\n8 + 9.50 + 6 + 7.50 = 31\n\\]\n\nThe final answer is 31.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 95, "total_tokens": 364, "completion_tokens": 269, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 269, "latency_s": 16.05, "error": null, "agree_key": "31", "pred": "31", "correct": true}
+{"id": "gsm8k_test_1312", "task": "gsm8k", "prompt_sha256": "2db6418d0301399f", "content": "Let me solve this step by step.\n\n**Step 1: Calculate the revenue per reservation per night**\n\n- 2 meals at $5 each: 2 × $5 = $10\n- 1 bottle of wine: $5\n- Total per reservation: $10 + $5 = $15\n\n**Step 2: Calculate the revenue per night**\n\n- 6 reservations × $15 = $90 per night\n\n**Step 3: Calculate the weekly revenue**\n\n- Open 2 days a week: $90 × 2 = $180\n\nThe final answer is 180", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 94, "total_tokens": 229, "completion_tokens": 135, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 135, "latency_s": 9.692, "error": null, "agree_key": "180", "pred": "180", "correct": true}

runtime-v2/runs/final_frozen_v1/eval/raw_humaneval.jsonl

@@ -0,0 +1,40 @@
+{"id": "HumanEval_6", "task": "humaneval", "prompt_sha256": "343ac82f24475f94", "content": "```python\nfrom typing import List\n\n\ndef parse_nested_parens(paren_string: str) -> List[int]:\n    \"\"\" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.\n    For each of the group, output the deepest level of nesting of parentheses.\n    E.g. (()()) has maximum two levels of nesting while ((())) has three.\n\n    >>> parse_nested_parens('(()()) ((())) () ((())()())')\n    [2, 3, 1, 3]\n    \"\"\"\n    groups = paren_string.split()\n    result = []\n    \n    for group in groups:\n        max_depth = 0\n        current_depth = 0\n        for char in group:\n            if char == '(':\n                current_depth += 1\n                max_depth = max(max_depth, current_depth)\n            elif char == ')':\n                current_depth -= 1\n        result.append(max_depth)\n    \n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 158, "total_tokens": 370, "completion_tokens": 212, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 212, "latency_s": 19.508, "error": null, "agree_key": "from typing import List\n\n\n\nfrom typing import List\n\n\ndef parse_nested_parens(paren_string: str) -> List[int]:\n    \"\"\" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.\n    For each of the group, output the deepest level of nesting of parentheses.\n    E.g. (()()) has maximum two levels of nesting while ((())) has three.\n\n    >>> parse_nested_parens('(()()) ((())) () ((())()())')\n    [2, 3, 1, 3]\n    \"\"\"\n    groups = paren_string.split()\n    result = []\n    \n    for group in groups:\n        max_depth = 0\n        current_depth = 0\n        for char in group:\n            if char == '(':\n                current_depth += 1\n                max_depth = max(max_depth, current_depth)\n            elif char == ')':\n                current_depth -= 1\n        result.append(max_depth)\n    \n    return result", "pred": "from typing import List\n\n\n\nfrom typing import List\n\n\ndef parse_nested_parens(paren_string: str) -> List[int]:\n    \"\"\" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.\n    For each of the group, output the deepest level of nesting of parentheses.\n    E.g. (()()) has maximum two levels of nesting while ((())) has three.\n\n    >>> parse_nested_parens('(()()) ((())) () ((())()())')\n    [2, 3, 1, 3]\n    \"\"\"\n    groups = paren_string.split()\n    result = []\n    \n    for group in groups:\n        max_depth = 0\n        current_depth = 0\n        for char in group:\n            if char == '(':\n                current_depth += 1\n                max_depth = max(max_depth, current_depth)\n            elif char == ')':\n                current_depth -= 1\n        result.append(max_depth)\n    \n    return result\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_9", "task": "humaneval", "prompt_sha256": "26b624f47688b5a8", "content": "```python\nfrom typing import List, Tuple\n\n\ndef rolling_max(numbers: List[int]) -> List[int]:\n    \"\"\" From a given list of integers, generate a list of rolling maximum element found until given moment\n    in the sequence.\n    >>> rolling_max([1, 2, 3, 2, 3, 4, 2])\n    [1, 2, 3, 3, 3, 4, 4]\n    \"\"\"\n    result = []\n    current_max = float('-inf')\n    for num in numbers:\n        if num > current_max:\n            current_max = num\n        result.append(current_max)\n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 144, "total_tokens": 294, "completion_tokens": 150, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 150, "latency_s": 13.618, "error": null, "agree_key": "from typing import List, Tuple\n\n\n\nfrom typing import List, Tuple\n\n\ndef rolling_max(numbers: List[int]) -> List[int]:\n    \"\"\" From a given list of integers, generate a list of rolling maximum element found until given moment\n    in the sequence.\n    >>> rolling_max([1, 2, 3, 2, 3, 4, 2])\n    [1, 2, 3, 3, 3, 4, 4]\n    \"\"\"\n    result = []\n    current_max = float('-inf')\n    for num in numbers:\n        if num > current_max:\n            current_max = num\n        result.append(current_max)\n    return result", "pred": "from typing import List, Tuple\n\n\n\nfrom typing import List, Tuple\n\n\ndef rolling_max(numbers: List[int]) -> List[int]:\n    \"\"\" From a given list of integers, generate a list of rolling maximum element found until given moment\n    in the sequence.\n    >>> rolling_max([1, 2, 3, 2, 3, 4, 2])\n    [1, 2, 3, 3, 3, 4, 4]\n    \"\"\"\n    result = []\n    current_max = float('-inf')\n    for num in numbers:\n        if num > current_max:\n            current_max = num\n        result.append(current_max)\n    return result\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_11", "task": "humaneval", "prompt_sha256": "28538f58ee858d11", "content": "```python\nfrom typing import List\n\n\ndef string_xor(a: str, b: str) -> str:\n    \"\"\" Input are two strings a and b consisting only of 1s and 0s.\n    Perform binary XOR on these inputs and return result also as a string.\n    >>> string_xor('010', '110')\n    '100'\n    \"\"\"\n    return ''.join(str(int(x) ^ int(y)) for x, y in zip(a, b))\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 127, "total_tokens": 237, "completion_tokens": 110, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 110, "latency_s": 10.578, "error": null, "agree_key": "from typing import List\n\n\n\nfrom typing import List\n\n\ndef string_xor(a: str, b: str) -> str:\n    \"\"\" Input are two strings a and b consisting only of 1s and 0s.\n    Perform binary XOR on these inputs and return result also as a string.\n    >>> string_xor('010', '110')\n    '100'\n    \"\"\"\n    return ''.join(str(int(x) ^ int(y)) for x, y in zip(a, b))", "pred": "from typing import List\n\n\n\nfrom typing import List\n\n\ndef string_xor(a: str, b: str) -> str:\n    \"\"\" Input are two strings a and b consisting only of 1s and 0s.\n    Perform binary XOR on these inputs and return result also as a string.\n    >>> string_xor('010', '110')\n    '100'\n    \"\"\"\n    return ''.join(str(int(x) ^ int(y)) for x, y in zip(a, b))\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_16", "task": "humaneval", "prompt_sha256": "115626b4d45b8485", "content": "```python\ndef count_distinct_characters(string: str) -> int:\n    \"\"\" Given a string, find out how many distinct characters (regardless of case) does it consist of\n    >>> count_distinct_characters('xyzXYZ')\n    3\n    >>> count_distinct_characters('Jerry')\n    4\n    \"\"\"\n    return len(set(string.lower()))\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 113, "total_tokens": 194, "completion_tokens": 81, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 81, "latency_s": 7.606, "error": null, "agree_key": "def count_distinct_characters(string: str) -> int:\n    \"\"\" Given a string, find out how many distinct characters (regardless of case) does it consist of\n    >>> count_distinct_characters('xyzXYZ')\n    3\n    >>> count_distinct_characters('Jerry')\n    4\n    \"\"\"\n    return len(set(string.lower()))", "pred": "\n\n\ndef count_distinct_characters(string: str) -> int:\n    \"\"\" Given a string, find out how many distinct characters (regardless of case) does it consist of\n    >>> count_distinct_characters('xyzXYZ')\n    3\n    >>> count_distinct_characters('Jerry')\n    4\n    \"\"\"\n    return len(set(string.lower()))\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_21", "task": "humaneval", "prompt_sha256": "0d413b3455e2e25c", "content": "```python\nfrom typing import List\n\n\ndef rescale_to_unit(numbers: List[float]) -> List[float]:\n    \"\"\" Given list of numbers (of at least two elements), apply a linear transform to that list,\n    such that the smallest number will become 0 and the largest will become 1\n    >>> rescale_to_unit([1.0, 2.0, 3.0, 4.0, 5.0])\n    [0.0, 0.25, 0.5, 0.75, 1.0]\n    \"\"\"\n    min_val = min(numbers)\n    max_val = max(numbers)\n    range_val = max_val - min_val\n    if range_val == 0:\n        return [0.0] * len(numbers)\n    return [(x - min_val) / range_val for x in numbers]\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 169, "total_tokens": 362, "completion_tokens": 193, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 193, "latency_s": 16.002, "error": null, "agree_key": "from typing import List\n\n\n\nfrom typing import List\n\n\ndef rescale_to_unit(numbers: List[float]) -> List[float]:\n    \"\"\" Given list of numbers (of at least two elements), apply a linear transform to that list,\n    such that the smallest number will become 0 and the largest will become 1\n    >>> rescale_to_unit([1.0, 2.0, 3.0, 4.0, 5.0])\n    [0.0, 0.25, 0.5, 0.75, 1.0]\n    \"\"\"\n    min_val = min(numbers)\n    max_val = max(numbers)\n    range_val = max_val - min_val\n    if range_val == 0:\n        return [0.0] * len(numbers)\n    return [(x - min_val) / range_val for x in numbers]", "pred": "from typing import List\n\n\n\nfrom typing import List\n\n\ndef rescale_to_unit(numbers: List[float]) -> List[float]:\n    \"\"\" Given list of numbers (of at least two elements), apply a linear transform to that list,\n    such that the smallest number will become 0 and the largest will become 1\n    >>> rescale_to_unit([1.0, 2.0, 3.0, 4.0, 5.0])\n    [0.0, 0.25, 0.5, 0.75, 1.0]\n    \"\"\"\n    min_val = min(numbers)\n    max_val = max(numbers)\n    range_val = max_val - min_val\n    if range_val == 0:\n        return [0.0] * len(numbers)\n    return [(x - min_val) / range_val for x in numbers]\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_23", "task": "humaneval", "prompt_sha256": "12333ed9b6291977", "content": "```python\ndef strlen(string: str) -> int:\n    \"\"\" Return length of given string\n    >>> strlen('')\n    0\n    >>> strlen('abc')\n    3\n    \"\"\"\n    return len(string)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 87, "total_tokens": 140, "completion_tokens": 53, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 53, "latency_s": 4.553, "error": null, "agree_key": "def strlen(string: str) -> int:\n    \"\"\" Return length of given string\n    >>> strlen('')\n    0\n    >>> strlen('abc')\n    3\n    \"\"\"\n    return len(string)", "pred": "\n\n\ndef strlen(string: str) -> int:\n    \"\"\" Return length of given string\n    >>> strlen('')\n    0\n    >>> strlen('abc')\n    3\n    \"\"\"\n    return len(string)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_30", "task": "humaneval", "prompt_sha256": "011424fa9059e02a", "content": "```python\ndef get_positive(l: list):\n    \"\"\"Return only positive numbers in the list.\n    >>> get_positive([-1, 2, -4, 5, 6])\n    [2, 5, 6]\n    >>> get_positive([5, 3, -5, 2, -3, 3, 9, 0, 123, 1, -10])\n    [5, 3, 2, 3, 9, 123, 1]\n    \"\"\"\n    return [x for x in l if x > 0]\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 164, "total_tokens": 303, "completion_tokens": 139, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 139, "latency_s": 12.066, "error": null, "agree_key": "def get_positive(l: list):\n    \"\"\"Return only positive numbers in the list.\n    >>> get_positive([-1, 2, -4, 5, 6])\n    [2, 5, 6]\n    >>> get_positive([5, 3, -5, 2, -3, 3, 9, 0, 123, 1, -10])\n    [5, 3, 2, 3, 9, 123, 1]\n    \"\"\"\n    return [x for x in l if x > 0]", "pred": "\n\n\ndef get_positive(l: list):\n    \"\"\"Return only positive numbers in the list.\n    >>> get_positive([-1, 2, -4, 5, 6])\n    [2, 5, 6]\n    >>> get_positive([5, 3, -5, 2, -3, 3, 9, 0, 123, 1, -10])\n    [5, 3, 2, 3, 9, 123, 1]\n    \"\"\"\n    return [x for x in l if x > 0]\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_34", "task": "humaneval", "prompt_sha256": "937d51026f04595c", "content": "```python\ndef unique(l: list):\n    \"\"\"Return sorted unique elements in a list\n    >>> unique([5, 3, 5, 2, 3, 3, 9, 0, 123])\n    [0, 2, 3, 5, 9, 123]\n    \"\"\"\n    return sorted(set(l))\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 120, "total_tokens": 207, "completion_tokens": 87, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 87, "latency_s": 7.81, "error": null, "agree_key": "def unique(l: list):\n    \"\"\"Return sorted unique elements in a list\n    >>> unique([5, 3, 5, 2, 3, 3, 9, 0, 123])\n    [0, 2, 3, 5, 9, 123]\n    \"\"\"\n    return sorted(set(l))", "pred": "\n\n\ndef unique(l: list):\n    \"\"\"Return sorted unique elements in a list\n    >>> unique([5, 3, 5, 2, 3, 3, 9, 0, 123])\n    [0, 2, 3, 5, 9, 123]\n    \"\"\"\n    return sorted(set(l))\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_37", "task": "humaneval", "prompt_sha256": "18018d80d1df5c6d", "content": "Looking at this problem, I need to:\n1. Extract values at even indices (0, 2, 4, ...)\n2. Sort those values\n3. Put them back at the even indices\n4. Keep odd indices unchanged\n\nLet me trace through the examples:\n- `[1, 2, 3]`: even indices are 0,2 → values [1,3] → sorted [1,3] → result [1,2,3]\n- `[5, 6, 3, 4]`: even indices are 0,2 → values [5,3] → sorted [3,5] → result [3,6,5,4]\n\n```python\ndef sort_even(l: list):\n    \"\"\"This function takes a list l and returns a list l' such that\n    l' is identical to l in the odd indicies, while its values at the even indicies are equal\n    to the values of the even indicies of l, but sorted.\n    >>> sort_even([1, 2, 3])\n    [1, 2, 3]\n    >>> sort_even([5, 6, 3, 4])\n    [3, 6, 5, 4]\n    \"\"\"\n    even_values = sorted([l[i] for i in range(0, len(l), 2)])\n    result = l[:]\n    for i, val in zip(range(0, len(l), 2), even_values):\n        result[i] = val\n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 169, "total_tokens": 507, "completion_tokens": 338, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 338, "latency_s": 30.981, "error": null, "agree_key": "def sort_even(l: list):\n    \"\"\"This function takes a list l and returns a list l' such that\n    l' is identical to l in the odd indicies, while its values at the even indicies are equal\n    to the values of the even indicies of l, but sorted.\n    >>> sort_even([1, 2, 3])\n    [1, 2, 3]\n    >>> sort_even([5, 6, 3, 4])\n    [3, 6, 5, 4]\n    \"\"\"\n    even_values = sorted([l[i] for i in range(0, len(l), 2)])\n    result = l[:]\n    for i, val in zip(range(0, len(l), 2), even_values):\n        result[i] = val\n    return result", "pred": "\n\n\ndef sort_even(l: list):\n    \"\"\"This function takes a list l and returns a list l' such that\n    l' is identical to l in the odd indicies, while its values at the even indicies are equal\n    to the values of the even indicies of l, but sorted.\n    >>> sort_even([1, 2, 3])\n    [1, 2, 3]\n    >>> sort_even([5, 6, 3, 4])\n    [3, 6, 5, 4]\n    \"\"\"\n    even_values = sorted([l[i] for i in range(0, len(l), 2)])\n    result = l[:]\n    for i, val in zip(range(0, len(l), 2), even_values):\n        result[i] = val\n    return result\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_38", "task": "humaneval", "prompt_sha256": "94f8935b0049ccdc", "content": "```python\ndef decode_cyclic(s: str):\n    \"\"\"\n    takes as input string encoded with encode_cyclic function. Returns decoded string.\n    \"\"\"\n    # split string to groups. Each of length 3.\n    groups = [s[(3 * i):min((3 * i + 3), len(s))] for i in range((len(s) + 2) // 3)]\n    # reverse the cyclic shift in each group. Unless group has fewer elements than 3.\n    groups = [(group[-1] + group[:-1]) if len(group) == 3 else group for group in groups]\n    return \"\".join(groups)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 210, "total_tokens": 355, "completion_tokens": 145, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 145, "latency_s": 12.771, "error": null, "agree_key": "def encode_cyclic(s: str):\n    \"\"\"\n    returns encoded string by cycling groups of three characters.\n    \"\"\"\n    # split string to groups. Each of length 3.\n    groups = [s[(3 * i):min((3 * i + 3), len(s))] for i in range((len(s) + 2) // 3)]\n    # cycle elements in each group. Unless group has fewer elements than 3.\n    groups = [(group[1:] + group[0]) if len(group) == 3 else group for group in groups]\n    return \"\".join(groups)\n\n\n\ndef decode_cyclic(s: str):\n    \"\"\"\n    takes as input string encoded with encode_cyclic function. Returns decoded string.\n    \"\"\"\n    # split string to groups. Each of length 3.\n    groups = [s[(3 * i):min((3 * i + 3), len(s))] for i in range((len(s) + 2) // 3)]\n    # reverse the cyclic shift in each group. Unless group has fewer elements than 3.\n    groups = [(group[-1] + group[:-1]) if len(group) == 3 else group for group in groups]\n    return \"\".join(groups)", "pred": "\n\ndef encode_cyclic(s: str):\n    \"\"\"\n    returns encoded string by cycling groups of three characters.\n    \"\"\"\n    # split string to groups. Each of length 3.\n    groups = [s[(3 * i):min((3 * i + 3), len(s))] for i in range((len(s) + 2) // 3)]\n    # cycle elements in each group. Unless group has fewer elements than 3.\n    groups = [(group[1:] + group[0]) if len(group) == 3 else group for group in groups]\n    return \"\".join(groups)\n\n\n\ndef decode_cyclic(s: str):\n    \"\"\"\n    takes as input string encoded with encode_cyclic function. Returns decoded string.\n    \"\"\"\n    # split string to groups. Each of length 3.\n    groups = [s[(3 * i):min((3 * i + 3), len(s))] for i in range((len(s) + 2) // 3)]\n    # reverse the cyclic shift in each group. Unless group has fewer elements than 3.\n    groups = [(group[-1] + group[:-1]) if len(group) == 3 else group for group in groups]\n    return \"\".join(groups)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_43", "task": "humaneval", "prompt_sha256": "a0c0121c6ae10f5b", "content": "```python\ndef pairs_sum_to_zero(l):\n    \"\"\"\n    pairs_sum_to_zero takes a list of integers as an input.\n    it returns True if there are two distinct elements in the list that\n    sum to zero, and False otherwise.\n    >>> pairs_sum_to_zero([1, 3, 5, 0])\n    False\n    >>> pairs_sum_to_zero([1, 3, -2, 1])\n    False\n    >>> pairs_sum_to_zero([1, 2, 3, 7])\n    False\n    >>> pairs_sum_to_zero([2, 4, -5, 3, 5, 7])\n    True\n    >>> pairs_sum_to_zero([1])\n    False\n    \"\"\"\n    seen = set()\n    for num in l:\n        if -num in seen:\n            return True\n        seen.add(num)\n    return False\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 206, "total_tokens": 407, "completion_tokens": 201, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 201, "latency_s": 16.627, "error": null, "agree_key": "def pairs_sum_to_zero(l):\n    \"\"\"\n    pairs_sum_to_zero takes a list of integers as an input.\n    it returns True if there are two distinct elements in the list that\n    sum to zero, and False otherwise.\n    >>> pairs_sum_to_zero([1, 3, 5, 0])\n    False\n    >>> pairs_sum_to_zero([1, 3, -2, 1])\n    False\n    >>> pairs_sum_to_zero([1, 2, 3, 7])\n    False\n    >>> pairs_sum_to_zero([2, 4, -5, 3, 5, 7])\n    True\n    >>> pairs_sum_to_zero([1])\n    False\n    \"\"\"\n    seen = set()\n    for num in l:\n        if -num in seen:\n            return True\n        seen.add(num)\n    return False", "pred": "\n\n\ndef pairs_sum_to_zero(l):\n    \"\"\"\n    pairs_sum_to_zero takes a list of integers as an input.\n    it returns True if there are two distinct elements in the list that\n    sum to zero, and False otherwise.\n    >>> pairs_sum_to_zero([1, 3, 5, 0])\n    False\n    >>> pairs_sum_to_zero([1, 3, -2, 1])\n    False\n    >>> pairs_sum_to_zero([1, 2, 3, 7])\n    False\n    >>> pairs_sum_to_zero([2, 4, -5, 3, 5, 7])\n    True\n    >>> pairs_sum_to_zero([1])\n    False\n    \"\"\"\n    seen = set()\n    for num in l:\n        if -num in seen:\n            return True\n        seen.add(num)\n    return False\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_45", "task": "humaneval", "prompt_sha256": "d8e7d755c67ae356", "content": "```python\ndef triangle_area(a, h):\n    \"\"\"Given length of a side and high return area for a triangle.\n    >>> triangle_area(5, 3)\n    7.5\n    \"\"\"\n    return 0.5 * a * h\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 89, "total_tokens": 149, "completion_tokens": 60, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 60, "latency_s": 5.436, "error": null, "agree_key": "def triangle_area(a, h):\n    \"\"\"Given length of a side and high return area for a triangle.\n    >>> triangle_area(5, 3)\n    7.5\n    \"\"\"\n    return 0.5 * a * h", "pred": "\n\n\ndef triangle_area(a, h):\n    \"\"\"Given length of a side and high return area for a triangle.\n    >>> triangle_area(5, 3)\n    7.5\n    \"\"\"\n    return 0.5 * a * h\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_53", "task": "humaneval", "prompt_sha256": "c329fe79809417c1", "content": "```python\ndef add(x: int, y: int):\n    \"\"\"Add two numbers x and y\n    >>> add(2, 3)\n    5\n    >>> add(5, 7)\n    12\n    \"\"\"\n    return x + y\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 97, "total_tokens": 160, "completion_tokens": 63, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 63, "latency_s": 5.645, "error": null, "agree_key": "def add(x: int, y: int):\n    \"\"\"Add two numbers x and y\n    >>> add(2, 3)\n    5\n    >>> add(5, 7)\n    12\n    \"\"\"\n    return x + y", "pred": "\n\n\ndef add(x: int, y: int):\n    \"\"\"Add two numbers x and y\n    >>> add(2, 3)\n    5\n    >>> add(5, 7)\n    12\n    \"\"\"\n    return x + y\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_54", "task": "humaneval", "prompt_sha256": "19d03fbbb46b1d7d", "content": "Looking at the examples, I need to understand what \"same characters\" means here:\n\n- `same_chars('eabcdzzzz', 'dddzzzzzzzddeddabc')` → True\n- `same_chars('abcd', 'dddddddabc')` → True\n- `same_chars('eabcd', 'dddddddabc')` → False\n- `same_chars('abcd', 'dddddddabce')` → False\n- `same_chars('eabcdzzzz', 'dddzzzzzzzddddabc')` → False\n\nThe pattern is: two strings have the \"same characters\" if the set of unique characters in both strings is identical.\n\nLet me verify:\n- `eabcdzzzz` → {a,b,c,d,e} and `dddzzzzzzzddeddabc` → {a,b,c,d,e} → True ✓\n- `abcd` → {a,b,c,d} and `dddddddabc` → {a,b,c,d} → True ✓\n- `eabcd` → {a,b,c,d,e} and `dddddddabc` → {a,b,c,d} → False ✓\n- `abcd` → {a,b,c,d} and `dddddddabce` → {a,b,c,d,e} → False ✓\n- `eabcdzzzz` → {a,b,c,d,e} and `dddzzzzzzzddddabc` → {a,b,c,d} → False ✓\n\nSo the solution is to compare the sets of characters in both strings.\n\n```python\ndef same_chars(s0: str, s1: str):\n    \"\"\"\n    Check if two words have the same characters.\n    >>> same_chars('eabcdzzzz', 'dddzzzzzzzddeddabc')\n    True\n    >>> same_chars('abcd', 'dddddddabc')\n    True\n    >>> same_chars('dddddddabc', 'abcd')\n    True\n    >>> same_chars('eabcd', 'dddddddabc')\n    False\n    >>> same_chars('abcd', 'dddddddabce')\n    False\n    >>> same_chars('eabcdzzzz', 'dddzzzzzzzddddabc')\n    False\n    \"\"\"\n    return set(s0) == set(s1)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 189, "total_tokens": 672, "completion_tokens": 483, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 483, "latency_s": 40.856, "error": null, "agree_key": "def same_chars(s0: str, s1: str):\n    \"\"\"\n    Check if two words have the same characters.\n    >>> same_chars('eabcdzzzz', 'dddzzzzzzzddeddabc')\n    True\n    >>> same_chars('abcd', 'dddddddabc')\n    True\n    >>> same_chars('dddddddabc', 'abcd')\n    True\n    >>> same_chars('eabcd', 'dddddddabc')\n    False\n    >>> same_chars('abcd', 'dddddddabce')\n    False\n    >>> same_chars('eabcdzzzz', 'dddzzzzzzzddddabc')\n    False\n    \"\"\"\n    return set(s0) == set(s1)", "pred": "\n\n\ndef same_chars(s0: str, s1: str):\n    \"\"\"\n    Check if two words have the same characters.\n    >>> same_chars('eabcdzzzz', 'dddzzzzzzzddeddabc')\n    True\n    >>> same_chars('abcd', 'dddddddabc')\n    True\n    >>> same_chars('dddddddabc', 'abcd')\n    True\n    >>> same_chars('eabcd', 'dddddddabc')\n    False\n    >>> same_chars('abcd', 'dddddddabce')\n    False\n    >>> same_chars('eabcdzzzz', 'dddzzzzzzzddddabc')\n    False\n    \"\"\"\n    return set(s0) == set(s1)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_59", "task": "humaneval", "prompt_sha256": "a15e12b139e7a7b3", "content": "```python\ndef largest_prime_factor(n: int):\n    \"\"\"Return the largest prime factor of n. Assume n > 1 and is not a prime.\n    >>> largest_prime_factor(13195)\n    29\n    >>> largest_prime_factor(2048)\n    2\n    \"\"\"\n    largest = 1\n    d = 2\n    while d * d <= n:\n        while n % d == 0:\n            largest = d\n            n //= d\n        d += 1\n    if n > 1:\n        largest = n\n    return largest\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 113, "total_tokens": 250, "completion_tokens": 137, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 137, "latency_s": 12.768, "error": null, "agree_key": "def largest_prime_factor(n: int):\n    \"\"\"Return the largest prime factor of n. Assume n > 1 and is not a prime.\n    >>> largest_prime_factor(13195)\n    29\n    >>> largest_prime_factor(2048)\n    2\n    \"\"\"\n    largest = 1\n    d = 2\n    while d * d <= n:\n        while n % d == 0:\n            largest = d\n            n //= d\n        d += 1\n    if n > 1:\n        largest = n\n    return largest", "pred": "\n\n\ndef largest_prime_factor(n: int):\n    \"\"\"Return the largest prime factor of n. Assume n > 1 and is not a prime.\n    >>> largest_prime_factor(13195)\n    29\n    >>> largest_prime_factor(2048)\n    2\n    \"\"\"\n    largest = 1\n    d = 2\n    while d * d <= n:\n        while n % d == 0:\n            largest = d\n            n //= d\n        d += 1\n    if n > 1:\n        largest = n\n    return largest\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_73", "task": "humaneval", "prompt_sha256": "9a075b0f032fe691", "content": "```python\ndef smallest_change(arr):\n    \"\"\"\n    Given an array arr of integers, find the minimum number of elements that\n    need to be changed to make the array palindromic. A palindromic array is an array that\n    is read the same backwards and forwards. In one change, you can change one element to any other element.\n\n    For example:\n    smallest_change([1,2,3,5,4,7,9,6]) == 4\n    smallest_change([1, 2, 3, 4, 3, 2, 2]) == 1\n    smallest_change([1, 2, 3, 2, 1]) == 0\n    \"\"\"\n    count = 0\n    n = len(arr)\n    for i in range(n // 2):\n        if arr[i] != arr[n - 1 - i]:\n            count += 1\n    return count\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 201, "total_tokens": 411, "completion_tokens": 210, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 210, "latency_s": 17.607, "error": null, "agree_key": "def smallest_change(arr):\n    \"\"\"\n    Given an array arr of integers, find the minimum number of elements that\n    need to be changed to make the array palindromic. A palindromic array is an array that\n    is read the same backwards and forwards. In one change, you can change one element to any other element.\n\n    For example:\n    smallest_change([1,2,3,5,4,7,9,6]) == 4\n    smallest_change([1, 2, 3, 4, 3, 2, 2]) == 1\n    smallest_change([1, 2, 3, 2, 1]) == 0\n    \"\"\"\n    count = 0\n    n = len(arr)\n    for i in range(n // 2):\n        if arr[i] != arr[n - 1 - i]:\n            count += 1\n    return count", "pred": "\n\ndef smallest_change(arr):\n    \"\"\"\n    Given an array arr of integers, find the minimum number of elements that\n    need to be changed to make the array palindromic. A palindromic array is an array that\n    is read the same backwards and forwards. In one change, you can change one element to any other element.\n\n    For example:\n    smallest_change([1,2,3,5,4,7,9,6]) == 4\n    smallest_change([1, 2, 3, 4, 3, 2, 2]) == 1\n    smallest_change([1, 2, 3, 2, 1]) == 0\n    \"\"\"\n    count = 0\n    n = len(arr)\n    for i in range(n // 2):\n        if arr[i] != arr[n - 1 - i]:\n            count += 1\n    return count\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_74", "task": "humaneval", "prompt_sha256": "6fcbd360d01bc784", "content": "```python\ndef total_match(lst1, lst2):\n    '''\n    Write a function that accepts two lists of strings and returns the list that has \n    total number of chars in the all strings of the list less than the other list.\n\n    if the two lists have the same number of chars, return the first list.\n\n    Examples\n    total_match([], []) ➞ []\n    total_match(['hi', 'admin'], ['hI', 'Hi']) ➞ ['hI', 'Hi']\n    total_match(['hi', 'admin'], ['hi', 'hi', 'admin', 'project']) ➞ ['hi', 'admin']\n    total_match(['hi', 'admin'], ['hI', 'hi', 'hi']) ➞ ['hI', 'hi', 'hi']\n    total_match(['4'], ['1', '2', '3', '4', '5']) ➞ ['4']\n    '''\n    sum1 = sum(len(s) for s in lst1)\n    sum2 = sum(len(s) for s in lst2)\n    return lst1 if sum1 <= sum2 else lst2\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 242, "total_tokens": 488, "completion_tokens": 246, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 246, "latency_s": 19.833, "error": null, "agree_key": "def total_match(lst1, lst2):\n    '''\n    Write a function that accepts two lists of strings and returns the list that has \n    total number of chars in the all strings of the list less than the other list.\n\n    if the two lists have the same number of chars, return the first list.\n\n    Examples\n    total_match([], []) ➞ []\n    total_match(['hi', 'admin'], ['hI', 'Hi']) ➞ ['hI', 'Hi']\n    total_match(['hi', 'admin'], ['hi', 'hi', 'admin', 'project']) ➞ ['hi', 'admin']\n    total_match(['hi', 'admin'], ['hI', 'hi', 'hi']) ➞ ['hI', 'hi', 'hi']\n    total_match(['4'], ['1', '2', '3', '4', '5']) ➞ ['4']\n    '''\n    sum1 = sum(len(s) for s in lst1)\n    sum2 = sum(len(s) for s in lst2)\n    return lst1 if sum1 <= sum2 else lst2", "pred": "\n\ndef total_match(lst1, lst2):\n    '''\n    Write a function that accepts two lists of strings and returns the list that has \n    total number of chars in the all strings of the list less than the other list.\n\n    if the two lists have the same number of chars, return the first list.\n\n    Examples\n    total_match([], []) ➞ []\n    total_match(['hi', 'admin'], ['hI', 'Hi']) ➞ ['hI', 'Hi']\n    total_match(['hi', 'admin'], ['hi', 'hi', 'admin', 'project']) ➞ ['hi', 'admin']\n    total_match(['hi', 'admin'], ['hI', 'hi', 'hi']) ➞ ['hI', 'hi', 'hi']\n    total_match(['4'], ['1', '2', '3', '4', '5']) ➞ ['4']\n    '''\n    sum1 = sum(len(s) for s in lst1)\n    sum2 = sum(len(s) for s in lst2)\n    return lst1 if sum1 <= sum2 else lst2\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_75", "task": "humaneval", "prompt_sha256": "5e2fc8748c93907b", "content": "```python\ndef is_multiply_prime(a):\n    \"\"\"Write a function that returns true if the given number is the multiplication of 3 prime numbers\n    and false otherwise.\n    Knowing that (a) is less then 100. \n    Example:\n    is_multiply_prime(30) == True\n    30 = 2 * 3 * 5\n    \"\"\"\n    def is_prime(n):\n        if n < 2:\n            return False\n        for i in range(2, int(n**0.5) + 1):\n            if n % i == 0:\n                return False\n        return True\n    \n    count = 0\n    temp = a\n    for i in range(2, temp + 1):\n        if temp % i == 0:\n            if is_prime(i):\n                count += 1\n                temp //= i\n            else:\n                return False\n            if temp == 1:\n                break\n    return count == 3\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 126, "total_tokens": 348, "completion_tokens": 222, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 222, "latency_s": 20.329, "error": null, "agree_key": "def is_multiply_prime(a):\n    \"\"\"Write a function that returns true if the given number is the multiplication of 3 prime numbers\n    and false otherwise.\n    Knowing that (a) is less then 100. \n    Example:\n    is_multiply_prime(30) == True\n    30 = 2 * 3 * 5\n    \"\"\"\n    def is_prime(n):\n        if n < 2:\n            return False\n        for i in range(2, int(n**0.5) + 1):\n            if n % i == 0:\n                return False\n        return True\n    \n    count = 0\n    temp = a\n    for i in range(2, temp + 1):\n        if temp % i == 0:\n            if is_prime(i):\n                count += 1\n                temp //= i\n            else:\n                return False\n            if temp == 1:\n                break\n    return count == 3", "pred": "\n\ndef is_multiply_prime(a):\n    \"\"\"Write a function that returns true if the given number is the multiplication of 3 prime numbers\n    and false otherwise.\n    Knowing that (a) is less then 100. \n    Example:\n    is_multiply_prime(30) == True\n    30 = 2 * 3 * 5\n    \"\"\"\n    def is_prime(n):\n        if n < 2:\n            return False\n        for i in range(2, int(n**0.5) + 1):\n            if n % i == 0:\n                return False\n        return True\n    \n    count = 0\n    temp = a\n    for i in range(2, temp + 1):\n        if temp % i == 0:\n            if is_prime(i):\n                count += 1\n                temp //= i\n            else:\n                return False\n            if temp == 1:\n                break\n    return count == 3\n", "correct": false, "exec_error": "Traceback (most recent call last):\n  File \"/tmp/he_qot9pc47/prog.py\", line 48, in <module>\n    check(is_multiply_prime)\n  File \"/tmp/he_qot9pc47/prog.py\", line 37, in check\n    assert candidate(8) == True\n           ^^^^^^^^^^^^^^^^^^^^\nAssertionError\n"}
+{"id": "HumanEval_93", "task": "humaneval", "prompt_sha256": "d202ca4125d033e6", "content": "```python\ndef encode(message):\n    \"\"\"\n    Write a function that takes a message, and encodes in such a way that it swaps case of all letters, replaces all vowels in \n    the message with the letter that appears 2 places ahead of that \n    vowel in the english alphabet. \n    Assume only letters. \n    \n    Examples:\n    >>> encode('test')\n    'TGST'\n    >>> encode('This is a message')\n    'tHKS KS C MGSSCGG'\n    \"\"\"\n    vowels = 'aeiou'\n    result = ''\n    for char in message:\n        if char in vowels:\n            # Replace vowel with letter 2 places ahead\n            result += chr(ord(char) + 2)\n        else:\n            # Swap case\n            result += char.swapcase()\n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 156, "total_tokens": 341, "completion_tokens": 185, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 185, "latency_s": 17.235, "error": null, "agree_key": "def encode(message):\n    \"\"\"\n    Write a function that takes a message, and encodes in such a way that it swaps case of all letters, replaces all vowels in \n    the message with the letter that appears 2 places ahead of that \n    vowel in the english alphabet. \n    Assume only letters. \n    \n    Examples:\n    >>> encode('test')\n    'TGST'\n    >>> encode('This is a message')\n    'tHKS KS C MGSSCGG'\n    \"\"\"\n    vowels = 'aeiou'\n    result = ''\n    for char in message:\n        if char in vowels:\n            # Replace vowel with letter 2 places ahead\n            result += chr(ord(char) + 2)\n        else:\n            # Swap case\n            result += char.swapcase()\n    return result", "pred": "\n\ndef encode(message):\n    \"\"\"\n    Write a function that takes a message, and encodes in such a way that it swaps case of all letters, replaces all vowels in \n    the message with the letter that appears 2 places ahead of that \n    vowel in the english alphabet. \n    Assume only letters. \n    \n    Examples:\n    >>> encode('test')\n    'TGST'\n    >>> encode('This is a message')\n    'tHKS KS C MGSSCGG'\n    \"\"\"\n    vowels = 'aeiou'\n    result = ''\n    for char in message:\n        if char in vowels:\n            # Replace vowel with letter 2 places ahead\n            result += chr(ord(char) + 2)\n        else:\n            # Swap case\n            result += char.swapcase()\n    return result\n", "correct": false, "exec_error": "Traceback (most recent call last):\n  File \"/tmp/he_jpd51pdf/prog.py\", line 41, in <module>\n    check(encode)\n  File \"/tmp/he_jpd51pdf/prog.py\", line 31, in check\n    assert candidate('TEST') == 'tgst', \"This prints if this assert fails 1 (good for debugging!)\"\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: This prints if this assert fails 1 (good for debugging!)\n"}
+{"id": "HumanEval_97", "task": "humaneval", "prompt_sha256": "fec1bce2a34727a9", "content": "```python\ndef multiply(a, b):\n    \"\"\"Complete the function that takes two integers and returns \n    the product of their unit digits.\n    Assume the input is always valid.\n    Examples:\n    multiply(148, 412) should return 16.\n    multiply(19, 28) should return 72.\n    multiply(2020, 1851) should return 0.\n    multiply(14,-15) should return 20.\n    \"\"\"\n    return (a % 10) * (b % 10)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 161, "total_tokens": 300, "completion_tokens": 139, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 139, "latency_s": 12.072, "error": null, "agree_key": "def multiply(a, b):\n    \"\"\"Complete the function that takes two integers and returns \n    the product of their unit digits.\n    Assume the input is always valid.\n    Examples:\n    multiply(148, 412) should return 16.\n    multiply(19, 28) should return 72.\n    multiply(2020, 1851) should return 0.\n    multiply(14,-15) should return 20.\n    \"\"\"\n    return (a % 10) * (b % 10)", "pred": "\n\ndef multiply(a, b):\n    \"\"\"Complete the function that takes two integers and returns \n    the product of their unit digits.\n    Assume the input is always valid.\n    Examples:\n    multiply(148, 412) should return 16.\n    multiply(19, 28) should return 72.\n    multiply(2020, 1851) should return 0.\n    multiply(14,-15) should return 20.\n    \"\"\"\n    return (a % 10) * (b % 10)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_100", "task": "humaneval", "prompt_sha256": "f37d90b868e1c7a6", "content": "Looking at this problem, I need to:\n1. Start with `n` stones in the first level\n2. For each subsequent level, add the next odd number if `n` is odd, or the next even number if `n` is even\n3. Return a list of all levels\n\nLet me trace through the example:\n- `n = 3` (odd): first level = 3\n- Next odd number after 3 is 5: second level = 5\n- Next odd number after 5 is 7: third level = 7\n- Result: [3, 5, 7] ✓\n\nSo the pattern is:\n- If `n` is odd: each level increases by 2 (3, 5, 7, 9, ...)\n- If `n` is even: each level increases by 2 (2, 4, 6, 8, ...)\n\nWait, let me re-read. \"The next odd number if n is odd\" - so if n=3 (odd), the next odd number is 5. If n=4 (even), the next even number is 6.\n\nSo it's always adding 2 to get the next number of the same parity.\n\n```python\ndef make_a_pile(n):\n    \"\"\"\n    Given a positive integer n, you have to make a pile of n levels of stones.\n    The first level has n stones.\n    The number of stones in the next level is:\n        - the next odd number if n is odd.\n        - the next even number if n is even.\n    Return the number of stones in each level in a list, where element at index\n    i represents the number of stones in the level (i+1).\n\n    Examples:\n    >>> make_a_pile(3)\n    [3, 5, 7]\n    \"\"\"\n    result = [n]\n    current = n\n    for i in range(1, n):\n        if n % 2 == 0:\n            current += 2\n        else:\n            current += 2\n        result.append(current)\n    return result\n```\n\nWait, both cases add 2. Let me verify:\n- n=3 (odd): 3, 5, 7 → [3, 5, 7] ✓\n- n=4 (even): 4, 6, 8, 10 → [4, 6, 8, 10]\n\nYes, in both cases we're adding 2 each time. The parity determines what \"next\" means, but since we're already at the right parity, the next number of the same parity is just +2.\n\n```python\ndef make_a_pile(n):\n    \"\"\"\n    Given a positive integer n, you have to make a pile of n levels of stones.\n    The first level has n stones.\n    The number of stones in the next level is:\n        - the next odd number if n is odd.\n        - the next even number if n is even.\n    Return the number of stones in each level in a list, where element at index\n    i represents the number of stones in the level (i+1).\n\n    Examples:\n    >>> make_a_pile(3)\n    [3, 5, 7]\n    \"\"\"\n    result = [n]\n    current = n\n    for i in range(1, n):\n        current += 2\n        result.append(current)\n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 183, "total_tokens": 952, "completion_tokens": 769, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 769, "latency_s": 52.154, "error": null, "agree_key": "def make_a_pile(n):\n    \"\"\"\n    Given a positive integer n, you have to make a pile of n levels of stones.\n    The first level has n stones.\n    The number of stones in the next level is:\n        - the next odd number if n is odd.\n        - the next even number if n is even.\n    Return the number of stones in each level in a list, where element at index\n    i represents the number of stones in the level (i+1).\n\n    Examples:\n    >>> make_a_pile(3)\n    [3, 5, 7]\n    \"\"\"\n    result = [n]\n    current = n\n    for i in range(1, n):\n        if n % 2 == 0:\n            current += 2\n        else:\n            current += 2\n        result.append(current)\n    return result", "pred": "\n\ndef make_a_pile(n):\n    \"\"\"\n    Given a positive integer n, you have to make a pile of n levels of stones.\n    The first level has n stones.\n    The number of stones in the next level is:\n        - the next odd number if n is odd.\n        - the next even number if n is even.\n    Return the number of stones in each level in a list, where element at index\n    i represents the number of stones in the level (i+1).\n\n    Examples:\n    >>> make_a_pile(3)\n    [3, 5, 7]\n    \"\"\"\n    result = [n]\n    current = n\n    for i in range(1, n):\n        if n % 2 == 0:\n            current += 2\n        else:\n            current += 2\n        result.append(current)\n    return result\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_108", "task": "humaneval", "prompt_sha256": "a4c3508b82b64fa8", "content": "```python\ndef count_nums(arr):\n    \"\"\"\n    Write a function count_nums which takes an array of integers and returns\n    the number of elements which has a sum of digits > 0.\n    If a number is negative, then its first signed digit will be negative:\n    e.g. -123 has signed digits -1, 2, and 3.\n    >>> count_nums([]) == 0\n    >>> count_nums([-1, 11, -11]) == 1\n    >>> count_nums([1, 1, 2]) == 3\n    \"\"\"\n    count = 0\n    for num in arr:\n        if num == 0:\n            continue\n        sign = -1 if num < 0 else 1\n        n = abs(num)\n        s = 0\n        while n > 0:\n            s += sign * (n % 10)\n            n //= 10\n        if s > 0:\n            count += 1\n    return count\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 173, "total_tokens": 403, "completion_tokens": 230, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 230, "latency_s": 19.895, "error": null, "agree_key": "def count_nums(arr):\n    \"\"\"\n    Write a function count_nums which takes an array of integers and returns\n    the number of elements which has a sum of digits > 0.\n    If a number is negative, then its first signed digit will be negative:\n    e.g. -123 has signed digits -1, 2, and 3.\n    >>> count_nums([]) == 0\n    >>> count_nums([-1, 11, -11]) == 1\n    >>> count_nums([1, 1, 2]) == 3\n    \"\"\"\n    count = 0\n    for num in arr:\n        if num == 0:\n            continue\n        sign = -1 if num < 0 else 1\n        n = abs(num)\n        s = 0\n        while n > 0:\n            s += sign * (n % 10)\n            n //= 10\n        if s > 0:\n            count += 1\n    return count", "pred": "\n\ndef count_nums(arr):\n    \"\"\"\n    Write a function count_nums which takes an array of integers and returns\n    the number of elements which has a sum of digits > 0.\n    If a number is negative, then its first signed digit will be negative:\n    e.g. -123 has signed digits -1, 2, and 3.\n    >>> count_nums([]) == 0\n    >>> count_nums([-1, 11, -11]) == 1\n    >>> count_nums([1, 1, 2]) == 3\n    \"\"\"\n    count = 0\n    for num in arr:\n        if num == 0:\n            continue\n        sign = -1 if num < 0 else 1\n        n = abs(num)\n        s = 0\n        while n > 0:\n            s += sign * (n % 10)\n            n //= 10\n        if s > 0:\n            count += 1\n    return count\n", "correct": false, "exec_error": "Traceback (most recent call last):\n  File \"/tmp/he_7o7jpa8x/prog.py\", line 45, in <module>\n    check(count_nums)\n  File \"/tmp/he_7o7jpa8x/prog.py\", line 36, in check\n    assert candidate([12, 23, 34, -45, -56, 0]) == 5\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError\n"}
+{"id": "HumanEval_109", "task": "humaneval", "prompt_sha256": "df6a310718007ad4", "content": "```python\ndef move_one_ball(arr):\n    \"\"\"We have an array 'arr' of N integers arr[1], arr[2], ..., arr[N].The\n    numbers in the array will be randomly ordered. Your task is to determine if\n    it is possible to get an array sorted in non-decreasing order by performing \n    the following operation on the given array:\n        You are allowed to perform right shift operation any number of times.\n    \n    One right shift operation means shifting all elements of the array by one\n    position in the right direction. The last element of the array will be moved to\n    the starting position in the array i.e. 0th index. \n\n    If it is possible to obtain the sorted array by performing the above operation\n    then return True else return False.\n    If the given array is empty then return True.\n\n    Note: The given list is guaranteed to have unique elements.\n\n    For Example:\n    \n    move_one_ball([3, 4, 5, 1, 2])==>True\n    Explanation: By performin 2 right shift operations, non-decreasing order can\n                 be achieved for the given array.\n    move_one_ball([3, 5, 4, 1, 2])==>False\n    Explanation:It is not possible to get non-decreasing order for the given\n                array by performing any number of right shift operations.\n                \n    \"\"\"\n    if not arr:\n        return True\n    \n    # Count the number of \"breaks\" where arr[i] > arr[i+1]\n    # In a valid circular sorted array, there should be at most 1 break\n    breaks = 0\n    for i in range(len(arr)):\n        if arr[i] > arr[(i + 1) % len(arr)]:\n            breaks += 1\n            if breaks > 1:\n                return False\n    \n    return True\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 348, "total_tokens": 761, "completion_tokens": 413, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 413, "latency_s": 31.991, "error": null, "agree_key": "def move_one_ball(arr):\n    \"\"\"We have an array 'arr' of N integers arr[1], arr[2], ..., arr[N].The\n    numbers in the array will be randomly ordered. Your task is to determine if\n    it is possible to get an array sorted in non-decreasing order by performing \n    the following operation on the given array:\n        You are allowed to perform right shift operation any number of times.\n    \n    One right shift operation means shifting all elements of the array by one\n    position in the right direction. The last element of the array will be moved to\n    the starting position in the array i.e. 0th index. \n\n    If it is possible to obtain the sorted array by performing the above operation\n    then return True else return False.\n    If the given array is empty then return True.\n\n    Note: The given list is guaranteed to have unique elements.\n\n    For Example:\n    \n    move_one_ball([3, 4, 5, 1, 2])==>True\n    Explanation: By performin 2 right shift operations, non-decreasing order can\n                 be achieved for the given array.\n    move_one_ball([3, 5, 4, 1, 2])==>False\n    Explanation:It is not possible to get non-decreasing order for the given\n                array by performing any number of right shift operations.\n                \n    \"\"\"\n    if not arr:\n        return True\n    \n    # Count the number of \"breaks\" where arr[i] > arr[i+1]\n    # In a valid circular sorted array, there should be at most 1 break\n    breaks = 0\n    for i in range(len(arr)):\n        if arr[i] > arr[(i + 1) % len(arr)]:\n            breaks += 1\n            if breaks > 1:\n                return False\n    \n    return True", "pred": "\n\ndef move_one_ball(arr):\n    \"\"\"We have an array 'arr' of N integers arr[1], arr[2], ..., arr[N].The\n    numbers in the array will be randomly ordered. Your task is to determine if\n    it is possible to get an array sorted in non-decreasing order by performing \n    the following operation on the given array:\n        You are allowed to perform right shift operation any number of times.\n    \n    One right shift operation means shifting all elements of the array by one\n    position in the right direction. The last element of the array will be moved to\n    the starting position in the array i.e. 0th index. \n\n    If it is possible to obtain the sorted array by performing the above operation\n    then return True else return False.\n    If the given array is empty then return True.\n\n    Note: The given list is guaranteed to have unique elements.\n\n    For Example:\n    \n    move_one_ball([3, 4, 5, 1, 2])==>True\n    Explanation: By performin 2 right shift operations, non-decreasing order can\n                 be achieved for the given array.\n    move_one_ball([3, 5, 4, 1, 2])==>False\n    Explanation:It is not possible to get non-decreasing order for the given\n                array by performing any number of right shift operations.\n                \n    \"\"\"\n    if not arr:\n        return True\n    \n    # Count the number of \"breaks\" where arr[i] > arr[i+1]\n    # In a valid circular sorted array, there should be at most 1 break\n    breaks = 0\n    for i in range(len(arr)):\n        if arr[i] > arr[(i + 1) % len(arr)]:\n            breaks += 1\n            if breaks > 1:\n                return False\n    \n    return True\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_113", "task": "humaneval", "prompt_sha256": "ab33998346afae55", "content": "```python\ndef odd_count(lst):\n    \"\"\"Given a list of strings, where each string consists of only digits, return a list.\n    Each element i of the output should be \"the number of odd elements in the\n    string i of the input.\" where all the i's should be replaced by the number\n    of odd digits in the i'th string of the input.\n\n    >>> odd_count(['1234567'])\n    [\"the number of odd elements 4n the str4ng 4 of the 4nput.\"]\n    >>> odd_count(['3',\"11111111\"])\n    [\"the number of odd elements 1n the str1ng 1 of the 1nput.\",\n     \"the number of odd elements 8n the str8ng 8 of the 8nput.\"]\n    \"\"\"\n    result = []\n    for s in lst:\n        odd_count = sum(1 for c in s if int(c) % 2 == 1)\n        result.append(f\"the number of odd elements {odd_count}n the str{odd_count}ng {odd_count} of the {odd_count}nput.\")\n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 230, "total_tokens": 495, "completion_tokens": 265, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 265, "latency_s": 23.003, "error": null, "agree_key": "def odd_count(lst):\n    \"\"\"Given a list of strings, where each string consists of only digits, return a list.\n    Each element i of the output should be \"the number of odd elements in the\n    string i of the input.\" where all the i's should be replaced by the number\n    of odd digits in the i'th string of the input.\n\n    >>> odd_count(['1234567'])\n    [\"the number of odd elements 4n the str4ng 4 of the 4nput.\"]\n    >>> odd_count(['3',\"11111111\"])\n    [\"the number of odd elements 1n the str1ng 1 of the 1nput.\",\n     \"the number of odd elements 8n the str8ng 8 of the 8nput.\"]\n    \"\"\"\n    result = []\n    for s in lst:\n        odd_count = sum(1 for c in s if int(c) % 2 == 1)\n        result.append(f\"the number of odd elements {odd_count}n the str{odd_count}ng {odd_count} of the {odd_count}nput.\")\n    return result", "pred": "\n\ndef odd_count(lst):\n    \"\"\"Given a list of strings, where each string consists of only digits, return a list.\n    Each element i of the output should be \"the number of odd elements in the\n    string i of the input.\" where all the i's should be replaced by the number\n    of odd digits in the i'th string of the input.\n\n    >>> odd_count(['1234567'])\n    [\"the number of odd elements 4n the str4ng 4 of the 4nput.\"]\n    >>> odd_count(['3',\"11111111\"])\n    [\"the number of odd elements 1n the str1ng 1 of the 1nput.\",\n     \"the number of odd elements 8n the str8ng 8 of the 8nput.\"]\n    \"\"\"\n    result = []\n    for s in lst:\n        odd_count = sum(1 for c in s if int(c) % 2 == 1)\n        result.append(f\"the number of odd elements {odd_count}n the str{odd_count}ng {odd_count} of the {odd_count}nput.\")\n    return result\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_115", "task": "humaneval", "prompt_sha256": "71a71d90908102e9", "content": "```python\ndef max_fill(grid, capacity):\n    import math\n    \"\"\"\n    You are given a rectangular grid of wells. Each row represents a single well,\n    and each 1 in a row represents a single unit of water.\n    Each well has a corresponding bucket that can be used to extract water from it, \n    and all buckets have the same capacity.\n    Your task is to use the buckets to empty the wells.\n    Output the number of times you need to lower the buckets.\n    \"\"\"\n    total_water = sum(sum(row) for row in grid)\n    return math.ceil(total_water / capacity)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 380, "total_tokens": 517, "completion_tokens": 137, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 137, "latency_s": 11.807, "error": null, "agree_key": "def max_fill(grid, capacity):\n    import math\n    \"\"\"\n    You are given a rectangular grid of wells. Each row represents a single well,\n    and each 1 in a row represents a single unit of water.\n    Each well has a corresponding bucket that can be used to extract water from it, \n    and all buckets have the same capacity.\n    Your task is to use the buckets to empty the wells.\n    Output the number of times you need to lower the buckets.\n    \"\"\"\n    total_water = sum(sum(row) for row in grid)\n    return math.ceil(total_water / capacity)", "pred": "\n\ndef max_fill(grid, capacity):\n    import math\n    \"\"\"\n    You are given a rectangular grid of wells. Each row represents a single well,\n    and each 1 in a row represents a single unit of water.\n    Each well has a corresponding bucket that can be used to extract water from it, \n    and all buckets have the same capacity.\n    Your task is to use the buckets to empty the wells.\n    Output the number of times you need to lower the buckets.\n    \"\"\"\n    total_water = sum(sum(row) for row in grid)\n    return math.ceil(total_water / capacity)\n", "correct": false, "exec_error": "Traceback (most recent call last):\n  File \"/tmp/he_sc49nqws/prog.py\", line 33, in <module>\n    check(max_fill)\n  File \"/tmp/he_sc49nqws/prog.py\", line 29, in check\n    assert candidate([[1,1,1,1], [1,1,1,1]], 9) == 2, \"Error\"\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Error\n"}
+{"id": "HumanEval_118", "task": "humaneval", "prompt_sha256": "83fc8adc07703b4b", "content": "```python\ndef get_closest_vowel(word):\n    \"\"\"You are given a word. Your task is to find the closest vowel that stands between \n    two consonants from the right side of the word (case sensitive).\n    \n    Vowels in the beginning and ending doesn't count. Return empty string if you didn't\n    find any vowel met the above condition. \n\n    You may assume that the given string contains English letter only.\n\n    Example:\n    get_closest_vowel(\"yogurt\") ==> \"u\"\n    get_closest_vowel(\"FULL\") ==> \"U\"\n    get_closest_vowel(\"quick\") ==> \"\"\n    get_closest_vowel(\"ab\") ==> \"\"\n    \"\"\"\n    vowels = set('aeiouAEIOU')\n    consonants = set('bcdfghjklmnpqrstvwxyzBCDFGHJKLMNPQRSTVWXYZ')\n    \n    # Find the rightmost vowel that is between two consonants\n    for i in range(len(word) - 1, 0, -1):\n        if word[i] in vowels and word[i-1] in consonants and word[i+1] in consonants:\n            return word[i]\n    \n    return \"\"\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 197, "total_tokens": 464, "completion_tokens": 267, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 267, "latency_s": 23.561, "error": null, "agree_key": "def get_closest_vowel(word):\n    \"\"\"You are given a word. Your task is to find the closest vowel that stands between \n    two consonants from the right side of the word (case sensitive).\n    \n    Vowels in the beginning and ending doesn't count. Return empty string if you didn't\n    find any vowel met the above condition. \n\n    You may assume that the given string contains English letter only.\n\n    Example:\n    get_closest_vowel(\"yogurt\") ==> \"u\"\n    get_closest_vowel(\"FULL\") ==> \"U\"\n    get_closest_vowel(\"quick\") ==> \"\"\n    get_closest_vowel(\"ab\") ==> \"\"\n    \"\"\"\n    vowels = set('aeiouAEIOU')\n    consonants = set('bcdfghjklmnpqrstvwxyzBCDFGHJKLMNPQRSTVWXYZ')\n    \n    # Find the rightmost vowel that is between two consonants\n    for i in range(len(word) - 1, 0, -1):\n        if word[i] in vowels and word[i-1] in consonants and word[i+1] in consonants:\n            return word[i]\n    \n    return \"\"", "pred": "\n\ndef get_closest_vowel(word):\n    \"\"\"You are given a word. Your task is to find the closest vowel that stands between \n    two consonants from the right side of the word (case sensitive).\n    \n    Vowels in the beginning and ending doesn't count. Return empty string if you didn't\n    find any vowel met the above condition. \n\n    You may assume that the given string contains English letter only.\n\n    Example:\n    get_closest_vowel(\"yogurt\") ==> \"u\"\n    get_closest_vowel(\"FULL\") ==> \"U\"\n    get_closest_vowel(\"quick\") ==> \"\"\n    get_closest_vowel(\"ab\") ==> \"\"\n    \"\"\"\n    vowels = set('aeiouAEIOU')\n    consonants = set('bcdfghjklmnpqrstvwxyzBCDFGHJKLMNPQRSTVWXYZ')\n    \n    # Find the rightmost vowel that is between two consonants\n    for i in range(len(word) - 1, 0, -1):\n        if word[i] in vowels and word[i-1] in consonants and word[i+1] in consonants:\n            return word[i]\n    \n    return \"\"\n", "correct": false, "exec_error": "le>\n    check(get_closest_vowel)\n  File \"/tmp/he_jkb3csp1/prog.py\", line 36, in check\n    assert candidate(\"ali\") == \"\"\n           ^^^^^^^^^^^^^^^^\n  File \"/tmp/he_jkb3csp1/prog.py\", line 23, in get_closest_vowel\n    if word[i] in vowels and word[i-1] in consonants and word[i+1] in consonants:\n                                                         ~~~~^^^^^\nIndexError: string index out of range\n"}
+{"id": "HumanEval_121", "task": "humaneval", "prompt_sha256": "f8483f2fc0dea20d", "content": "```python\ndef solution(lst):\n    \"\"\"Given a non-empty list of integers, return the sum of all of the odd elements that are in even positions.\n    \n    Examples\n    solution([5, 8, 7, 1]) ==> 12\n    solution([3, 3, 3, 3, 3]) ==> 9\n    solution([30, 13, 24, 321]) ==>0\n    \"\"\"\n    return sum(lst[i] for i in range(0, len(lst), 2) if lst[i] % 2 != 0)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 145, "total_tokens": 281, "completion_tokens": 136, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 136, "latency_s": 11.692, "error": null, "agree_key": "def solution(lst):\n    \"\"\"Given a non-empty list of integers, return the sum of all of the odd elements that are in even positions.\n    \n    Examples\n    solution([5, 8, 7, 1]) ==> 12\n    solution([3, 3, 3, 3, 3]) ==> 9\n    solution([30, 13, 24, 321]) ==>0\n    \"\"\"\n    return sum(lst[i] for i in range(0, len(lst), 2) if lst[i] % 2 != 0)", "pred": "\n\ndef solution(lst):\n    \"\"\"Given a non-empty list of integers, return the sum of all of the odd elements that are in even positions.\n    \n    Examples\n    solution([5, 8, 7, 1]) ==> 12\n    solution([3, 3, 3, 3, 3]) ==> 9\n    solution([30, 13, 24, 321]) ==>0\n    \"\"\"\n    return sum(lst[i] for i in range(0, len(lst), 2) if lst[i] % 2 != 0)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_123", "task": "humaneval", "prompt_sha256": "e1523e76c7eb60cb", "content": "```python\ndef get_odd_collatz(n):\n    \"\"\"\n    Given a positive integer n, return a sorted list that has the odd numbers in collatz sequence.\n\n    The Collatz conjecture is a conjecture in mathematics that concerns a sequence defined\n    as follows: start with any positive integer n. Then each term is obtained from the \n    previous term as follows: if the previous term is even, the next term is one half of \n    the previous term. If the previous term is odd, the next term is 3 times the previous\n    term plus 1. The conjecture is that no matter what value of n, the sequence will always reach 1.\n\n    Note: \n        1. Collatz(1) is [1].\n        2. returned list sorted in increasing order.\n\n    For example:\n    get_odd_collatz(5) returns [1, 5] # The collatz sequence for 5 is [5, 16, 8, 4, 2, 1], so the odd numbers are only 1, and 5.\n    \"\"\"\n    odd_numbers = []\n    current = n\n    while current != 1:\n        if current % 2 == 1:\n            odd_numbers.append(current)\n            current = 3 * current + 1\n        else:\n            current = current // 2\n    odd_numbers.append(1)\n    return sorted(odd_numbers)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 280, "total_tokens": 596, "completion_tokens": 316, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 316, "latency_s": 23.63, "error": null, "agree_key": "def get_odd_collatz(n):\n    \"\"\"\n    Given a positive integer n, return a sorted list that has the odd numbers in collatz sequence.\n\n    The Collatz conjecture is a conjecture in mathematics that concerns a sequence defined\n    as follows: start with any positive integer n. Then each term is obtained from the \n    previous term as follows: if the previous term is even, the next term is one half of \n    the previous term. If the previous term is odd, the next term is 3 times the previous\n    term plus 1. The conjecture is that no matter what value of n, the sequence will always reach 1.\n\n    Note: \n        1. Collatz(1) is [1].\n        2. returned list sorted in increasing order.\n\n    For example:\n    get_odd_collatz(5) returns [1, 5] # The collatz sequence for 5 is [5, 16, 8, 4, 2, 1], so the odd numbers are only 1, and 5.\n    \"\"\"\n    odd_numbers = []\n    current = n\n    while current != 1:\n        if current % 2 == 1:\n            odd_numbers.append(current)\n            current = 3 * current + 1\n        else:\n            current = current // 2\n    odd_numbers.append(1)\n    return sorted(odd_numbers)", "pred": "\n\ndef get_odd_collatz(n):\n    \"\"\"\n    Given a positive integer n, return a sorted list that has the odd numbers in collatz sequence.\n\n    The Collatz conjecture is a conjecture in mathematics that concerns a sequence defined\n    as follows: start with any positive integer n. Then each term is obtained from the \n    previous term as follows: if the previous term is even, the next term is one half of \n    the previous term. If the previous term is odd, the next term is 3 times the previous\n    term plus 1. The conjecture is that no matter what value of n, the sequence will always reach 1.\n\n    Note: \n        1. Collatz(1) is [1].\n        2. returned list sorted in increasing order.\n\n    For example:\n    get_odd_collatz(5) returns [1, 5] # The collatz sequence for 5 is [5, 16, 8, 4, 2, 1], so the odd numbers are only 1, and 5.\n    \"\"\"\n    odd_numbers = []\n    current = n\n    while current != 1:\n        if current % 2 == 1:\n            odd_numbers.append(current)\n            current = 3 * current + 1\n        else:\n            current = current // 2\n    odd_numbers.append(1)\n    return sorted(odd_numbers)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_125", "task": "humaneval", "prompt_sha256": "aafe850603df4418", "content": "```python\ndef split_words(txt):\n    '''\n    Given a string of words, return a list of words split on whitespace, if no whitespaces exists in the text you\n    should split on commas ',' if no commas exists you should return the number of lower-case letters with odd order in the\n    alphabet, ord('a') = 0, ord('b') = 1, ... ord('z') = 25\n    Examples\n    split_words(\"Hello world!\") ➞ [\"Hello\", \"world!\"]\n    split_words(\"Hello,world!\") ➞ [\"Hello\", \"world!\"]\n    split_words(\"abcdef\") == 3 \n    '''\n    if ' ' in txt:\n        return txt.split(' ')\n    elif ',' in txt:\n        return txt.split(',')\n    else:\n        count = 0\n        for char in txt:\n            if char.isalpha():\n                order = ord(char.lower()) - ord('a')\n                if order % 2 == 1:\n                    count += 1\n        return count\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 185, "total_tokens": 416, "completion_tokens": 231, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 231, "latency_s": 18.981, "error": null, "agree_key": "def split_words(txt):\n    '''\n    Given a string of words, return a list of words split on whitespace, if no whitespaces exists in the text you\n    should split on commas ',' if no commas exists you should return the number of lower-case letters with odd order in the\n    alphabet, ord('a') = 0, ord('b') = 1, ... ord('z') = 25\n    Examples\n    split_words(\"Hello world!\") ➞ [\"Hello\", \"world!\"]\n    split_words(\"Hello,world!\") ➞ [\"Hello\", \"world!\"]\n    split_words(\"abcdef\") == 3 \n    '''\n    if ' ' in txt:\n        return txt.split(' ')\n    elif ',' in txt:\n        return txt.split(',')\n    else:\n        count = 0\n        for char in txt:\n            if char.isalpha():\n                order = ord(char.lower()) - ord('a')\n                if order % 2 == 1:\n                    count += 1\n        return count", "pred": "\n\ndef split_words(txt):\n    '''\n    Given a string of words, return a list of words split on whitespace, if no whitespaces exists in the text you\n    should split on commas ',' if no commas exists you should return the number of lower-case letters with odd order in the\n    alphabet, ord('a') = 0, ord('b') = 1, ... ord('z') = 25\n    Examples\n    split_words(\"Hello world!\") ➞ [\"Hello\", \"world!\"]\n    split_words(\"Hello,world!\") ➞ [\"Hello\", \"world!\"]\n    split_words(\"abcdef\") == 3 \n    '''\n    if ' ' in txt:\n        return txt.split(' ')\n    elif ',' in txt:\n        return txt.split(',')\n    else:\n        count = 0\n        for char in txt:\n            if char.isalpha():\n                order = ord(char.lower()) - ord('a')\n                if order % 2 == 1:\n                    count += 1\n        return count\n", "correct": false, "exec_error": "Traceback (most recent call last):\n  File \"/tmp/he_uxcti5vm/prog.py\", line 39, in <module>\n    check(split_words)\n  File \"/tmp/he_uxcti5vm/prog.py\", line 35, in check\n    assert candidate(\"aaaBb\") == 1\n           ^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError\n"}
+{"id": "HumanEval_128", "task": "humaneval", "prompt_sha256": "c6d4780f06bd73b4", "content": "```python\ndef prod_signs(arr):\n    \"\"\"\n    You are given an array arr of integers and you need to return\n    sum of magnitudes of integers multiplied by product of all signs\n    of each number in the array, represented by 1, -1 or 0.\n    Note: return None for empty arr.\n\n    Example:\n    >>> prod_signs([1, 2, 2, -4]) == -9\n    >>> prod_signs([0, 1]) == 0\n    >>> prod_signs([]) == None\n    \"\"\"\n    if not arr:\n        return None\n    \n    # Calculate the product of signs\n    sign_product = 1\n    for num in arr:\n        if num == 0:\n            sign_product = 0\n            break\n        elif num < 0:\n            sign_product *= -1\n    \n    # If any element is 0, the product of signs is 0\n    if sign_product == 0:\n        return 0\n    \n    # Sum of magnitudes\n    sum_magnitudes = sum(abs(num) for num in arr)\n    \n    return sum_magnitudes * sign_product\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 167, "total_tokens": 424, "completion_tokens": 257, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 257, "latency_s": 20.943, "error": null, "agree_key": "def prod_signs(arr):\n    \"\"\"\n    You are given an array arr of integers and you need to return\n    sum of magnitudes of integers multiplied by product of all signs\n    of each number in the array, represented by 1, -1 or 0.\n    Note: return None for empty arr.\n\n    Example:\n    >>> prod_signs([1, 2, 2, -4]) == -9\n    >>> prod_signs([0, 1]) == 0\n    >>> prod_signs([]) == None\n    \"\"\"\n    if not arr:\n        return None\n    \n    # Calculate the product of signs\n    sign_product = 1\n    for num in arr:\n        if num == 0:\n            sign_product = 0\n            break\n        elif num < 0:\n            sign_product *= -1\n    \n    # If any element is 0, the product of signs is 0\n    if sign_product == 0:\n        return 0\n    \n    # Sum of magnitudes\n    sum_magnitudes = sum(abs(num) for num in arr)\n    \n    return sum_magnitudes * sign_product", "pred": "\n\ndef prod_signs(arr):\n    \"\"\"\n    You are given an array arr of integers and you need to return\n    sum of magnitudes of integers multiplied by product of all signs\n    of each number in the array, represented by 1, -1 or 0.\n    Note: return None for empty arr.\n\n    Example:\n    >>> prod_signs([1, 2, 2, -4]) == -9\n    >>> prod_signs([0, 1]) == 0\n    >>> prod_signs([]) == None\n    \"\"\"\n    if not arr:\n        return None\n    \n    # Calculate the product of signs\n    sign_product = 1\n    for num in arr:\n        if num == 0:\n            sign_product = 0\n            break\n        elif num < 0:\n            sign_product *= -1\n    \n    # If any element is 0, the product of signs is 0\n    if sign_product == 0:\n        return 0\n    \n    # Sum of magnitudes\n    sum_magnitudes = sum(abs(num) for num in arr)\n    \n    return sum_magnitudes * sign_product\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_131", "task": "humaneval", "prompt_sha256": "9df68b8388c6a8fa", "content": "```python\ndef digits(n):\n    \"\"\"Given a positive integer n, return the product of the odd digits.\n    Return 0 if all digits are even.\n    For example:\n    digits(1)  == 1\n    digits(4)  == 0\n    digits(235) == 15\n    \"\"\"\n    product = 1\n    has_odd = False\n    for digit in str(n):\n        d = int(digit)\n        if d % 2 == 1:\n            product *= d\n            has_odd = True\n    return product if has_odd else 0\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 118, "total_tokens": 256, "completion_tokens": 138, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 138, "latency_s": 10.565, "error": null, "agree_key": "def digits(n):\n    \"\"\"Given a positive integer n, return the product of the odd digits.\n    Return 0 if all digits are even.\n    For example:\n    digits(1)  == 1\n    digits(4)  == 0\n    digits(235) == 15\n    \"\"\"\n    product = 1\n    has_odd = False\n    for digit in str(n):\n        d = int(digit)\n        if d % 2 == 1:\n            product *= d\n            has_odd = True\n    return product if has_odd else 0", "pred": "\n\ndef digits(n):\n    \"\"\"Given a positive integer n, return the product of the odd digits.\n    Return 0 if all digits are even.\n    For example:\n    digits(1)  == 1\n    digits(4)  == 0\n    digits(235) == 15\n    \"\"\"\n    product = 1\n    has_odd = False\n    for digit in str(n):\n        d = int(digit)\n        if d % 2 == 1:\n            product *= d\n            has_odd = True\n    return product if has_odd else 0\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_139", "task": "humaneval", "prompt_sha256": "33fbe7c9f21ce974", "content": "```python\nimport math\n\ndef special_factorial(n):\n    \"\"\"The Brazilian factorial is defined as:\n    brazilian_factorial(n) = n! * (n-1)! * (n-2)! * ... * 1!\n    where n > 0\n\n    For example:\n    >>> special_factorial(4)\n    288\n\n    The function will receive an integer as input and should return the special\n    factorial of this integer.\n    \"\"\"\n    result = 1\n    for i in range(1, n + 1):\n        result *= math.factorial(i)\n    return result\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 143, "total_tokens": 282, "completion_tokens": 139, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 139, "latency_s": 12.252, "error": null, "agree_key": "import math\n\ndef special_factorial(n):\n    \"\"\"The Brazilian factorial is defined as:\n    brazilian_factorial(n) = n! * (n-1)! * (n-2)! * ... * 1!\n    where n > 0\n\n    For example:\n    >>> special_factorial(4)\n    288\n\n    The function will receive an integer as input and should return the special\n    factorial of this integer.\n    \"\"\"\n    result = 1\n    for i in range(1, n + 1):\n        result *= math.factorial(i)\n    return result", "pred": "\n\nimport math\n\ndef special_factorial(n):\n    \"\"\"The Brazilian factorial is defined as:\n    brazilian_factorial(n) = n! * (n-1)! * (n-2)! * ... * 1!\n    where n > 0\n\n    For example:\n    >>> special_factorial(4)\n    288\n\n    The function will receive an integer as input and should return the special\n    factorial of this integer.\n    \"\"\"\n    result = 1\n    for i in range(1, n + 1):\n        result *= math.factorial(i)\n    return result\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_142", "task": "humaneval", "prompt_sha256": "e4a73d7bf037e43b", "content": "```python\ndef sum_squares(lst):\n    \"\"\"\n    This function will take a list of integers. For all entries in the list, the function shall square the integer entry if its index is a \n    multiple of 3 and will cube the integer entry if its index is a multiple of 4 and not a multiple of 3. The function will not \n    change the entries in the list whose indexes are not a multiple of 3 or 4. The function shall then return the sum of all entries. \n    \n    Examples:\n    For lst = [1,2,3] the output should be 6\n    For lst = []  the output should be 0\n    For lst = [-1,-5,2,-1,-5]  the output should be -126\n    \"\"\"\n    total = 0\n    for i, num in enumerate(lst):\n        if i % 3 == 0:\n            total += num ** 2\n        elif i % 4 == 0:\n            total += num ** 3\n        else:\n            total += num\n    return total\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 214, "total_tokens": 454, "completion_tokens": 240, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 240, "latency_s": 17.985, "error": null, "agree_key": "def sum_squares(lst):\n    \"\"\"\n    This function will take a list of integers. For all entries in the list, the function shall square the integer entry if its index is a \n    multiple of 3 and will cube the integer entry if its index is a multiple of 4 and not a multiple of 3. The function will not \n    change the entries in the list whose indexes are not a multiple of 3 or 4. The function shall then return the sum of all entries. \n    \n    Examples:\n    For lst = [1,2,3] the output should be 6\n    For lst = []  the output should be 0\n    For lst = [-1,-5,2,-1,-5]  the output should be -126\n    \"\"\"\n    total = 0\n    for i, num in enumerate(lst):\n        if i % 3 == 0:\n            total += num ** 2\n        elif i % 4 == 0:\n            total += num ** 3\n        else:\n            total += num\n    return total", "pred": "\n\n\n\ndef sum_squares(lst):\n    \"\"\"\n    This function will take a list of integers. For all entries in the list, the function shall square the integer entry if its index is a \n    multiple of 3 and will cube the integer entry if its index is a multiple of 4 and not a multiple of 3. The function will not \n    change the entries in the list whose indexes are not a multiple of 3 or 4. The function shall then return the sum of all entries. \n    \n    Examples:\n    For lst = [1,2,3] the output should be 6\n    For lst = []  the output should be 0\n    For lst = [-1,-5,2,-1,-5]  the output should be -126\n    \"\"\"\n    total = 0\n    for i, num in enumerate(lst):\n        if i % 3 == 0:\n            total += num ** 2\n        elif i % 4 == 0:\n            total += num ** 3\n        else:\n            total += num\n    return total\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_143", "task": "humaneval", "prompt_sha256": "3c153f4161cf3d8d", "content": "```python\ndef words_in_sentence(sentence):\n    \"\"\"\n    You are given a string representing a sentence,\n    the sentence contains some words separated by a space,\n    and you have to return a string that contains the words from the original sentence,\n    whose lengths are prime numbers,\n    the order of the words in the new string should be the same as the original one.\n\n    Example 1:\n        Input: sentence = \"This is a test\"\n        Output: \"is\"\n\n    Example 2:\n        Input: sentence = \"lets go for swimming\"\n        Output: \"go for\"\n\n    Constraints:\n        * 1 <= len(sentence) <= 100\n        * sentence contains only letters\n    \"\"\"\n    def is_prime(n):\n        if n < 2:\n            return False\n        if n == 2:\n            return True\n        if n % 2 == 0:\n            return False\n        for i in range(3, int(n**0.5) + 1, 2):\n            if n % i == 0:\n                return False\n        return True\n    \n    words = sentence.split()\n    result = [word for word in words if is_prime(len(word))]\n    return ' '.join(result)\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 203, "total_tokens": 484, "completion_tokens": 281, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 281, "latency_s": 19.752, "error": null, "agree_key": "def words_in_sentence(sentence):\n    \"\"\"\n    You are given a string representing a sentence,\n    the sentence contains some words separated by a space,\n    and you have to return a string that contains the words from the original sentence,\n    whose lengths are prime numbers,\n    the order of the words in the new string should be the same as the original one.\n\n    Example 1:\n        Input: sentence = \"This is a test\"\n        Output: \"is\"\n\n    Example 2:\n        Input: sentence = \"lets go for swimming\"\n        Output: \"go for\"\n\n    Constraints:\n        * 1 <= len(sentence) <= 100\n        * sentence contains only letters\n    \"\"\"\n    def is_prime(n):\n        if n < 2:\n            return False\n        if n == 2:\n            return True\n        if n % 2 == 0:\n            return False\n        for i in range(3, int(n**0.5) + 1, 2):\n            if n % i == 0:\n                return False\n        return True\n    \n    words = sentence.split()\n    result = [word for word in words if is_prime(len(word))]\n    return ' '.join(result)", "pred": "\n\ndef words_in_sentence(sentence):\n    \"\"\"\n    You are given a string representing a sentence,\n    the sentence contains some words separated by a space,\n    and you have to return a string that contains the words from the original sentence,\n    whose lengths are prime numbers,\n    the order of the words in the new string should be the same as the original one.\n\n    Example 1:\n        Input: sentence = \"This is a test\"\n        Output: \"is\"\n\n    Example 2:\n        Input: sentence = \"lets go for swimming\"\n        Output: \"go for\"\n\n    Constraints:\n        * 1 <= len(sentence) <= 100\n        * sentence contains only letters\n    \"\"\"\n    def is_prime(n):\n        if n < 2:\n            return False\n        if n == 2:\n            return True\n        if n % 2 == 0:\n            return False\n        for i in range(3, int(n**0.5) + 1, 2):\n            if n % i == 0:\n                return False\n        return True\n    \n    words = sentence.split()\n    result = [word for word in words if is_prime(len(word))]\n    return ' '.join(result)\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_144", "task": "humaneval", "prompt_sha256": "1cb3054fdaa00d6f", "content": "```python\ndef simplify(x, n):\n    \"\"\"Your task is to implement a function that will simplify the expression\n    x * n. The function returns True if x * n evaluates to a whole number and False\n    otherwise. Both x and n, are string representation of a fraction, and have the following format,\n    <numerator>/<denominator> where both numerator and denominator are positive whole numbers.\n\n    You can assume that x, and n are valid fractions, and do not have zero as denominator.\n\n    simplify(\"1/5\", \"5/1\") = True\n    simplify(\"1/6\", \"2/1\") = False\n    simplify(\"7/10\", \"10/2\") = False\n    \"\"\"\n    # Parse the fractions\n    x_num, x_den = map(int, x.split('/'))\n    n_num, n_den = map(int, n.split('/'))\n    \n    # Multiply the fractions: (x_num * n_num) / (x_den * n_den)\n    numerator = x_num * n_num\n    denominator = x_den * n_den\n    \n    # Check if the result is a whole number (i.e., denominator divides numerator evenly)\n    return numerator % denominator == 0\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 201, "total_tokens": 468, "completion_tokens": 267, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 267, "latency_s": 18.902, "error": null, "agree_key": "def simplify(x, n):\n    \"\"\"Your task is to implement a function that will simplify the expression\n    x * n. The function returns True if x * n evaluates to a whole number and False\n    otherwise. Both x and n, are string representation of a fraction, and have the following format,\n    <numerator>/<denominator> where both numerator and denominator are positive whole numbers.\n\n    You can assume that x, and n are valid fractions, and do not have zero as denominator.\n\n    simplify(\"1/5\", \"5/1\") = True\n    simplify(\"1/6\", \"2/1\") = False\n    simplify(\"7/10\", \"10/2\") = False\n    \"\"\"\n    # Parse the fractions\n    x_num, x_den = map(int, x.split('/'))\n    n_num, n_den = map(int, n.split('/'))\n    \n    # Multiply the fractions: (x_num * n_num) / (x_den * n_den)\n    numerator = x_num * n_num\n    denominator = x_den * n_den\n    \n    # Check if the result is a whole number (i.e., denominator divides numerator evenly)\n    return numerator % denominator == 0", "pred": "\n\ndef simplify(x, n):\n    \"\"\"Your task is to implement a function that will simplify the expression\n    x * n. The function returns True if x * n evaluates to a whole number and False\n    otherwise. Both x and n, are string representation of a fraction, and have the following format,\n    <numerator>/<denominator> where both numerator and denominator are positive whole numbers.\n\n    You can assume that x, and n are valid fractions, and do not have zero as denominator.\n\n    simplify(\"1/5\", \"5/1\") = True\n    simplify(\"1/6\", \"2/1\") = False\n    simplify(\"7/10\", \"10/2\") = False\n    \"\"\"\n    # Parse the fractions\n    x_num, x_den = map(int, x.split('/'))\n    n_num, n_den = map(int, n.split('/'))\n    \n    # Multiply the fractions: (x_num * n_num) / (x_den * n_den)\n    numerator = x_num * n_num\n    denominator = x_den * n_den\n    \n    # Check if the result is a whole number (i.e., denominator divides numerator evenly)\n    return numerator % denominator == 0\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_145", "task": "humaneval", "prompt_sha256": "636fd83d92b5e558", "content": "```python\ndef order_by_points(nums):\n    \"\"\"\n    Write a function which sorts the given list of integers\n    in ascending order according to the sum of their digits.\n    Note: if there are several items with similar sum of their digits,\n    order them based on their index in original list.\n\n    For example:\n    >>> order_by_points([1, 11, -1, -11, -12]) == [-1, -11, 1, -12, 11]\n    >>> order_by_points([]) == []\n    \"\"\"\n    def digit_sum(n):\n        return sum(int(d) for d in str(abs(n)))\n    \n    return [n for _, n in sorted(enumerate(nums), key=lambda x: (digit_sum(x[1]), x[0]))]\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 169, "total_tokens": 350, "completion_tokens": 181, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 181, "latency_s": 14.161, "error": null, "agree_key": "def order_by_points(nums):\n    \"\"\"\n    Write a function which sorts the given list of integers\n    in ascending order according to the sum of their digits.\n    Note: if there are several items with similar sum of their digits,\n    order them based on their index in original list.\n\n    For example:\n    >>> order_by_points([1, 11, -1, -11, -12]) == [-1, -11, 1, -12, 11]\n    >>> order_by_points([]) == []\n    \"\"\"\n    def digit_sum(n):\n        return sum(int(d) for d in str(abs(n)))\n    \n    return [n for _, n in sorted(enumerate(nums), key=lambda x: (digit_sum(x[1]), x[0]))]", "pred": "\n\ndef order_by_points(nums):\n    \"\"\"\n    Write a function which sorts the given list of integers\n    in ascending order according to the sum of their digits.\n    Note: if there are several items with similar sum of their digits,\n    order them based on their index in original list.\n\n    For example:\n    >>> order_by_points([1, 11, -1, -11, -12]) == [-1, -11, 1, -12, 11]\n    >>> order_by_points([]) == []\n    \"\"\"\n    def digit_sum(n):\n        return sum(int(d) for d in str(abs(n)))\n    \n    return [n for _, n in sorted(enumerate(nums), key=lambda x: (digit_sum(x[1]), x[0]))]\n", "correct": false, "exec_error": "Traceback (most recent call last):\n  File \"/tmp/he_ax3cb18x/prog.py\", line 35, in <module>\n    check(order_by_points)\n  File \"/tmp/he_ax3cb18x/prog.py\", line 23, in check\n    assert candidate([1, 11, -1, -11, -12]) == [-1, -11, 1, -12, 11]\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError\n"}
+{"id": "HumanEval_148", "task": "humaneval", "prompt_sha256": "74cc1071ffa7ac65", "content": "```python\ndef bf(planet1, planet2):\n    planets = [\"Mercury\", \"Venus\", \"Earth\", \"Mars\", \"Jupiter\", \"Saturn\", \"Uranus\", \"Neptune\"]\n    if planet1 not in planets or planet2 not in planets:\n        return ()\n    if planets.index(planet1) > planets.index(planet2):\n        planet1, planet2 = planet2, planet1\n    return tuple(planets[planets.index(planet1) + 1:planets.index(planet2)])\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 257, "total_tokens": 381, "completion_tokens": 124, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 124, "latency_s": 9.418, "error": null, "agree_key": "def bf(planet1, planet2):\n    planets = [\"Mercury\", \"Venus\", \"Earth\", \"Mars\", \"Jupiter\", \"Saturn\", \"Uranus\", \"Neptune\"]\n    if planet1 not in planets or planet2 not in planets:\n        return ()\n    if planets.index(planet1) > planets.index(planet2):\n        planet1, planet2 = planet2, planet1\n    return tuple(planets[planets.index(planet1) + 1:planets.index(planet2)])", "pred": "\n\ndef bf(planet1, planet2):\n    planets = [\"Mercury\", \"Venus\", \"Earth\", \"Mars\", \"Jupiter\", \"Saturn\", \"Uranus\", \"Neptune\"]\n    if planet1 not in planets or planet2 not in planets:\n        return ()\n    if planets.index(planet1) > planets.index(planet2):\n        planet1, planet2 = planet2, planet1\n    return tuple(planets[planets.index(planet1) + 1:planets.index(planet2)])\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_149", "task": "humaneval", "prompt_sha256": "64ac429cbd426991", "content": "```python\ndef sorted_list_sum(lst):\n    \"\"\"Write a function that accepts a list of strings as a parameter,\n    deletes the strings that have odd lengths from it,\n    and returns the resulted list with a sorted order,\n    The list is always a list of strings and never an array of numbers,\n    and it may contain duplicates.\n    The order of the list should be ascending by length of each word, and you\n    should return the list sorted by that rule.\n    If two words have the same length, sort the list alphabetically.\n    The function should return a list of strings in sorted order.\n    You may assume that all words will have the same length.\n    For example:\n    assert list_sort([\"aa\", \"a\", \"aaa\"]) => [\"aa\"]\n    assert list_sort([\"ab\", \"a\", \"aaa\", \"cd\"]) => [\"ab\", \"cd\"]\n    \"\"\"\n    # Filter out strings with odd lengths\n    even_length_strings = [s for s in lst if len(s) % 2 == 0]\n    \n    # Sort by length first, then alphabetically for same length\n    even_length_strings.sort(key=lambda s: (len(s), s))\n    \n    return even_length_strings\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 240, "total_tokens": 510, "completion_tokens": 270, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 270, "latency_s": 17.425, "error": null, "agree_key": "def sorted_list_sum(lst):\n    \"\"\"Write a function that accepts a list of strings as a parameter,\n    deletes the strings that have odd lengths from it,\n    and returns the resulted list with a sorted order,\n    The list is always a list of strings and never an array of numbers,\n    and it may contain duplicates.\n    The order of the list should be ascending by length of each word, and you\n    should return the list sorted by that rule.\n    If two words have the same length, sort the list alphabetically.\n    The function should return a list of strings in sorted order.\n    You may assume that all words will have the same length.\n    For example:\n    assert list_sort([\"aa\", \"a\", \"aaa\"]) => [\"aa\"]\n    assert list_sort([\"ab\", \"a\", \"aaa\", \"cd\"]) => [\"ab\", \"cd\"]\n    \"\"\"\n    # Filter out strings with odd lengths\n    even_length_strings = [s for s in lst if len(s) % 2 == 0]\n    \n    # Sort by length first, then alphabetically for same length\n    even_length_strings.sort(key=lambda s: (len(s), s))\n    \n    return even_length_strings", "pred": "\n\ndef sorted_list_sum(lst):\n    \"\"\"Write a function that accepts a list of strings as a parameter,\n    deletes the strings that have odd lengths from it,\n    and returns the resulted list with a sorted order,\n    The list is always a list of strings and never an array of numbers,\n    and it may contain duplicates.\n    The order of the list should be ascending by length of each word, and you\n    should return the list sorted by that rule.\n    If two words have the same length, sort the list alphabetically.\n    The function should return a list of strings in sorted order.\n    You may assume that all words will have the same length.\n    For example:\n    assert list_sort([\"aa\", \"a\", \"aaa\"]) => [\"aa\"]\n    assert list_sort([\"ab\", \"a\", \"aaa\", \"cd\"]) => [\"ab\", \"cd\"]\n    \"\"\"\n    # Filter out strings with odd lengths\n    even_length_strings = [s for s in lst if len(s) % 2 == 0]\n    \n    # Sort by length first, then alphabetically for same length\n    even_length_strings.sort(key=lambda s: (len(s), s))\n    \n    return even_length_strings\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_152", "task": "humaneval", "prompt_sha256": "785c5fc525f3e89d", "content": "```python\ndef compare(game, guess):\n    return [abs(g - s) for g, s in zip(game, guess)]\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 278, "total_tokens": 309, "completion_tokens": 31, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 31, "latency_s": 3.074, "error": null, "agree_key": "def compare(game, guess):\n    return [abs(g - s) for g, s in zip(game, guess)]", "pred": "\n\ndef compare(game, guess):\n    return [abs(g - s) for g, s in zip(game, guess)]\n", "correct": true, "exec_error": null}
+{"id": "HumanEval_154", "task": "humaneval", "prompt_sha256": "f08238ab12c9cb5c", "content": "```python\ndef cycpattern_check(a, b):\n    \"\"\"You are given 2 words. You need to return True if the second word or any of its rotations is a substring in the first word\n    cycpattern_check(\"abcd\",\"abd\") => False\n    cycpattern_check(\"hello\",\"ell\") => True\n    cycpattern_check(\"whassup\",\"psus\") => False\n    cycpattern_check(\"abab\",\"baa\") => True\n    cycpattern_check(\"efef\",\"eeff\") => False\n    cycpattern_check(\"himenss\",\"simen\") => True\n\n    \"\"\"\n    if len(b) > len(a):\n        return False\n    # Create a doubled version of b to check all rotations\n    doubled_b = b + b\n    # Check if any rotation of b is a substring of a\n    for i in range(len(b)):\n        if b[i:] + b[:i] in a:\n            return True\n    return False\n```", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 170, "total_tokens": 380, "completion_tokens": 210, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 210, "latency_s": 13.373, "error": null, "agree_key": "def cycpattern_check(a, b):\n    \"\"\"You are given 2 words. You need to return True if the second word or any of its rotations is a substring in the first word\n    cycpattern_check(\"abcd\",\"abd\") => False\n    cycpattern_check(\"hello\",\"ell\") => True\n    cycpattern_check(\"whassup\",\"psus\") => False\n    cycpattern_check(\"abab\",\"baa\") => True\n    cycpattern_check(\"efef\",\"eeff\") => False\n    cycpattern_check(\"himenss\",\"simen\") => True\n\n    \"\"\"\n    if len(b) > len(a):\n        return False\n    # Create a doubled version of b to check all rotations\n    doubled_b = b + b\n    # Check if any rotation of b is a substring of a\n    for i in range(len(b)):\n        if b[i:] + b[:i] in a:\n            return True\n    return False", "pred": "\n\ndef cycpattern_check(a, b):\n    \"\"\"You are given 2 words. You need to return True if the second word or any of its rotations is a substring in the first word\n    cycpattern_check(\"abcd\",\"abd\") => False\n    cycpattern_check(\"hello\",\"ell\") => True\n    cycpattern_check(\"whassup\",\"psus\") => False\n    cycpattern_check(\"abab\",\"baa\") => True\n    cycpattern_check(\"efef\",\"eeff\") => False\n    cycpattern_check(\"himenss\",\"simen\") => True\n\n    \"\"\"\n    if len(b) > len(a):\n        return False\n    # Create a doubled version of b to check all rotations\n    doubled_b = b + b\n    # Check if any rotation of b is a substring of a\n    for i in range(len(b)):\n        if b[i:] + b[:i] in a:\n            return True\n    return False\n", "correct": true, "exec_error": null}

runtime-v2/runs/final_frozen_v1/eval/raw_tools.jsonl

@@ -0,0 +1,20 @@
+{"id": "tool_simple_01", "task": "tools", "prompt_sha256": "cd8374fa6d9f55a0", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_18345dad4f5c40e199d89bc9", "index": 0, "type": "function", "function": {"name": "get_current_weather", "arguments": "{\"city\": \"Lisbon\", \"unit\": \"celsius\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 333, "total_tokens": 375, "completion_tokens": 42, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 42, "latency_s": 7.435, "error": null, "agree_key": "[{\"arguments\": {\"city\": \"Lisbon\", \"unit\": \"celsius\"}, \"name\": \"get_current_weather\"}]", "pred": [{"name": "get_current_weather", "arguments": {"city": "Lisbon", "unit": "celsius"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_simple_02", "task": "tools", "prompt_sha256": "0f6d7d46d0e5a223", "content": "To convert 250 US dollars to euros, I'll use the current exchange rate. Let me fetch the latest rate and perform the conversion.\n\n", "reasoning_content": "", "tool_calls": [{"id": "call_804c53d7ed394df9ae5c9a58", "index": 0, "type": "function", "function": {"name": "convert_currency", "arguments": "{\"amount\": 250, \"from_currency\": \"USD\", \"to_currency\": \"EUR\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 351, "total_tokens": 437, "completion_tokens": 86, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 86, "latency_s": 10.623, "error": null, "agree_key": "[{\"arguments\": {\"amount\": 250, \"from_currency\": \"USD\", \"to_currency\": \"EUR\"}, \"name\": \"convert_currency\"}]", "pred": [{"name": "convert_currency", "arguments": {"amount": 250, "from_currency": "USD", "to_currency": "EUR"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_simple_03", "task": "tools", "prompt_sha256": "e05b53b527821340", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_fd6fd6ff61a74661a4b464cd", "index": 0, "type": "function", "function": {"name": "set_timer", "arguments": "{\"minutes\": 25, \"label\": \"pomodoro\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 309, "total_tokens": 348, "completion_tokens": 39, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 39, "latency_s": 7.435, "error": null, "agree_key": "[{\"arguments\": {\"label\": \"pomodoro\", \"minutes\": 25}, \"name\": \"set_timer\"}]", "pred": [{"name": "set_timer", "arguments": {"minutes": 25, "label": "pomodoro"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_simple_04", "task": "tools", "prompt_sha256": "28415a81c8465c15", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_cf371efb117249ba87eb5fd2", "index": 0, "type": "function", "function": {"name": "get_stock_price", "arguments": "{\"ticker\": \"NVDA\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 303, "total_tokens": 331, "completion_tokens": 28, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 28, "latency_s": 5.618, "error": null, "agree_key": "[{\"arguments\": {\"ticker\": \"NVDA\"}, \"name\": \"get_stock_price\"}]", "pred": [{"name": "get_stock_price", "arguments": {"ticker": "NVDA"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_simple_05", "task": "tools", "prompt_sha256": "51d5f569c38dd0a7", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_555a3ea4612a4877a95342fc", "index": 0, "type": "function", "function": {"name": "calculate_bmi", "arguments": "{\"weight_kg\": 72.5, \"height_m\": 1.8}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 334, "total_tokens": 382, "completion_tokens": 48, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 48, "latency_s": 7.642, "error": null, "agree_key": "[{\"arguments\": {\"height_m\": 1.8, \"weight_kg\": 72.5}, \"name\": \"calculate_bmi\"}]", "pred": [{"name": "calculate_bmi", "arguments": {"weight_kg": 72.5, "height_m": 1.8}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_simple_06", "task": "tools", "prompt_sha256": "d50d52b081543b68", "content": "I can't list `/var/log` because I don't have access to your local filesystem. I'm an AI assistant running in a sandboxed environment — I can't execute shell commands like `find` or `ls`, and I have no visibility into your machine's files or directories.\n\nIf you're on a Linux/Unix system, you can run this yourself:\n\n```bash\nfind /var/log -type f\n```\n\nor, to include directory structure:\n\n```bash\nfind /var/log\n```\n\nIf you'd like, you can paste the output here and I can help you interpret it, filter it, or explain what specific log files mean.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 307, "total_tokens": 445, "completion_tokens": 138, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 138, "latency_s": 16.254, "error": null, "agree_key": "[]", "pred": [], "correct": false, "names_correct": false, "parse_source": "none", "format": {"n_tool_call_tags": 0, "balanced_tags": true, "all_blocks_parsed": true, "suffix_after_last_call": false, "well_formed": false}}
+{"id": "tool_simple_07", "task": "tools", "prompt_sha256": "019f21bc623f8681", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_2409c4cc8a1c4f7c8e78d418", "index": 0, "type": "function", "function": {"name": "get_time_in_timezone", "arguments": "{\"timezone\": \"Asia/Tokyo\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 313, "total_tokens": 344, "completion_tokens": 31, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 31, "latency_s": 6.39, "error": null, "agree_key": "[{\"arguments\": {\"timezone\": \"Asia/Tokyo\"}, \"name\": \"get_time_in_timezone\"}]", "pred": [{"name": "get_time_in_timezone", "arguments": {"timezone": "Asia/Tokyo"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_simple_08", "task": "tools", "prompt_sha256": "5fd20ce8ffb3efeb", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_a5e28a5bc92a4b8b9bf9cf8c", "index": 0, "type": "function", "function": {"name": "set_light_state", "arguments": "{\"room\": \"kitchen\", \"on\": true, \"brightness\": 40}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 337, "total_tokens": 390, "completion_tokens": 53, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 53, "latency_s": 8.398, "error": null, "agree_key": "[{\"arguments\": {\"brightness\": 40, \"on\": true, \"room\": \"kitchen\"}, \"name\": \"set_light_state\"}]", "pred": [{"name": "set_light_state", "arguments": {"room": "kitchen", "on": true, "brightness": 40}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_multiple_01", "task": "tools", "prompt_sha256": "ad1e98c47d664ea9", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_dc381a3e23ac46979025b1bf", "index": 0, "type": "function", "function": {"name": "search_flights", "arguments": "{\"origin\": \"SFO\", \"destination\": \"JFK\", \"date\": \"2026-12-03\", \"passengers\": 2}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 688, "total_tokens": 762, "completion_tokens": 74, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 74, "latency_s": 9.47, "error": null, "agree_key": "[{\"arguments\": {\"date\": \"2026-12-03\", \"destination\": \"JFK\", \"origin\": \"SFO\", \"passengers\": 2}, \"name\": \"search_flights\"}]", "pred": [{"name": "search_flights", "arguments": {"origin": "SFO", "destination": "JFK", "date": "2026-12-03", "passengers": 2}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_multiple_02", "task": "tools", "prompt_sha256": "7315e2e51aa692c2", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_0a5c1c14f332472c9d0976b3", "index": 0, "type": "function", "function": {"name": "send_email", "arguments": "{\"to\": \"bob@example.com\", \"subject\": \"Lunch\", \"body\": \"Are we still on for noon tomorrow?\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 596, "total_tokens": 657, "completion_tokens": 61, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 61, "latency_s": 8.954, "error": null, "agree_key": "[{\"arguments\": {\"body\": \"Are we still on for noon tomorrow?\", \"subject\": \"Lunch\", \"to\": \"bob@example.com\"}, \"name\": \"send_email\"}]", "pred": [{"name": "send_email", "arguments": {"to": "bob@example.com", "subject": "Lunch", "body": "Are we still on for noon tomorrow?"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_multiple_03", "task": "tools", "prompt_sha256": "6b7967916dcf13b1", "content": "Wo ist der Bahnhof?", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 514, "total_tokens": 520, "completion_tokens": 6, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 6, "latency_s": 3.968, "error": null, "agree_key": "[]", "pred": [], "correct": false, "names_correct": false, "parse_source": "none", "format": {"n_tool_call_tags": 0, "balanced_tags": true, "all_blocks_parsed": true, "suffix_after_last_call": false, "well_formed": false}}
+{"id": "tool_multiple_04", "task": "tools", "prompt_sha256": "00b9b3d384b4f457", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_54023a3f754d4f2d94ecb669", "index": 0, "type": "function", "function": {"name": "create_calendar_event", "arguments": "{\"title\": \"Quarterly review\", \"date\": \"2026-11-14\", \"start_time\": \"15:30\", \"duration_minutes\": 90}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 573, "total_tokens": 651, "completion_tokens": 78, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 78, "latency_s": 9.641, "error": null, "agree_key": "[{\"arguments\": {\"date\": \"2026-11-14\", \"duration_minutes\": 90, \"start_time\": \"15:30\", \"title\": \"Quarterly review\"}, \"name\": \"create_calendar_event\"}]", "pred": [{"name": "create_calendar_event", "arguments": {"title": "Quarterly review", "date": "2026-11-14", "start_time": "15:30", "duration_minutes": 90}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_multiple_05", "task": "tools", "prompt_sha256": "fd61dbb1c231face", "content": "I'll run that query on the warehouse for you.\n\n", "reasoning_content": "", "tool_calls": [{"id": "call_0cf7443c9ac847d3a70748dc", "index": 0, "type": "function", "function": {"name": "run_sql_query", "arguments": "{\"query\": \"SELECT COUNT(*) FROM orders\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 466, "total_tokens": 509, "completion_tokens": 43, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 43, "latency_s": 7.429, "error": null, "agree_key": "[{\"arguments\": {\"query\": \"SELECT COUNT(*) FROM orders\"}, \"name\": \"run_sql_query\"}]", "pred": [{"name": "run_sql_query", "arguments": {"query": "SELECT COUNT(*) FROM orders"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_multiple_06", "task": "tools", "prompt_sha256": "082dbd94f9082bb3", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_a6b6543cbe8f4cd7b6fba7cf", "index": 0, "type": "function", "function": {"name": "create_support_ticket", "arguments": "{\"customer_id\": \"C-1042\", \"priority\": \"high\", \"summary\": \"Router reboot loop\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 591, "total_tokens": 650, "completion_tokens": 59, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 59, "latency_s": 8.589, "error": null, "agree_key": "[{\"arguments\": {\"customer_id\": \"C-1042\", \"priority\": \"high\", \"summary\": \"Router reboot loop\"}, \"name\": \"create_support_ticket\"}]", "pred": [{"name": "create_support_ticket", "arguments": {"customer_id": "C-1042", "priority": "high", "summary": "Router reboot loop"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_parallel_01", "task": "tools", "prompt_sha256": "43cbd354ffa57502", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_6459da6dc69748e7bd260b32", "index": 0, "type": "function", "function": {"name": "get_current_weather", "arguments": "{\"city\": \"Oslo\", \"unit\": \"celsius\"}"}}, {"id": "call_3df40710378643bfb1de0723", "index": 1, "type": "function", "function": {"name": "get_current_weather", "arguments": "{\"city\": \"Cairo\", \"unit\": \"celsius\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 336, "total_tokens": 418, "completion_tokens": 82, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 82, "latency_s": 9.972, "error": null, "agree_key": "[{\"arguments\": {\"city\": \"Oslo\", \"unit\": \"celsius\"}, \"name\": \"get_current_weather\"}, {\"arguments\": {\"city\": \"Cairo\", \"unit\": \"celsius\"}, \"name\": \"get_current_weather\"}]", "pred": [{"name": "get_current_weather", "arguments": {"city": "Oslo", "unit": "celsius"}}, {"name": "get_current_weather", "arguments": {"city": "Cairo", "unit": "celsius"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_parallel_02", "task": "tools", "prompt_sha256": "93bac9f69c00a022", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_08417c89bc0f4fc3b9c6fabc", "index": 0, "type": "function", "function": {"name": "get_stock_price", "arguments": "{\"ticker\": \"AAPL\"}"}}, {"id": "call_e112b438ba8f4824b6abc4bc", "index": 1, "type": "function", "function": {"name": "get_stock_price", "arguments": "{\"ticker\": \"MSFT\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 430, "total_tokens": 486, "completion_tokens": 56, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 56, "latency_s": 8.587, "error": null, "agree_key": "[{\"arguments\": {\"ticker\": \"AAPL\"}, \"name\": \"get_stock_price\"}, {\"arguments\": {\"ticker\": \"MSFT\"}, \"name\": \"get_stock_price\"}]", "pred": [{"name": "get_stock_price", "arguments": {"ticker": "AAPL"}}, {"name": "get_stock_price", "arguments": {"ticker": "MSFT"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_parallel_03", "task": "tools", "prompt_sha256": "c8a8b3adb8239f03", "content": "", "reasoning_content": "", "tool_calls": [{"id": "call_92a93878494b4133a9637818", "index": 0, "type": "function", "function": {"name": "set_light_state", "arguments": "{\"room\": \"Bedroom\", \"on\": false}"}}, {"id": "call_bf98153a98854958bbfa3e84", "index": 1, "type": "function", "function": {"name": "set_timer", "arguments": "{\"minutes\": 10, \"label\": \"Timer\"}"}}], "finish_reason": "tool_calls", "usage": {"prompt_tokens": 424, "total_tokens": 502, "completion_tokens": 78, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 78, "latency_s": 6.649, "error": null, "agree_key": "[{\"arguments\": {\"on\": false, \"room\": \"Bedroom\"}, \"name\": \"set_light_state\"}, {\"arguments\": {\"label\": \"Timer\", \"minutes\": 10}, \"name\": \"set_timer\"}]", "pred": [{"name": "set_light_state", "arguments": {"room": "Bedroom", "on": false}}, {"name": "set_timer", "arguments": {"minutes": 10, "label": "Timer"}}], "correct": true, "names_correct": true, "parse_source": "server", "format": {"well_formed": true, "args_valid_json": true}}
+{"id": "tool_irrelevance_01", "task": "tools", "prompt_sha256": "7c1b4360276a07f1", "content": "\"Pride and Prejudice\" was written by Jane Austen.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 457, "total_tokens": 472, "completion_tokens": 15, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 15, "latency_s": 2.55, "error": null, "agree_key": "[]", "pred": [], "correct": true, "names_correct": true, "parse_source": "none", "format": {"n_tool_call_tags": 0, "balanced_tags": true, "all_blocks_parsed": true, "suffix_after_last_call": false, "well_formed": false}}
+{"id": "tool_irrelevance_02", "task": "tools", "prompt_sha256": "a838065102eca734", "content": "To calculate 17 multiplied by 23, you can break it down step by step:\n\n1. **Break down the numbers**:\n   - 17 = 10 + 7\n   - 23 = 20 + 3\n\n2. **Multiply each part**:\n   - 10 × 20 = 200\n   - 10 × 3 = 30\n   - 7 × 20 = 140\n   - 7 × 3 = 21\n\n3. **Add the results**:\n   - 200 + 30 + 140 + 21 = 391\n\nSo, 17 × 23 = **391**.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 388, "total_tokens": 555, "completion_tokens": 167, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 167, "latency_s": 10.322, "error": null, "agree_key": "[]", "pred": [], "correct": true, "names_correct": true, "parse_source": "none", "format": {"n_tool_call_tags": 0, "balanced_tags": true, "all_blocks_parsed": true, "suffix_after_last_call": false, "well_formed": false}}
+{"id": "tool_irrelevance_03", "task": "tools", "prompt_sha256": "21047009f9d3ecb5", "content": "Golden whispers drift on the October breeze,\nA quiet farewell to the year's last light.", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 488, "total_tokens": 508, "completion_tokens": 20, "prompt_tokens_details": null, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 20, "latency_s": 2.712, "error": null, "agree_key": "[]", "pred": [], "correct": true, "names_correct": true, "parse_source": "none", "format": {"n_tool_call_tags": 0, "balanced_tags": true, "all_blocks_parsed": true, "suffix_after_last_call": false, "well_formed": false}}

runtime-v2/runs/final_frozen_v1/eval/raw_vqa.jsonl

@@ -0,0 +1,40 @@
+{"id": "textvqa_val_34730", "task": "vqa", "prompt_sha256": "28a8cee8b40b5d5f", "content": "AJ52 UYV", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 1027, "total_tokens": 1034, "completion_tokens": 7, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 992}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 7, "latency_s": 10.073, "error": null, "agree_key": "aj52 uyv", "pred": "aj52 uyv", "correct": true, "vqa_soft_acc": 0.6666666666666666, "any_annotator_match": true}
+{"id": "textvqa_val_34752", "task": "vqa", "prompt_sha256": "8cb40c4dd11db457", "content": "Auto-Owners Insurance", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 706, "total_tokens": 711, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 11.468, "error": null, "agree_key": "auto owners insurance", "pred": "auto owners insurance", "correct": false, "vqa_soft_acc": 0.0, "any_annotator_match": false}
+{"id": "textvqa_val_34784", "task": "vqa", "prompt_sha256": "1d8997ec6867bc7d", "content": "SECOND PART", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 801, "total_tokens": 804, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.559, "error": null, "agree_key": "second part", "pred": "second part", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34817", "task": "vqa", "prompt_sha256": "f47d7ba7ec50408a", "content": "Lufthansa", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 704, "total_tokens": 709, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 10.058, "error": null, "agree_key": "lufthansa", "pred": "lufthansa", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34821", "task": "vqa", "prompt_sha256": "5f3476205a044338", "content": "LG", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 832, "total_tokens": 834, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 800}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.564, "error": null, "agree_key": "lg", "pred": "lg", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34836", "task": "vqa", "prompt_sha256": "b6e3d582f0a38f53", "content": "no", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 703, "total_tokens": 705, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.549, "error": null, "agree_key": "no", "pred": "no", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34865", "task": "vqa", "prompt_sha256": "e55b2533f52492cf", "content": "Yes", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 799, "total_tokens": 801, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.547, "error": null, "agree_key": "yes", "pred": "yes", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34933", "task": "vqa", "prompt_sha256": "4c5bda87d534f84d", "content": "left", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 800, "total_tokens": 802, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.547, "error": null, "agree_key": "left", "pred": "left", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34976", "task": "vqa", "prompt_sha256": "9ab50fb52b5a0b30", "content": "ferry", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 768, "total_tokens": 771, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 736}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.55, "error": null, "agree_key": "ferry", "pred": "ferry", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_34988", "task": "vqa", "prompt_sha256": "d0d4599795887edc", "content": "weight", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 707, "total_tokens": 709, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.536, "error": null, "agree_key": "weight", "pred": "weight", "correct": false, "vqa_soft_acc": 0.3333333333333333, "any_annotator_match": true}
+{"id": "textvqa_val_35018", "task": "vqa", "prompt_sha256": "9eafee090e7e0564", "content": "15.0%", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 768, "total_tokens": 774, "completion_tokens": 6, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 736}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 6, "latency_s": 10.046, "error": null, "agree_key": "15.0", "pred": "15.0", "correct": false, "vqa_soft_acc": 0.0, "any_annotator_match": false}
+{"id": "textvqa_val_35040", "task": "vqa", "prompt_sha256": "4a72929eda1156c9", "content": "Danger Deep water", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 705, "total_tokens": 709, "completion_tokens": 4, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 4, "latency_s": 9.536, "error": null, "agree_key": "danger deep water", "pred": "danger deep water", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35140", "task": "vqa", "prompt_sha256": "b4e0457f352fc063", "content": "10", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 510, "total_tokens": 513, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 480}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.539, "error": null, "agree_key": "10", "pred": "10", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35201", "task": "vqa", "prompt_sha256": "ee32236111db84ac", "content": "for your profile", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 800, "total_tokens": 804, "completion_tokens": 4, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 4, "latency_s": 9.541, "error": null, "agree_key": "for your profile", "pred": "for your profile", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35212", "task": "vqa", "prompt_sha256": "453a760b85676667", "content": "Meg Gardiner", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 804, "total_tokens": 808, "completion_tokens": 4, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 4, "latency_s": 9.529, "error": null, "agree_key": "meg gardiner", "pred": "meg gardiner", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35246", "task": "vqa", "prompt_sha256": "213e1f6ff4b2bc19", "content": "Corona", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 801, "total_tokens": 804, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.535, "error": null, "agree_key": "corona", "pred": "corona", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35291", "task": "vqa", "prompt_sha256": "f1bf33d655caaa6a", "content": "64067", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 677, "total_tokens": 683, "completion_tokens": 6, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 640}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 6, "latency_s": 9.676, "error": null, "agree_key": "64067", "pred": "64067", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35356", "task": "vqa", "prompt_sha256": "498ee1f3696b886e", "content": "yes", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 702, "total_tokens": 704, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.204, "error": null, "agree_key": "yes", "pred": "yes", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35394", "task": "vqa", "prompt_sha256": "531243ce455045de", "content": "Dreyer's", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 1060, "total_tokens": 1065, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 1024}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 9.67, "error": null, "agree_key": "dreyer's", "pred": "dreyer's", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35469", "task": "vqa", "prompt_sha256": "5e04af1147030b41", "content": "art", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 608, "total_tokens": 610, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 576}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.226, "error": null, "agree_key": "art", "pred": "art", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35542", "task": "vqa", "prompt_sha256": "41ab38b847b69099", "content": "chemical and engineering news", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 803, "total_tokens": 808, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 9.216, "error": null, "agree_key": "chemical and engineering news", "pred": "chemical and engineering news", "correct": true, "vqa_soft_acc": 0.6666666666666666, "any_annotator_match": true}
+{"id": "textvqa_val_35555", "task": "vqa", "prompt_sha256": "ff481b1b0fda9d6a", "content": "Who is a professor that made a difference for you?", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 710, "total_tokens": 722, "completion_tokens": 12, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 12, "latency_s": 14.779, "error": null, "agree_key": "who is professor that made difference for you", "pred": "who is professor that made difference for you", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35592", "task": "vqa", "prompt_sha256": "386b7cd105f8ed44", "content": "sugar", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 705, "total_tokens": 708, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.215, "error": null, "agree_key": "sugar", "pred": "sugar", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35731", "task": "vqa", "prompt_sha256": "f837dab9a17eea6f", "content": "Apple", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 800, "total_tokens": 802, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.223, "error": null, "agree_key": "apple", "pred": "apple", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35776", "task": "vqa", "prompt_sha256": "0f3959ab5e14b6f2", "content": "405", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 704, "total_tokens": 708, "completion_tokens": 4, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 4, "latency_s": 9.232, "error": null, "agree_key": "405", "pred": "405", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35818", "task": "vqa", "prompt_sha256": "d6f9d15c798365bb", "content": "2:23PM", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 800, "total_tokens": 806, "completion_tokens": 6, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 6, "latency_s": 9.657, "error": null, "agree_key": "2 23pm", "pred": "2 23pm", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35827", "task": "vqa", "prompt_sha256": "e394cf0e3f5afd79", "content": "Croft", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 738, "total_tokens": 741, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 704}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.204, "error": null, "agree_key": "croft", "pred": "croft", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35831", "task": "vqa", "prompt_sha256": "b1745a2f0b4c9a49", "content": "LG", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 608, "total_tokens": 610, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 576}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 9.207, "error": null, "agree_key": "lg", "pred": "lg", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35833", "task": "vqa", "prompt_sha256": "f79da370f92e0f3d", "content": "Shanea Wisler", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 607, "total_tokens": 612, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 576}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 9.73, "error": null, "agree_key": "shanea wisler", "pred": "shanea wisler", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35871", "task": "vqa", "prompt_sha256": "6452d9210285fbbf", "content": "Jen Hatmaker", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 737, "total_tokens": 742, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 704}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 9.168, "error": null, "agree_key": "jen hatmaker", "pred": "jen hatmaker", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35874", "task": "vqa", "prompt_sha256": "9a87fcff9c395eb4", "content": "23", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 675, "total_tokens": 678, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 640}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 9.17, "error": null, "agree_key": "23", "pred": "23", "correct": false, "vqa_soft_acc": 0.0, "any_annotator_match": false}
+{"id": "textvqa_val_35895", "task": "vqa", "prompt_sha256": "e623a6fc81af4628", "content": "8", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 704, "total_tokens": 706, "completion_tokens": 2, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 2, "latency_s": 7.349, "error": null, "agree_key": "8", "pred": "8", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_35975", "task": "vqa", "prompt_sha256": "bf5d6285781d08e1", "content": "WWF", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 703, "total_tokens": 706, "completion_tokens": 3, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 3, "latency_s": 5.539, "error": null, "agree_key": "wwf", "pred": "wwf", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_36000", "task": "vqa", "prompt_sha256": "0522007ba69421d6", "content": "1863km", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 1058, "total_tokens": 1064, "completion_tokens": 6, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 1024}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 6, "latency_s": 5.707, "error": null, "agree_key": "1863km", "pred": "1863km", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_36004", "task": "vqa", "prompt_sha256": "0f3e0b2e4ca41019", "content": "Smith-Corona", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 770, "total_tokens": 775, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 736}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 5.704, "error": null, "agree_key": "smith corona", "pred": "smith corona", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_36005", "task": "vqa", "prompt_sha256": "647a06960f3ac2b1", "content": "chocolate bock", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 803, "total_tokens": 808, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 5.713, "error": null, "agree_key": "chocolate bock", "pred": "chocolate bock", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_36110", "task": "vqa", "prompt_sha256": "20bcec7eadea7531", "content": "Crock-Pot", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 1058, "total_tokens": 1063, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 1024}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 5.715, "error": null, "agree_key": "crock pot", "pred": "crock pot", "correct": true, "vqa_soft_acc": 0.6666666666666666, "any_annotator_match": true}
+{"id": "textvqa_val_36207", "task": "vqa", "prompt_sha256": "e397b5fd699f013c", "content": "1819", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 800, "total_tokens": 805, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 768}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 5.719, "error": null, "agree_key": "1819", "pred": "1819", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_36267", "task": "vqa", "prompt_sha256": "ff31e339ac3eee21", "content": "Lucky Charms", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 1059, "total_tokens": 1064, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 1024}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 5.7, "error": null, "agree_key": "lucky charms", "pred": "lucky charms", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}
+{"id": "textvqa_val_36268", "task": "vqa", "prompt_sha256": "ea5ddf4f725997d1", "content": "Integrative Nutrition", "reasoning_content": "", "tool_calls": [], "finish_reason": "stop", "usage": {"prompt_tokens": 708, "total_tokens": 713, "completion_tokens": 5, "prompt_tokens_details": {"cached_tokens": 0, "image_tokens": 672}, "reasoning_tokens": 0, "completion_tokens_details": null}, "completion_tokens": 5, "latency_s": 5.701, "error": null, "agree_key": "integrative nutrition", "pred": "integrative nutrition", "correct": true, "vqa_soft_acc": 1.0, "any_annotator_match": true}

runtime-v2/runs/final_frozen_v1/eval/summary.json

@@ -0,0 +1,148 @@
+{
+  "eval_version": "sn-loop-eval-v1",
+  "model": "x",
+  "base_url": "http://127.0.0.1:30080/v1",
+  "started": "2026-10-30T23:20:23-0700",
+  "enable_thinking": false,
+  "sampling": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "seed": 0,
+    "max_tokens": {
+      "ifeval": 1024,
+      "gsm8k": 1024,
+      "mmlu_pro": 1024,
+      "humaneval": 1024,
+      "tools": 512,
+      "vqa": 64
+    }
+  },
+  "concurrency": 16,
+  "limit": null,
+  "host": "tux",
+  "data_sha256": {
+    "ifeval": "3e9e9e7bd42ef54862f0c71c9a4e6f0b17a9c259f8f4b2b4e5fbcad917228118",
+    "gsm8k": "5bb91d328c1ffc424d6679fe8edb36ed08152e4c75e75b343303e15cacc2da49",
+    "mmlu_pro": "e9d8d5717def43a0b995710ab093a13de48b4cfb4db19cd7c66affacfb492ea7",
+    "humaneval": "1e775e30085b890c19c48a583bfabb8f5f54598b5bf54796839b92e5bafdab9f",
+    "tools": "037b2e288f4f063e8a2d994799b4a51e4d97740b1cf62da75844243749dbb08a",
+    "vqa": "7d47303ecdf3386d04a852ab8b7e8be8d34137a1c4d10143331cc7175ce80d9d"
+  },
+  "tasks": {
+    "ifeval": {
+      "n": 60,
+      "correct": 46,
+      "accuracy": 0.7666666666666667,
+      "wilson95": [
+        0.6456372951896278,
+        0.855604383192118
+      ],
+      "n_errors": 0,
+      "n_truncated": 4,
+      "mean_completion_tokens": 268.68333333333334,
+      "sum_completion_tokens": 16121,
+      "instruction_level_acc": 0.8072289156626506,
+      "n_instructions": 83,
+      "wall_s": 143.2
+    },
+    "gsm8k": {
+      "n": 80,
+      "correct": 76,
+      "accuracy": 0.95,
+      "wilson95": [
+        0.8783772371965909,
+        0.9803864456007073
+      ],
+      "n_errors": 0,
+      "n_truncated": 1,
+      "mean_completion_tokens": 187.55,
+      "sum_completion_tokens": 15004,
+      "wall_s": 95.8
+    },
+    "mmlu_pro": {
+      "n": 60,
+      "correct": 42,
+      "accuracy": 0.7,
+      "wilson95": [
+        0.5749129195048435,
+        0.8010183392814656
+      ],
+      "n_errors": 0,
+      "n_truncated": 9,
+      "mean_completion_tokens": 577.3333333333334,
+      "sum_completion_tokens": 34640,
+      "wall_s": 209.0
+    },
+    "humaneval": {
+      "n": 40,
+      "correct": 33,
+      "accuracy": 0.825,
+      "wilson95": [
+        0.6805000953869541,
+        0.9125458630150444
+      ],
+      "n_errors": 0,
+      "n_truncated": 0,
+      "mean_completion_tokens": 208.9,
+      "sum_completion_tokens": 8356,
+      "wall_s": 60.0
+    },
+    "tools": {
+      "n": 20,
+      "correct": 18,
+      "accuracy": 0.9,
+      "wilson95": [
+        0.6989663528135878,
+        0.9721335190383086
+      ],
+      "n_errors": 0,
+      "n_truncated": 0,
+      "mean_completion_tokens": 60.2,
+      "sum_completion_tokens": 1204,
+      "names_correct": 0.9,
+      "parse_source_counts": {
+        "server": 15,
+        "none": 5
+      },
+      "format_well_formed": 1.0,
+      "raw_tool_call_tags_in_content": 0,
+      "by_category": {
+        "irrelevance": "3/3",
+        "multiple": "5/6",
+        "parallel": "3/3",
+        "simple": "7/8"
+      },
+      "wall_s": 16.7
+    },
+    "vqa": {
+      "n": 40,
+      "correct": 36,
+      "accuracy": 0.9,
+      "wilson95": [
+        0.7694822464406407,
+        0.9604204715925885
+      ],
+      "n_errors": 0,
+      "n_truncated": 0,
+      "mean_completion_tokens": 4.075,
+      "sum_completion_tokens": 163,
+      "vqa_soft_acc": 0.8833333333333334,
+      "wall_s": 24.5
+    }
+  },
+  "server_models": {
+    "object": "list",
+    "data": [
+      {
+        "id": "sensenova-loop-9b-nvfp4",
+        "object": "model",
+        "created": 1793427623,
+        "owned_by": "sglang",
+        "root": "sensenova-loop-9b-nvfp4",
+        "parent": null,
+        "max_model_len": 32768
+      }
+    ]
+  },
+  "wall_s": 549.2
+}

runtime-v2/runs/final_frozen_v1/eval_compare.txt

@@ -0,0 +1,6 @@
+task          n    ref   cand   delta            boot95  r-/c+ mcnemar  agree  ident       tok r/c
+gsm8k        80  0.963  0.950  -0.013 [-0.037,+0.000]    1/0   1.000  0.988  0.113   190.4/187.6
+humaneval    40  0.800  0.825  +0.025 [-0.050,+0.100]    1/2   1.000  0.500  0.500   191.3/208.9
+tools        20  0.900  0.900  +0.000 [+0.000,+0.000]    0/0   1.000  1.000  0.250     60.2/60.2
+ALL(micro)  140  0.907  0.907  +0.000 [-0.029,+0.029]      /     nan  0.850  0.243     None/None
+wrote runs/final_frozen_v1/eval/compare_vs_ref.json

runtime-v2/runs/final_frozen_v1/gpu_after.txt

@@ -0,0 +1 @@
+62, 2476 MHz, 22.19 W