Publish state-32k BF16, NVFP4, and Q8_0 GGUF with shared model card

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+dory-decisions-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+nvfp4/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,49 @@
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

NOTICE

@@ -0,0 +1,7 @@
+Derived from nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16, revision b366b87f762ea334ba3eb55912031baba1515810.
+Original model: NVIDIA. Base model licence: OpenMDW-1.1.
+Modifications: typed-decision LoRA fine-tuning, action selection, state tracking, planning, human-judgement training, BF16 merging, and NVFP4 quantization.
+Training source attribution, pinned versions, and recorded source licences appear in README.md and training-data.json.
+SCONE attribution: Long, Pasupat and Liang (2016). HelpSteer2 attribution: NVIDIA and Scale AI.
+
+Additional export: Q8_0 GGUF with Dory chat and typed-decision prompt metadata for llama.cpp.

README.md

@@ -0,0 +1,391 @@
+---
+license: other
+license_name: openmdw-1.1
+license_link: LICENSE
+base_model: nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16
+base_model_relation: finetune
+pipeline_tag: text-generation
+language:
+- en
+tags:
+- dory
+- decision-making
+- lora
+- eider
+- bf16
+- nvfp4
+- gguf
+- llama-cpp
+datasets:
+- ZefanCai/Open-Jev
+- SargeDev/jev-distill-corpus-v3
+- tasksource/procedural-typed-decisions
+- ZefanCai/Open-Jev-v1.1
+- Team-ACE/ToolACE
+- ehovy/race
+- openai/gsm8k
+- google/code_x_glue_cc_defect_detection
+- jat-project/jat-dataset
+- osunlp/early-experience
+- yyyyyyjjjjzzz/competitive-game-dataset
+- nvidia/HelpSteer2
+---
+
+# Dory 1.0 4B Typed Decisions
+
+**Release: `state-32k`.** A Dory fine-tune for fast typed decisions, action selection, and state tracking in software agents.
+This release continues `actions-48k` with 32,000 questions about state tracking, planning, human judgements, and earlier tasks.
+On 512 reserved new questions, native NVFP4 accuracy improves from **46.48% to 75.59%**.
+The main gains come from state tracking and planning. Human-judgement accuracy remains almost unchanged.
+
+The base model is [NVIDIA Dory 1.0 4B](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16).
+An application supplies a state, a question, and permitted answers.
+The model scores those answers and returns a binary probability, a named choice, or an ordered score.
+
+This experiment belongs to [Eider](https://github.com/rdaum/eider), a Rust/CUDA inference server for NVIDIA DGX Spark.
+Eider provides the evaluated prompt format and an optimized serving implementation.
+The learned behaviour can also run in another engine that implements Dory and the same decision format.
+
+## Available formats
+
+All three formats contain the same fine-tuned `state-32k` model. They share this model card and training history.
+
+| Format | Location | Use |
+|---|---|---|
+| BF16 safetensors | Root: eight shards and `model.safetensors.index.json` | Merged weights for a runtime with Dory support. |
+| NVFP4 safetensors | [`nvfp4/`](nvfp4) | Eider's evaluated packed 4-bit checkpoint, with its config and tokenizer. |
+| Q8_0 GGUF | [`dory-decisions-Q8_0.gguf`](dory-decisions-Q8_0.gguf), 4.23 GB | llama.cpp generation and native `/v1/systemone` decisions; instructions below. |
+| Rank-8 LoRA adapter | [`adapter/`](adapter/README.md) | Standalone training changes for the original NVIDIA BF16 model. |
+
+BF16 is the merged full-precision release; NVFP4 and Q8_0 are its quantized exports.
+The existing [BF16 repository](https://huggingface.co/rdaum/eider-dory-1.0-4b-decisions-bf16)
+and [NVFP4 repository](https://huggingface.co/rdaum/eider-dory-1.0-4b-decisions-nvfp4) remain available.
+
+The previous weights and calibration remain available under the [`actions-48k`](https://huggingface.co/rdaum/eider-dory-1.0-4b-decisions-bf16/tree/actions-48k) tag.
+This release has **no fitted probability-calibration profile**.
+
+## What this is for
+
+The intended use is a small, local decision component inside a larger application or agent.
+Examples include tool-call validation, workflow routing, document-grounding judgements, and assessments against a rubric.
+The application defines the question and answer meanings at request time, so the model can serve many classification tasks.
+
+The objective is a fast “System 1” judgement without an autoregressive explanation for every decision.
+The application can use the answer distribution to choose an action or request further analysis.
+These are intended uses, not guarantees of accuracy on every task.
+
+| Answer type | Meaning | Example |
+|---|---|---|
+| Binary (`noul` in the API) | Estimated probability of the supplied true condition | Does this proposed call satisfy the request? |
+| Choice | Distribution over named options and the highest-scoring option | Which queue receives this ticket? |
+| Ordered score | Distribution over rubric levels and their expected index | How well does this response satisfy the rubric? |
+
+## How the decision works
+
+The model keeps the original Dory architecture and vocabulary output head.
+LoRA training changes attention and MLP weights. It adds no separate classification head.
+The model uses **two recurrent loops**, which repeat internal computation through Dory layers without generating a chain of thought.
+
+The prompt combines the state, evaluation instructions, a question, and JSON-described options with single-token labels such as `A` and `B`.
+This excerpt illustrates the question portion:
+
+```text
+Question: Is the response supported by the supplied evidence?
+
+Options:
+A: {"option":"true","description":"Every claim is supported by the evidence."}
+B: {"option":"false","description":"At least one claim lacks support."}
+```
+
+The complete prompt also uses the Dory chat template, disabled explicit reasoning, and an assistant prefix of `Answer:\n`.
+At that position, the runtime reads the next-token logits for the permitted labels.
+It applies softmax across those labels and maps the resulting distribution to the requested answer type.
+These are conditional probabilities over the supplied options. They do not establish that the option list covers every possible answer.
+
+The application enforces the response type and option membership.
+The model supplies the judgement within that structure.
+This training does not establish support for arbitrary JSON schemas or open-ended structured generation.
+
+## How Eider serves decisions
+
+Eider exposes a `/v1/decisions` endpoint. A request contains a shared state and one or more typed questions.
+Each question supplies its permitted answers and their meanings.
+
+1. Eider renders the state and each question with the decision prompt described above.
+2. It processes the shared state once and evaluates separate question branches. Questions cannot attend to each other.
+3. It reads only the vocabulary-head rows for the answer labels at the end of each branch.
+4. It applies softmax over those labels and returns the requested binary probability, choice distribution, or ordered score.
+
+The decision path generates no answer tokens or explanation. Dory still performs its two internal recurrent loops.
+Eider batches question branches, reuses cached state prefixes, and uses a compact label readout to reduce repeated work.
+These are serving optimizations. The weights contain the learned decision behaviour and need no Eider-specific classification head.
+Another Dory runtime can reproduce the method with the same prompt, label mapping, recurrent settings, and probability calculation.
+
+A fitted calibration profile can adjust the distributions after inference, but this release does not include one.
+The probabilities describe the supplied options; they are not independently verified confidence estimates.
+
+## Training data
+
+The adapter carries five training stages. The tables below list all source families, grouped by purpose.
+A source can appear in several stages through replay.
+Pinned versions, source-file hashes, recorded source licences, and split rules are in [training-data.json](training-data.json).
+This repository contains model weights and metadata; they do not redistribute the source datasets.
+
+### Latest stage: state tracking, planning, and human judgements
+
+| Source | Questions in this stage | What the model learns |
+|---|---:|---|
+| [SCONE](https://nlp.stanford.edu/projects/scone/) — Alchemy and Tangrams | 6,000 | Track a world state through a complete instruction sequence and answer from the recorded final state. |
+| Independent Eider planning generators, motivated by [PlanBench](https://github.com/karthikv792/LLMs-Planning/tree/main/plan-bench) | 8,000 | Choose optimal next moves and assess legality or progress in block stacking and package delivery. |
+| [NVIDIA HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) | 2,000 | Learn correctness, helpfulness, coherence, and preference judgements from individual human ratings. |
+| Replay of earlier action questions | 8,000 | Retain action-selection behaviour from the four environment sources below. |
+| Replay of earlier general decision questions | 8,000 | Retain earlier decision, tool, document, reasoning, and code behaviour. |
+| **Total** | **32,000** | **16,000 new questions and 16,000 replay questions.** |
+
+SCONE supplies recorded state labels; its Scene domain is excluded pending an alignment audit.
+The planning generators and breadth-first search implementation are independent Eider code.
+No published PlanBench instances or generator code were copied.
+Planning targets share mass among equally optimal actions.
+HelpSteer2 targets use individual rating and preference frequencies, preserving disagreement and ties.
+Inputs exclude annotator ratings and justifications.
+
+### Earlier action and environment training
+
+These sources supplied the 24,000 new questions in the preceding 48,000-question stage; its other 24,000 questions were replay.
+
+| Source | New questions in that stage | Preparation |
+|---|---:|---|
+| [JAT BabyAI](https://huggingface.co/datasets/jat-project/jat-dataset) | 8,000 | Successful expert navigation, doors, keys, and object placement. Maps use past observations only. |
+| [ALFWorld through Early Experience](https://huggingface.co/datasets/osunlp/early-experience) | 6,000 | Recorded expert commands and admissible actions, with the supplied observation history. |
+| [Competitive Game Dataset](https://huggingface.co/datasets/yyyyyyjjjjzzz/competitive-game-dataset) — Connect Four | 4,000 | Solver-labelled legal moves, with equal target mass for tied best moves. |
+| [ScienceWorld gold paths](https://github.com/allenai/ScienceWorld/tree/e8216d6044e8e39be9fcb185e3b2dfb602584b52/goldpaths) | 6,000 | Select among 17 operation types using the goal, room, inventory, and past results. |
+
+ScienceWorld training concerns operation types, not complete object-specific commands.
+Expert imitation labels do not establish that every alternative action would fail.
+
+### General typed decisions and controlled scenarios
+
+| Source | Selected data and use |
+|---|---|
+| [Open-Jev](https://huggingface.co/datasets/ZefanCai/Open-Jev) | `release-v2-redistributable`: control, workflow, reasoning, and game questions. Original target distributions are preserved. |
+| [SargeDev/jev-distill-corpus-v3](https://huggingface.co/datasets/SargeDev/jev-distill-corpus-v3) | Only `yuri_v3` rows: diverse decision questions with supplied probability targets. |
+| [Procedural Typed Decisions](https://huggingface.co/datasets/tasksource/procedural-typed-decisions) | Rule-based and exact-probability tasks covering retrieval, state tracking, arithmetic, routing, and planning. |
+| [Open-Jev-v1.1](https://huggingface.co/datasets/ZefanCai/Open-Jev-v1.1) | `community-hard-mix-v2-redistributable`: WANLI-derived inference and controlled tool, document, support, and rubric scenarios. |
+
+### Tools, documents, reading, mathematics, and code
+
+| Source | Preparation |
+|---|---|
+| [ToolACE](https://huggingface.co/datasets/Team-ACE/ToolACE) | Judge reference tool calls and deliberately invalid variants that violate the supplied argument schema. |
+| [RAGTruth](https://github.com/ParticleMedia/RAGTruth) | Evidence-grounding questions from human span annotations, responses, and their supplied documents. |
+| [RACE](https://huggingface.co/datasets/ehovy/race) | Reading-comprehension questions with original options and labels. |
+| [GSM8K](https://huggingface.co/datasets/openai/gsm8k) | Numeric answer selection with generated distractors. Reference reasoning is excluded from the input. |
+| [Devign through CodeXGLUE](https://huggingface.co/datasets/google/code_x_glue_cc_defect_detection) | Binary C-function defect questions using the source labels. |
+
+### Data separation and target meaning
+
+Complete source groups stay together: episodes, games, goals, conversations, documents, or prompts.
+Planning groups include equivalent problems under block, package, and location renaming.
+Public test examples never enter training, and previous evaluation reservations remain excluded.
+New source groups contribute at most four questions each.
+The latest stage rejects inputs above 1,152 tokens instead of truncating them.
+
+**No JevMan or Pac-Man states, layouts, trajectories, rewards, or labels are used.**
+Other game and environment training is listed explicitly above.
+These results do not measure full-episode game performance.
+
+Training targets have different meanings across sources: recorded answers, expert actions, optimal actions, human votes, or supplied distributions.
+Optimal-action mass is not a win probability, and human vote frequencies do not establish outcome probabilities.
+
+## Training recipe and history
+
+Training starts from NVIDIA's BF16 revision `b366b87f762ea334ba3eb55912031baba1515810`.
+Rank-8 LoRA with alpha 16 updates 16,220,160 adapter parameters across 288 attention and MLP projections.
+The original weights remain frozen. Training and evaluation use two recurrent loops.
+The loss is soft-target conditional cross-entropy plus 0.5 times Brier error over the permitted answers.
+
+| Stage | Training questions | Mixture | Learning rate |
+|---|---:|---|---:|
+| Initial decisions | 49,152 | 24,576 Open-Jev + 24,576 Yuri v3 | 0.0001 |
+| Broader decisions | 65,536 | 24,576 procedural + 16,384 WANLI-derived + 8,192 controlled + 16,384 replay | 0.00002 |
+| Tools and documents | 32,000 | 4,000 tools + 4,000 documents + 8,000 general + 16,000 replay | 0.00001 |
+| Action selection | 48,000 | 24,000 environment questions + 24,000 replay | 0.00001 |
+| **State tracking, planning, and judging** | **32,000** | **16,000 new + 16,000 replay** | **0.00001** |
+
+Replay repeats earlier training questions to retain previous behaviour.
+These are questions processed per stage, not a total of unique examples.
+The latest stage continues the previous adapter with a fresh optimizer.
+It completed 8,000 updates of four questions each on one GB300, with about 45 minutes of training compute.
+
+## Accuracy evaluation
+
+The following results use **native Eider NVFP4 inference without probability calibration**.
+The previous and new adapters use the same frozen Eider executable, prompt format, and two-loop configuration.
+The parent NVFP4 export is byte-identical to the published `actions-48k` weights.
+
+Checkpoint selection uses 1,024 validation questions: 512 new questions and 512 earlier questions.
+The criterion is lowest Brier error, then cross-entropy, then earliest step.
+The unchanged parent and steps 2,000, 4,000, and 8,000 were eligible; step 8,000 won.
+Validation accuracy improved from 60.06% to 76.17%, and Brier error fell from 0.5027 to 0.2779.
+The 512 new test questions were scored after selection.
+
+| Question set | Model | Questions | Accuracy | Brier ↓ | Cross-entropy ↓ |
+|---|---|---:|---:|---:|---:|
+| New state, planning, and judging questions | Previous `actions-48k` NVFP4 | 512 | 46.48% | 0.7261 | 2.2838 |
+| New state, planning, and judging questions | This `state-32k` NVFP4 | 512 | 75.59% | 0.2706 | 1.0355 |
+| Earlier questions, reused | Previous `actions-48k` NVFP4 | 2,048 | 79.54% | 0.2521 | 0.7409 |
+| Earlier questions, reused | This `state-32k` NVFP4 | 2,048 | 79.35% | 0.2642 | 0.7846 |
+| General questions, reused | Previous `actions-48k` NVFP4 | 89 | 64.04% | 0.4997 | 1.0697 |
+| General questions, reused | This `state-32k` NVFP4 | 89 | 65.17% | 0.4791 | 1.0044 |
+
+The new test breaks down as follows:
+
+| Source | Questions | Previous NVFP4 | This NVFP4 | Brier, previous → current |
+|---|---:|---:|---:|---:|
+| SCONE state tracking | 192 | 42.19% | 75.52% | 0.8904 → 0.3779 |
+| Block-stacking and delivery planning | 192 | 48.44% | 92.19% | 0.6956 → 0.1909 |
+| HelpSteer2 human judgements | 128 | 50.00% | 50.78% | 0.5253 → 0.2290 |
+
+State tracking and planning improve substantially.
+HelpSteer2 accuracy changes by one answer; its predicted distributions move closer to the supplied human rating distributions.
+Earlier-question accuracy changes by four answers out of 2,048, while Brier and cross-entropy become slightly worse.
+The 89-question general check improves by one answer and is too small to establish broad generalization gains.
+
+Accuracy accepts any option with maximum target probability, including ties.
+Brier and cross-entropy compare the full predicted and target distributions.
+The earlier and general sets are reused development checks, not independent final benchmarks.
+Converted questions do not follow the original datasets' complete benchmark protocols.
+
+**These scores apply to NVFP4.** Neither the matching BF16 merge nor the Q8_0 GGUF has received a separate full accuracy evaluation.
+The original unfine-tuned NVIDIA model was not rescored in this comparison.
+[Full results](evaluation/nvfp4-comparison.json) include every candidate, answer type, and source.
+[Runtime metadata](evaluation/runtime.json) records the executable and script hashes.
+
+## Probability calibration
+
+No calibration profile has been fitted for `state-32k`.
+The previous profile was removed from the current release because it belongs to different weights.
+It remains in the original [NVFP4 repository’s `actions-48k` tag](https://huggingface.co/rdaum/eider-dory-1.0-4b-decisions-nvfp4/tree/actions-48k) for reproducibility.
+The 1,024-question calibration split remains separate from training, checkpoint selection, and the new test.
+Uncalibrated softmax values are conditional scores over the permitted answers; they are not proven real-world outcome probabilities.
+
+## Portability and files
+
+Another engine can implement the same method using Dory, its tokenizer and prompt, two loops, and selected-label scoring.
+Eider-specific CUDA kernels are not part of the learned weights.
+Eider accelerates serving with shared state processing, independent question branches, batching, CUDA graphs, and selected output logits.
+A different prompt, label mapping, loop count, or quantization can change the results.
+
+The root contains eight indexed BF16 safetensors shards, merged from the original NVIDIA weights and the selected adapter.
+Every saved tensor was checked against its CPU merge result. These merged BF16 files have not received a separate accuracy pass.
+
+The matching standalone LoRA adapter is in `adapter/`. The complete NVFP4 package also includes a copy.
+The adapter uses Eider tensor names rather than PEFT format; the supplied Python module loads that format.
+The adapter applies to the original NVIDIA BF16 checkpoint. Do not apply it again to the merged release weights.
+Its SHA-256 is `60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d`.
+
+The NVFP4 release quantizes our fine-tuned BF16 weights; it is not NVIDIA's original NVFP4 model.
+Its export uses packed E2M1 weights, E4M3 block scales, an F32 tensor scale, and a ModelOpt-compatible layout.
+The quantizer searches all 126 positive finite E4M3 scales for minimum block reconstruction error after BF16 merging.
+[Export metadata](evaluation/export.json) records the recipe, source hashes, and per-projection statistics.
+
+## Use the GGUF with llama.cpp
+
+The GGUF works with llama.cpp's native `/v1/systemone` decision endpoint. No Eider server, custom llama.cpp patch, or further training is required.
+**Dory architecture support is still in NVIDIA's unmerged [llama.cpp PR #30306](https://github.com/ggml-org/llama.cpp/pull/30306).**
+Use that branch until architecture support reaches upstream. This requirement also applies to ordinary generation.
+
+### Download
+
+Install the Hugging Face CLI and authenticate with `hf auth login` if this repository is private.
+Download only the format you need:
+
+```sh
+hf download rdaum/eider-dory-1.0-4b-decisions \
+    dory-decisions-Q8_0.gguf --local-dir ./dory-decisions
+```
+
+For NVFP4, use `--include 'nvfp4/*'` instead of the GGUF filename, then point your runtime at `./dory-decisions/nvfp4`.
+For BF16, use `--include '*.json' '*.jinja' '*.safetensors' --exclude 'nvfp4/*' 'adapter/*'` instead of the GGUF filename.
+
+### Build with Dory support and CUDA
+
+These commands require Git, CMake, a C++ compiler, and the CUDA toolkit.
+They build the Dory support branch for DGX Spark / GB10:
+
+```sh
+git clone https://github.com/ggml-org/llama.cpp llama.cpp-dory
+git -C llama.cpp-dory fetch origin pull/30306/head
+git -C llama.cpp-dory checkout --detach FETCH_HEAD
+cmake -S llama.cpp-dory -B llama.cpp-dory/build \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121 \
+    -DLLAMA_CURL=OFF \
+    -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF \
+    -DLLAMA_BUILD_SERVER=ON
+cmake --build llama.cpp-dory/build --target llama-server -j 6
+```
+
+For another NVIDIA GPU, select its CUDA architecture or omit `CMAKE_CUDA_ARCHITECTURES` for automatic detection.
+The compatibility checks below used GB10.
+
+### Start the server
+
+```sh
+./llama.cpp-dory/build/bin/llama-server \
+    --model ./dory-decisions/dory-decisions-Q8_0.gguf \
+    --alias dory-decisions \
+    --host 127.0.0.1 --port 18091 \
+    --ctx-size 4096 --parallel 1 \
+    --threads 4 --threads-batch 4 \
+    --batch-size 512 --ubatch-size 512 \
+    --fit off --gpu-layers all --no-warmup
+```
+
+The command offloads all layers to the GPU. The 4,096-token context is a small working example, not the model's context limit.
+
+### Ask a typed question
+
+```sh
+curl http://127.0.0.1:18091/v1/systemone \
+    -H 'Content-Type: application/json' \
+    -d '{"state":"The key is red.","questions":{"colour":{"type":"choice","instructions":"What colour is the key?","criteria":{"red":"Red","blue":"Blue"}}}}'
+```
+
+The endpoint returns answer probabilities with zero generated tokens.
+The GGUF provides a named `systemone` prompt template and selects llama.cpp's existing `granite-switch` decision convention.
+That name selects letter labels, true-before-false ordering, and selected next-token logits. **The architecture and weights are Dory, not IBM Granite.**
+The normal chat template is also retained.
+
+| Question type | Supported input |
+|---|---|
+| `choice` | 2–64 named options with text or null descriptions. |
+| `noul` | True/false, with optional text descriptions; defaults are `Yes` and `No`. |
+| `score` | 2–10 ordered text descriptions. |
+
+This llama.cpp convention supports 10 score levels; Eider supports 64.
+The tested server rejects excess score levels with HTTP 400. Template validation failures can return HTTP 500.
+The template accepts text, JSON states, and plain text chat. Serialize tool-call history as state text.
+Image and audio inputs are unsupported.
+
+### What was tested
+
+On 2026-10-21, the Q8_0 GGUF ran on GB10 with all 73 layers offloaded and no local llama.cpp source changes.
+Eight native prompt checks matched Eider's token IDs and answer labels exactly.
+They covered JSON, Unicode, chat, 64 choices, default yes/no descriptions, and 10 score levels.
+The 24-request GPU smoke check covered all three answer types and returned the expected answers with zero generated tokens.
+
+For short requests with changing state, median latency was 62 ms for one question and 191 ms for three questions.
+Each median used five timed requests. Server logs showed full prompt processing without prefix reuse.
+These small compatibility checks do not establish Q8_0 accuracy on the full evaluation sets.
+[GGUF provenance and test evidence](gguf/provenance.json) record the tested source revision, file hash, and checks.
+
+## Scope and provenance
+
+This experimental fine-tune targets typed decisions, action selection, state tracking, and planning.
+The results do not establish better general chat, reliable decisions in every domain, or full-context quality.
+[BF16 provenance](provenance.json), [NVFP4 provenance](nvfp4/provenance.json), and [format hashes](formats.json) bind the files to the selected adapter and source weights.
+
+NVIDIA supplies the base model, and the linked authors supply the training sources.
+[Eider](https://github.com/rdaum/eider) supplies the inference implementation and experiment tooling.
+The base model licence is [OpenMDW-1.1](LICENSE). This is an independent derivative.

adapter/README.md

@@ -0,0 +1,15 @@
+# Eider Dory adapter
+
+This adapter uses Eider tensor names. It is not a PEFT adapter.
+The complete model weights already include these changes. Do not apply the adapter twice.
+
+To load the adapter with the supplied Python module, use the original BF16 snapshot:
+
+```python
+from pathlib import Path
+from dory_lora import Dory
+
+model = Dory.load(Path("/path/to/original-bf16"), rank=8, alpha=16, loops=2)
+model.load_adapter(Path("adapter.safetensors"))
+model.eval()
+```

adapter/adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d
+size 64941920

adapter/dory_lora.py

@@ -0,0 +1,274 @@
+"""BF16 Dory with frozen checkpoint weights and trainable linear LoRA adapters.
+
+The nGPT equations follow the independent dory-checkpoint-reference.py oracle
+and vLLM revision 55076950b6c155f384d6766d63563c3bc20a0e3f. This is a training
+experiment, not a replacement for Eider's CUDA serving implementation.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+from pathlib import Path
+
+import torch
+from safetensors import safe_open
+from safetensors.torch import load_file, save_file
+from torch import nn
+from torch.nn import functional as F
+from torch.utils.checkpoint import checkpoint
+
+
+def norm(x):
+    value = x.float()
+    return (value * torch.rsqrt(value.square().sum(-1, keepdim=True) + 1e-6)).to(
+        x.dtype
+    )
+
+
+class Linear(nn.Module):
+    def __init__(self, weight, rank, alpha):
+        super().__init__()
+        self.register_buffer("weight", weight)
+        self.scale = alpha / rank if rank else 0
+        if rank:
+            self.lora_a = nn.Parameter(
+                torch.empty(rank, weight.shape[1], device=weight.device)
+            )
+            self.lora_b = nn.Parameter(
+                torch.zeros(weight.shape[0], rank, device=weight.device)
+            )
+            nn.init.kaiming_uniform_(self.lora_a, a=math.sqrt(5))
+        else:
+            self.register_parameter("lora_a", None)
+            self.register_parameter("lora_b", None)
+
+    def forward(self, x):
+        base = F.linear(x, self.weight)
+        if self.lora_a is None:
+            return base
+        delta = F.linear(F.linear(x, self.lora_a.to(x.dtype)), self.lora_b.to(x.dtype))
+        return (base.float() + self.scale * delta.float()).to(x.dtype)
+
+
+class Layer(nn.Module):
+    def __init__(self, config, index, read, rank, alpha):
+        super().__init__()
+        self.config = config
+        self.kind = config["layer_types"][index]
+        prefix = f"backbone.layers.{index}"
+        mixer = prefix + ".mixer"
+        std = config["initializer_range"]
+        if self.kind == "mlp":
+            names = ("gate_proj", "up_proj", "down_proj")
+            scale = math.sqrt(config["hidden_size"])
+            self.register_buffer("suv_gate", read(mixer + ".suv_gate").float() * scale)
+            self.register_buffer("suv_up", read(mixer + ".suv_up").float() * scale)
+            rate = read(prefix + ".mlp_alpha")
+        else:
+            names = ("q_proj", "k_proj", "v_proj", "g_proj", "o_proj")
+            dim = config["head_dim"]
+            self.register_buffer(
+                "sqk", (read(mixer + ".sqk").float() * (math.sqrt(dim) / std)).square()
+            )
+            g0 = config.get("attention_output_gate_logit_scale_init") or math.sqrt(
+                config["hidden_size"]
+            )
+            self.register_buffer(
+                "s_gate",
+                (read(mixer + ".s_gate").float() * (math.sqrt(g0) / std)).square(),
+            )
+            rate = read(prefix + ".attn_alpha")
+        self.register_buffer(
+            "rate", (rate.float() * config["ngpt_alpha_init"] / std).abs()
+        )
+        self.projections = nn.ModuleDict(
+            {
+                name: Linear(read(mixer + "." + name + ".weight"), rank, alpha)
+                for name in names
+            }
+        )
+
+    def forward(self, h, rotary, sliding_mask, reference=False):
+        p, c, dtype = self.projections, self.config, h.dtype
+        if self.kind == "mlp":
+            gate = (p["gate_proj"](h).float() * self.suv_gate).to(dtype)
+            up = (p["up_proj"](h).float() * self.suv_up).to(dtype)
+            out = p["down_proj"](norm((F.silu(gate.float()) * up.float()).to(dtype)))
+        else:
+            nq, nk, dim = (
+                c["num_attention_heads"],
+                c["num_key_value_heads"],
+                c["head_dim"],
+            )
+            cos, sin = rotary[self.kind]
+
+            def rope(x):
+                a, b = x.float().chunk(2, -1)
+                return torch.cat((a * cos - b * sin, b * cos + a * sin), -1).to(dtype)
+
+            q = norm(rope(p["q_proj"](h).reshape(*h.shape[:-1], nq, dim)))
+            k = norm(rope(p["k_proj"](h).reshape(*h.shape[:-1], nk, dim)))
+            q = (q.float() * self.sqk.reshape(nq, dim)).to(dtype)
+            v = p["v_proj"](h).reshape(*h.shape[:-1], nk, dim)
+            # Repeat GQA heads explicitly so both SDPA backends and the oracle
+            # use the same layout. Batch rows retain independent attention.
+            q = q.transpose(1, 2)
+            k = k.repeat_interleave(nq // nk, 2).transpose(1, 2)
+            v = v.repeat_interleave(nq // nk, 2).transpose(1, 2)
+            mask = sliding_mask if self.kind == "sliding_attention" else None
+            if reference:
+                if mask is None:
+                    mask = torch.ones(
+                        h.shape[1], h.shape[1], device=h.device, dtype=torch.bool
+                    ).tril()
+                scores = (q.float() @ k.float().transpose(-1, -2)) / math.sqrt(dim)
+                out = (
+                    scores.masked_fill(~mask, -torch.inf).softmax(-1) @ v.float()
+                ).to(dtype)
+            else:
+                out = F.scaled_dot_product_attention(
+                    q, k, v, attn_mask=mask, is_causal=mask is None
+                )
+            out = out.transpose(1, 2).reshape(*h.shape[:-1], nq * dim)
+            gate = p["g_proj"](h)
+            out = norm(
+                (out.float() * torch.sigmoid(gate.float() * self.s_gate)).to(dtype)
+            )
+            out = p["o_proj"](out)
+        a, b = norm(h).float(), norm(out).float()
+        return norm(a + self.rate * (b - a)).to(dtype)
+
+
+class Dory(nn.Module):
+    def __init__(self, config, read, *, rank=8, alpha=16, loops=2, checkpointing=False):
+        super().__init__()
+        if config.get("model_type") != "dory" or config.get("quantization_config"):
+            raise ValueError("requires an unquantized Dory checkpoint")
+        if loops < 1 or rank < 0 or alpha <= 0:
+            raise ValueError("invalid loop count or adapter dimensions")
+        self.config, self.loops, self.checkpointing = config, loops, checkpointing
+        self.register_buffer("embedding", read("backbone.embeddings.weight"))
+        self.register_buffer("head", read("lm_head.weight"))
+        self.register_buffer(
+            "head_scale",
+            read("lm_head.logit_scale").float() / config["initializer_range"],
+        )
+        self.layers = nn.ModuleList(
+            Layer(config, i, read, rank, alpha)
+            for i in range(config["num_hidden_layers"])
+        )
+
+    @classmethod
+    def load(cls, path: Path, *, device="cuda", **kwargs):
+        config = json.loads((path / "config.json").read_text())
+        with safe_open(
+            path / "model.safetensors", framework="pt", device="cpu"
+        ) as weights:
+            consumed = set()
+
+            def read(name):
+                value = weights.get_tensor(name)
+                if value.dtype != torch.bfloat16:
+                    raise ValueError(f"expected BF16 weight: {name}")
+                consumed.add(name)
+                return value.to(device)
+
+            model = cls(config, read, **kwargs)
+            if consumed != set(weights.keys()):
+                raise ValueError(
+                    f"unconsumed checkpoint tensors: {set(weights.keys()) - consumed}"
+                )
+        return model
+
+    def forward(self, ids, labels=None, *, lengths=None, reference=False):
+        c = self.config
+        single = ids.ndim == 1
+        if single:
+            ids = ids.unsqueeze(0)
+        positions = torch.arange(ids.shape[1], device=ids.device)
+        dim = c["head_dim"]
+        rotary = {}
+        for kind, params in c["rope_parameters"].items():
+            angles = positions.float()[:, None] * params["rope_theta"] ** (
+                -torch.arange(dim // 2, device=ids.device).float() / (dim // 2)
+            )
+            rotary[kind] = (angles.cos()[:, None, :], angles.sin()[:, None, :])
+        # For short training rows the sliding mask equals the causal mask.
+        sliding_mask = None
+        if ids.shape[1] > c["sliding_window"]:
+            distance = positions[:, None] - positions[None, :]
+            sliding_mask = (distance >= 0) & (distance < c["sliding_window"])
+
+        def apply(h, start, end):
+            for layer in self.layers[start:end]:
+                if self.training and self.checkpointing and not reference:
+                    h = checkpoint(layer, h, rotary, sliding_mask, use_reentrant=False)
+                else:
+                    # Keep the serial evaluator and oracle eager. Their batch-one,
+                    # inference-only graphs otherwise compete with training graphs.
+                    forward = (
+                        layer if self.training and not reference else layer.forward
+                    )
+                    h = forward(h, rotary, sliding_mask, reference)
+            return h
+
+        ni, nr = c["n_input_layers"], c["n_recurrent_layers"]
+        h = apply(F.embedding(ids, self.embedding), 0, ni)
+        input_h = h
+        for _ in range(self.loops):
+            h = norm(h + input_h)
+            h = norm(apply(h, ni, ni + nr))
+        h = apply(h, ni + nr, len(self.layers))
+        # Right-padding is always after each real query, so causal attention
+        # excludes it without a padding mask. Read only each row's final token.
+        last = (
+            h[:, -1]
+            if lengths is None
+            else h[torch.arange(ids.shape[0], device=ids.device), lengths - 1]
+        )
+        weight = self.head if labels is None else self.head[labels]
+        scale = self.head_scale if labels is None else self.head_scale[labels]
+        logits = (
+            torch.bmm(weight, last.unsqueeze(-1)).squeeze(-1)
+            if labels is not None and labels.ndim == 2
+            else F.linear(last, weight)
+        ).float() * scale
+        return logits.squeeze(0) if single else logits
+
+    def compile_layers(self):
+        # Compile individual layers so recurrent traversals reuse their graphs.
+        # Keep the BF16 rounding points used by the eager nGPT equations.
+        # All layer kinds share Layer.forward's code cache. Attention masks,
+        # input gradients and shape guards produce more than eight variants when
+        # training batches cross the sliding-window boundary. Keep a bounded
+        # cache large enough for those variants; fullgraph raises on overflow.
+        torch._dynamo.config.recompile_limit = max(
+            torch._dynamo.config.recompile_limit, 32
+        )
+        for layer in self.layers:
+            layer.compile(
+                fullgraph=True,
+                dynamic=True,
+                options={"emulate_precision_casts": True},
+            )
+
+    def save_adapter(self, path):
+        save_file(
+            {
+                name: p.detach().cpu().contiguous()
+                for name, p in self.named_parameters()
+            },
+            str(path),
+        )
+
+    def load_adapter(self, path):
+        values = load_file(str(path))
+        params = dict(self.named_parameters())
+        if values.keys() != params.keys():
+            raise ValueError("adapter does not match the model's projection modules")
+        with torch.no_grad():
+            for name, p in params.items():
+                if p.shape != values[name].shape:
+                    raise ValueError(f"adapter shape mismatch: {name}")
+                p.copy_(values[name])

adapter/eider_adapter_config.json

@@ -0,0 +1,12 @@
+{
+  "format": "eider.dory.lora/v1",
+  "base_model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16",
+  "base_revision": "b366b87f762ea334ba3eb55912031baba1515810",
+  "rank": 8,
+  "alpha": 16.0,
+  "loops": 2,
+  "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+  "tensor_names": "layers.{i}.projections.{name}.lora_a|lora_b",
+  "merge": "BF16(W + (alpha/rank) * F32(BF16(B)) @ F32(BF16(A)))",
+  "peft_compatible": false
+}

chat_template.jinja

@@ -0,0 +1,195 @@
+{% macro render_extra_keys(json_dict, handled_keys) %}
+    {%- if json_dict is mapping %}
+        {%- for json_key in json_dict if json_key not in handled_keys %}
+            {%- if json_dict[json_key] is mapping or (json_dict[json_key] is sequence and json_dict[json_key] is not string) %}
+                {{- '\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | tojson | safe) ~ '</' ~ json_key ~ '>' }}
+            {%- else %}
+                {{-'\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | string) ~ '</' ~ json_key ~ '>' }}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+{% endmacro %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else True %}
+{%- set medium_effort = medium_effort if medium_effort is defined else False %}
+{%- set truncate_history_thinking = truncate_history_thinking if truncate_history_thinking is defined else True %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- set loop_messages = messages %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = [] %}
+{%- endif %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if system_message is defined %}
+    {{- "<|im_start|>system\n" + system_message }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- "<|im_start|>system\n" }}
+    {%- endif %}
+{%- endif %}
+{%- if tools is iterable and tools | length > 0 %}
+    {%- if system_message is defined and system_message | length > 0 %}
+        {{- "\n\n" }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n" }}
+    {{- "<tools>" }}
+    {%- for tool in tools %}
+        {%- if tool.function is defined %}
+            {%- set tool = tool.function %}
+        {%- endif %}
+        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
+        {%- if tool.description is defined %}
+            {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
+        {%- endif %}
+        {{- '\n<parameters>' }}
+        {%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
+            {%- for param_name, param_fields in tool.parameters.properties|items %}
+                {{- '\n<parameter>' }}
+                {{- '\n<name>' ~ param_name ~ '</name>' }}
+                {%- if param_fields.type is defined %}
+                    {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
+                {%- endif %}
+                {%- if param_fields.description is defined %}
+                    {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
+                {%- endif %}
+                {%- if param_fields.enum is defined %}
+                    {{- '\n<enum>' ~ (param_fields.enum | tojson | safe) ~ '</enum>' }}
+                {%- endif %}
+                {%- set handled_keys = ['name', 'type', 'description', 'enum'] %}
+                {{- render_extra_keys(param_fields, handled_keys) }}
+                {{- '\n</parameter>' }}
+            {%- endfor %}
+        {%- endif %}
+        {% set handled_keys = ['type', 'properties', 'required'] %}
+        {{- render_extra_keys(tool.parameters, handled_keys) }}
+        {%- if tool.parameters is defined and tool.parameters.required is defined %}
+            {{- '\n<required>' ~ (tool.parameters.required | tojson | safe) ~ '</required>' }}
+        {%- endif %}
+        {{- '\n</parameters>' }}
+        {%- set handled_keys = ['type', 'name', 'description', 'parameters'] %}
+        {{- render_extra_keys(tool, handled_keys) }}
+        {{- '\n</function>' }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+{%- endif %}
+{%- if system_message is defined %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in loop_messages %}
+    {%- if message.role == "assistant" %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
+            {%- set content = "<think>\n" ~ message.reasoning_content ~ "</think>" ~ (message.content | default('', true)) %}
+        {%- else %}
+            {%- set content = message.content | default('', true) %}
+            {%- if content is string -%}
+                {%- if '<think>' not in content and '</think>' not in content -%}
+                    {%- set content = "<think></think>" ~ content -%}
+                {%- endif -%}
+            {%- else -%}
+                {%- set content = content -%}
+            {%- endif -%}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
+            {{- '<|im_start|>assistant\n' }}
+                {%- set include_content = not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {%- if content is string and content | trim | length > 0 %}
+                    {%- if include_content %}
+                        {{- (content | trim) ~ '\n' -}}
+                    {%- else %}
+                        {%- set c = (content | string) %}
+                        {%- if '</think>' in c %}
+                            {%- set c = c.split('</think>')[-1] %}
+                        {%- elif '<think>' in c %}
+                            {%- set c = c.split('<think>')[0] %}
+                        {%- endif %}
+                        {%- set c = "<think></think>" ~ c %}
+                        {%- if c | length > 0 %}
+                            {{- c ~ '\n' -}}
+                        {%- endif %}
+                    {%- endif %}
+                {%- else %}
+                    {{- "<think></think>" -}}
+                {%- endif %}
+                {%- for tool_call in message.tool_calls %}
+                    {%- if tool_call.function is defined %}
+                        {%- set tool_call = tool_call.function %}
+                    {%- endif %}
+                    {{- '<tool_call>\n<function=' ~ tool_call.name ~ '>\n' -}}
+                        {%- if tool_call.arguments is defined %}
+                            {%- for args_name, args_value in tool_call.arguments|items %}
+                                {{- '<parameter=' ~ args_name ~ '>\n' -}}
+                                    {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                                {{- args_value ~ '\n</parameter>\n' -}}
+                            {%- endfor %}
+                        {%- endif %}
+                    {{- '</function>\n</tool_call>\n' -}}
+                {%- endfor %}
+                {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- if not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {{- '<|im_start|>assistant\n' ~ (content | default('', true) | string | trim) ~ '<|im_end|>\n' }}
+            {%- else %}
+                {%- set c = (content | default('', true) | string) %}
+                {%- if '<think>' in c and '</think>' in c %}
+                    {%- set c = "<think></think>" ~ c.split('</think>')[-1] %}
+                {%- endif %}
+                {%- set c = c | trim %}
+                {%- if c | length > 0 %}
+                    {{- '<|im_start|>assistant\n' ~ c ~ '<|im_end|>\n' }}
+                {%- else %}
+                    {{- '<|im_start|>assistant\n<|im_end|>\n' }}
+                {%- endif %}
+            {%- endif %}
+        {%- endif %}
+    {%- elif message.role == "user" or message.role == "system" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- set content = message.content | string %}
+        {%- if message.role == "user" and loop.index0 == ns.last_user_idx and medium_effort %}
+            {{- content + '\n\n{reasoning effort: efficient}' }}
+        {%- else %}
+            {{- content }}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user\n' }}
+        {%- endif %}
+        {{- '<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>\n' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {%- if enable_thinking %}
+        {{- '<|im_start|>assistant\n<think>\n' }}
+    {%- else %}
+        {{- '<|im_start|>assistant\n<think></think>' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,113 @@
+{
+  "architectures": [
+    "DoryForCausalLM"
+  ],
+  "model_type": "dory",
+  "vocab_size": 131072,
+  "hidden_size": 2560,
+  "num_hidden_layers": 72,
+  "intermediate_size": 7168,
+  "num_attention_heads": 16,
+  "num_key_value_heads": 4,
+  "max_position_embeddings": 262144,
+  "tie_word_embeddings": false,
+  "ngpt_alpha_init": 0.05,
+  "attention_output_gate_logit_scale_init": null,
+  "mlp_bias": false,
+  "attention_bias": false,
+  "n_input_layers": 16,
+  "n_recurrent_layers": 40,
+  "n_output_layers": 16,
+  "n_recurrent_loops": 2,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "initializer_range": 0.0177,
+  "dtype": "bfloat16",
+  "sliding_window": 1024,
+  "rope_parameters": {
+    "sliding_attention": {
+      "rope_type": "default",
+      "rope_theta": 10000,
+      "partial_rotary_factor": 1.0
+    },
+    "full_attention": {
+      "rope_type": "default",
+      "rope_theta": 10000000,
+      "partial_rotary_factor": 1.0
+    }
+  },
+  "layer_types": [
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp"
+  ]
+}

dory-decisions-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb36b8ed263ce633649ccfad9c6e1dc065e9c3d333624fc27028f496089a38b8
+size 4232655616

evaluation/bf16-merge.json

@@ -0,0 +1,14 @@
+{
+  "method": "W + (alpha/rank) * BF16(B) @ BF16(A), accumulated F32, stored BF16",
+  "device": "cpu",
+  "torch": "2.10.0+cu128",
+  "source_sha256": "207483664dfef54c4c4438863260d3ba423bf3d6009b43fe84264b281e9fda23",
+  "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+  "quantizer_sha256": "baf5ad4237ff4390c41696a32ba97cb3b82ec87fe0bb78ea8878909f0a1b3772",
+  "projections": 288,
+  "tensor_count": 507,
+  "total_size": 7950458880,
+  "shards_verified": 8,
+  "accuracy_evaluation": "Not separately scored; latest reported native accuracy applies to NVFP4 only.",
+  "elapsed_s": 16.484787462744862
+}

evaluation/native-serving.json

@@ -0,0 +1,38 @@
+{
+  "command": [
+    "eider-serve",
+    "--model-dir",
+    "nvfp4-merged",
+    "--offline",
+    "--artifact-dir",
+    "nvfp4-merged",
+    "--served-model-name",
+    "dory-quant-eval",
+    "--dory-loops",
+    "2",
+    "--listen",
+    "127.0.0.1:18089",
+    "--max-active-sequences",
+    "1",
+    "--decode-capacity",
+    "1",
+    "--prefill-sequence-capacity",
+    "1",
+    "--prefill-token-capacity",
+    "2048",
+    "--decision-branch-capacity",
+    "8",
+    "--max-context-tokens",
+    "4096",
+    "--retained-prefix-gib",
+    "0"
+  ],
+  "binary_sha256": "935335c9df451b4b9108d65fc46882abbfd7a0b6cb20b3ba76f4e8c42fa7ab0c",
+  "script_sha256": "cbd6c833654b1fff319abf1d8f3cb491af5d96b78d541e593a283b684268da04",
+  "evaluation_data_dir": "data",
+  "evaluation_data_sha256": {
+    "new_test": "e2ca7413a306fc2b0e9e044aa685c94d5aa8e0def7cfcec6448941b7369ae625",
+    "test": "ace4bd84b914cee4c9ad8658d934386b045b7c643d490283d4f178f037acdba5",
+    "transfer": "3c5a77ed25fc7d0c2d1419151aa816bab9d12975b1e59de681934c114834d8e7"
+  }
+}

evaluation/nvfp4-comparison.json

@@ -0,0 +1,2033 @@
+{
+  "schema": "eider.dory.state-continuation-evaluation/v1",
+  "selection": {
+    "criterion": "NVFP4 validation Brier, cross-entropy, earliest step",
+    "candidates": [
+      {
+        "step": 0,
+        "examples_seen": 0,
+        "checkpoint": "step-000000",
+        "export": "step-000000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.6005859375,
+          "brier": 0.5027092950491365,
+          "cross_entropy": 1.6357855528374599,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.6030075187969924,
+              "brier": 0.4978527031414729,
+              "cross_entropy": 1.564796901472784
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.7361111111111112,
+              "brier": 0.4178836877295325,
+              "cross_entropy": 0.9008511909421006
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.5023255813953489,
+              "brier": 0.5745442767822497,
+              "cross_entropy": 2.347590209909279
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8359375,
+              "brier": 0.19931059120575503,
+              "cross_entropy": 0.7444107758516407
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.7083333333333334,
+              "brier": 0.428241328947602,
+              "cross_entropy": 1.050083253299498
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.39655172413793105,
+              "brier": 0.7305371209519513,
+              "cross_entropy": 1.4973851876562216
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 4.986938069711992e-05,
+              "cross_entropy": 0.003831617325445817
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.6,
+              "brier": 0.7805844658893524,
+              "cross_entropy": 1.7801497473019683
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.7142857142857143,
+              "brier": 0.4790055889595874,
+              "cross_entropy": 1.1121963454176031
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.6101325803838552,
+              "cross_entropy": 2.0033591272403792
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.7,
+              "brier": 0.6003855550573081,
+              "cross_entropy": 1.611779150841159
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.75,
+              "brier": 0.42100449664852624,
+              "cross_entropy": 1.9128633478153523
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5357142857142857,
+              "brier": 0.2741592752209492,
+              "cross_entropy": 1.860460665385586
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 2.0002025677443218e-06,
+              "cross_entropy": 0.0004476462766485558
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 0.5,
+              "brier": 0.41023068518803085,
+              "cross_entropy": 0.6006732852714238
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0013119361198151522,
+              "cross_entropy": 0.02163855006976333
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.4296875,
+              "brier": 0.5887451086780798,
+              "cross_entropy": 2.801737422463417
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.6041666666666666,
+              "brier": 0.5293956848932907,
+              "cross_entropy": 1.6646787560472525
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.8826525379691635,
+              "cross_entropy": 2.2968173663260414
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.31120510035647425,
+              "cross_entropy": 0.4402550888479215
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6953125,
+              "brier": 0.4588827899272394,
+              "cross_entropy": 1.3847298212396886
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.4739583333333333,
+              "brier": 0.8022954836937664,
+              "cross_entropy": 2.0042890400598963
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 2.2949296518888497e-08,
+              "cross_entropy": 9.582785411797238e-05
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7358490566037735,
+              "brier": 0.0564251534824245,
+              "cross_entropy": 0.8602018822761758
+            }
+          }
+        }
+      },
+      {
+        "step": 2000,
+        "examples_seen": 8000,
+        "checkpoint": "step-002000",
+        "export": "step-002000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.7275390625,
+          "brier": 0.31872868889688055,
+          "cross_entropy": 1.116584448395983,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7398496240601504,
+              "brier": 0.3486314543321744,
+              "cross_entropy": 1.1847615210242326
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8472222222222222,
+              "brier": 0.24375927242920462,
+              "cross_entropy": 0.5036143361404846
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6093023255813953,
+              "brier": 0.27645081427769413,
+              "cross_entropy": 1.3162586012657769
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8515625,
+              "brier": 0.20112173562767713,
+              "cross_entropy": 0.7568937670273396
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.75,
+              "brier": 0.3778614792863498,
+              "cross_entropy": 1.0070228429929422
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.5517241379310345,
+              "brier": 0.6998687564446888,
+              "cross_entropy": 1.449144847710849
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 8.502797038864707e-06,
+              "cross_entropy": 0.0017491505823356403
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.6,
+              "brier": 0.64090513735759,
+              "cross_entropy": 1.2842974245060323
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.7142857142857143,
+              "brier": 0.5732229366022509,
+              "cross_entropy": 1.3498112142920367
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.75,
+              "brier": 0.45820962916227576,
+              "cross_entropy": 1.7893452530760972
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.6,
+              "brier": 0.7878584523917267,
+              "cross_entropy": 2.649907510922513
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.8062445012416534,
+              "cross_entropy": 1.8181549807503623
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5714285714285714,
+              "brier": 0.268346840698743,
+              "cross_entropy": 1.8491581077937622
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 4.014360934536871e-05,
+              "cross_entropy": 0.0018805145026462434
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 0.43785613217853164,
+              "cross_entropy": 0.6306845393838758
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0023963047501729595,
+              "cross_entropy": 0.02626187987425209
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.546875,
+              "brier": 0.20973734203719346,
+              "cross_entropy": 1.5167245986871056
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9114583333333334,
+              "brier": 0.1816467064252445,
+              "cross_entropy": 0.8230249970733392
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.7315860423730358,
+              "cross_entropy": 2.0558892146843153
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.5,
+              "brier": 0.6525781536661275,
+              "cross_entropy": 0.9808095278526777
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6953125,
+              "brier": 0.430457617431226,
+              "cross_entropy": 1.3267025517668618
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.6979166666666666,
+              "brier": 0.44132333714721916,
+              "cross_entropy": 0.939129528604241
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 1.4302336637850583e-07,
+              "cross_entropy": 0.00026036870363595557
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7358490566037735,
+              "brier": 0.05366664986435064,
+              "cross_entropy": 0.8647152527990684
+            }
+          }
+        }
+      },
+      {
+        "step": 4000,
+        "examples_seen": 16000,
+        "checkpoint": "step-004000",
+        "export": "step-004000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.740234375,
+          "brier": 0.31259603426070215,
+          "cross_entropy": 1.0641685621781927,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7413533834586467,
+              "brier": 0.35520899641938364,
+              "cross_entropy": 1.150404749554445
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8333333333333334,
+              "brier": 0.2587362169867197,
+              "cross_entropy": 0.527162702406588
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6744186046511628,
+              "brier": 0.2168667033394475,
+              "cross_entropy": 1.157107070094022
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.859375,
+              "brier": 0.1875144540034567,
+              "cross_entropy": 0.6708904667559872
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.7083333333333334,
+              "brier": 0.4071248120462074,
+              "cross_entropy": 1.072867837361791
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.4827586206896552,
+              "brier": 0.7226287558174715,
+              "cross_entropy": 1.4270151869584489
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 6.599285085529234e-06,
+              "cross_entropy": 0.0016185379703032327
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.8,
+              "brier": 0.39998067070790544,
+              "cross_entropy": 1.08609476980073
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.8571428571428571,
+              "brier": 0.28687468365522234,
+              "cross_entropy": 0.7248149597143485
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.75,
+              "brier": 0.477438586205062,
+              "cross_entropy": 1.9109389754190291
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.6,
+              "brier": 0.7693115419877218,
+              "cross_entropy": 2.051549980800976
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.25,
+              "brier": 1.1149087692116901,
+              "cross_entropy": 2.466424832470031
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.6428571428571429,
+              "brier": 0.26805081416207754,
+              "cross_entropy": 1.8479820614570504
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 1.6518035601985627e-05,
+              "cross_entropy": 0.0012598198888345246
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 0.5,
+              "brier": 0.43338634881169164,
+              "cross_entropy": 0.6242116836662074
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0005199031989910008,
+              "cross_entropy": 0.013970904245707016
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.5625,
+              "brier": 0.2123470190045601,
+              "cross_entropy": 1.5161004021555795
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9010416666666666,
+              "brier": 0.18652079389211781,
+              "cross_entropy": 0.7723674627217165
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.6806357215652038,
+              "cross_entropy": 1.9339496668179939
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.5780328053534197,
+              "cross_entropy": 0.9064917649110132
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6796875,
+              "brier": 0.49342345206712573,
+              "cross_entropy": 1.4302781643246851
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.765625,
+              "brier": 0.3713316872108095,
+              "cross_entropy": 0.7410961018920553
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 3.664008183371336e-07,
+              "cross_entropy": 0.00041208643515311584
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7547169811320755,
+              "brier": 0.056719285382028715,
+              "cross_entropy": 0.8641864337643139
+            }
+          }
+        }
+      },
+      {
+        "step": 8000,
+        "examples_seen": 32000,
+        "checkpoint": "step-008000",
+        "export": "step-008000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.76171875,
+          "brier": 0.27786017825450965,
+          "cross_entropy": 0.9927829125866763,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7654135338345864,
+              "brier": 0.3189465668614991,
+              "cross_entropy": 1.0534065611001757
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8958333333333334,
+              "brier": 0.16471210037099535,
+              "cross_entropy": 0.39482479972079665
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6604651162790698,
+              "brier": 0.22656192147115203,
+              "cross_entropy": 1.2057654334760237
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8671875,
+              "brier": 0.1790825849971539,
+              "cross_entropy": 0.6080971490404414
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.5263100827359132,
+              "cross_entropy": 1.403012675529207
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.5517241379310345,
+              "brier": 0.684833327087956,
+              "cross_entropy": 1.5818612127101548
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 8.422208826818114e-06,
+              "cross_entropy": 0.0016334122276107624
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.8,
+              "brier": 0.39072800981047207,
+              "cross_entropy": 0.8926558622999824
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.8571428571428571,
+              "brier": 0.28439123582360026,
+              "cross_entropy": 0.7928804085918381
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.6286826358235,
+              "cross_entropy": 2.0004531092707
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.7,
+              "brier": 0.44538602593083176,
+              "cross_entropy": 1.117689014223367
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.25,
+              "brier": 1.4342464951649092,
+              "cross_entropy": 3.381933827755728
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5535714285714286,
+              "brier": 0.2622696208064482,
+              "cross_entropy": 1.8219529101323455
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 2.417104630113183e-06,
+              "cross_entropy": 0.0005900334430327367
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 0.43780762834608883,
+              "cross_entropy": 0.6306076256145883
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0003744313065983422,
+              "cross_entropy": 0.011988960485765076
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.5859375,
+              "brier": 0.20697473621325052,
+              "cross_entropy": 1.5034032587243362
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9635416666666666,
+              "brier": 0.11802009440200961,
+              "cross_entropy": 0.6579051066101222
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.7315329429054658,
+              "cross_entropy": 2.0647249708287245
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.609860991064251,
+              "cross_entropy": 0.9741778978250341
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.703125,
+              "brier": 0.44470820194957994,
+              "cross_entropy": 1.2102900099661185
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.796875,
+              "brier": 0.29338767561802165,
+              "cross_entropy": 0.6140028830315581
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 2.1804417353162663e-07,
+              "cross_entropy": 0.0003234946827488933
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7169811320754716,
+              "brier": 0.056977519481840835,
+              "cross_entropy": 0.860752644211149
+            }
+          }
+        }
+      }
+    ],
+    "winner": {
+      "step": 8000,
+      "examples_seen": 32000,
+      "checkpoint": "step-008000",
+      "export": "step-008000",
+      "validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.76171875,
+        "brier": 0.27786017825450965,
+        "cross_entropy": 0.9927829125866763,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.7654135338345864,
+            "brier": 0.3189465668614991,
+            "cross_entropy": 1.0534065611001757
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.8958333333333334,
+            "brier": 0.16471210037099535,
+            "cross_entropy": 0.39482479972079665
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.6604651162790698,
+            "brier": 0.22656192147115203,
+            "cross_entropy": 1.2057654334760237
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1790825849971539,
+            "cross_entropy": 0.6080971490404414
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5263100827359132,
+            "cross_entropy": 1.403012675529207
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.5517241379310345,
+            "brier": 0.684833327087956,
+            "cross_entropy": 1.5818612127101548
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 8.422208826818114e-06,
+            "cross_entropy": 0.0016334122276107624
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.39072800981047207,
+            "cross_entropy": 0.8926558622999824
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.8571428571428571,
+            "brier": 0.28439123582360026,
+            "cross_entropy": 0.7928804085918381
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6286826358235,
+            "cross_entropy": 2.0004531092707
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.44538602593083176,
+            "cross_entropy": 1.117689014223367
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.25,
+            "brier": 1.4342464951649092,
+            "cross_entropy": 3.381933827755728
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5535714285714286,
+            "brier": 0.2622696208064482,
+            "cross_entropy": 1.8219529101323455
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.417104630113183e-06,
+            "cross_entropy": 0.0005900334430327367
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.43780762834608883,
+            "cross_entropy": 0.6306076256145883
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003744313065983422,
+            "cross_entropy": 0.011988960485765076
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.20697473621325052,
+            "cross_entropy": 1.5034032587243362
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.9635416666666666,
+            "brier": 0.11802009440200961,
+            "cross_entropy": 0.6579051066101222
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.7315329429054658,
+            "cross_entropy": 2.0647249708287245
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.609860991064251,
+            "cross_entropy": 0.9741778978250341
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.703125,
+            "brier": 0.44470820194957994,
+            "cross_entropy": 1.2102900099661185
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.796875,
+            "brier": 0.29338767561802165,
+            "cross_entropy": 0.6140028830315581
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.1804417353162663e-07,
+            "cross_entropy": 0.0003234946827488933
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7169811320754716,
+            "brier": 0.056977519481840835,
+            "cross_entropy": 0.860752644211149
+          }
+        }
+      }
+    },
+    "validation_split": "new_validation"
+  },
+  "results": {
+    "actions-48k-parent-nvfp4": {
+      "new_validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.6005859375,
+        "brier": 0.5027092950491365,
+        "cross_entropy": 1.6357855528374599,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.6030075187969924,
+            "brier": 0.4978527031414729,
+            "cross_entropy": 1.564796901472784
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.7361111111111112,
+            "brier": 0.4178836877295325,
+            "cross_entropy": 0.9008511909421006
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.5023255813953489,
+            "brier": 0.5745442767822497,
+            "cross_entropy": 2.347590209909279
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8359375,
+            "brier": 0.19931059120575503,
+            "cross_entropy": 0.7444107758516407
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.7083333333333334,
+            "brier": 0.428241328947602,
+            "cross_entropy": 1.050083253299498
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.39655172413793105,
+            "brier": 0.7305371209519513,
+            "cross_entropy": 1.4973851876562216
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 4.986938069711992e-05,
+            "cross_entropy": 0.003831617325445817
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.6,
+            "brier": 0.7805844658893524,
+            "cross_entropy": 1.7801497473019683
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.7142857142857143,
+            "brier": 0.4790055889595874,
+            "cross_entropy": 1.1121963454176031
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6101325803838552,
+            "cross_entropy": 2.0033591272403792
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.6003855550573081,
+            "cross_entropy": 1.611779150841159
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.75,
+            "brier": 0.42100449664852624,
+            "cross_entropy": 1.9128633478153523
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5357142857142857,
+            "brier": 0.2741592752209492,
+            "cross_entropy": 1.860460665385586
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.0002025677443218e-06,
+            "cross_entropy": 0.0004476462766485558
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.41023068518803085,
+            "cross_entropy": 0.6006732852714238
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0013119361198151522,
+            "cross_entropy": 0.02163855006976333
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.4296875,
+            "brier": 0.5887451086780798,
+            "cross_entropy": 2.801737422463417
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.6041666666666666,
+            "brier": 0.5293956848932907,
+            "cross_entropy": 1.6646787560472525
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.8826525379691635,
+            "cross_entropy": 2.2968173663260414
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.31120510035647425,
+            "cross_entropy": 0.4402550888479215
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.6953125,
+            "brier": 0.4588827899272394,
+            "cross_entropy": 1.3847298212396886
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.4739583333333333,
+            "brier": 0.8022954836937664,
+            "cross_entropy": 2.0042890400598963
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.2949296518888497e-08,
+            "cross_entropy": 9.582785411797238e-05
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7358490566037735,
+            "brier": 0.0564251534824245,
+            "cross_entropy": 0.8602018822761758
+          }
+        }
+      },
+      "new_test": {
+        "questions": 512,
+        "modal_accuracy": 0.46484375,
+        "brier": 0.7260609345715332,
+        "cross_entropy": 2.283754099438618,
+        "by_kind": {
+          "choice": {
+            "questions": 241,
+            "modal_accuracy": 0.3941908713692946,
+            "brier": 0.814820900706993,
+            "cross_entropy": 2.425967860593386
+          },
+          "noul": {
+            "questions": 94,
+            "modal_accuracy": 0.5851063829787234,
+            "brier": 0.7046781668962778,
+            "cross_entropy": 1.348769508075005
+          },
+          "score": {
+            "questions": 177,
+            "modal_accuracy": 0.4971751412429379,
+            "brier": 0.6165627895027659,
+            "cross_entropy": 2.586663902545287
+          }
+        },
+        "by_source": {
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5,
+            "brier": 0.5252889060517718,
+            "cross_entropy": 2.7138956389335567
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.484375,
+            "brier": 0.695581487251716,
+            "cross_entropy": 2.128417640206426
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.421875,
+            "brier": 0.8903884009045245,
+            "cross_entropy": 2.1523295323408504
+          }
+        }
+      },
+      "test": {
+        "questions": 2048,
+        "modal_accuracy": 0.79541015625,
+        "brier": 0.25207741014224244,
+        "cross_entropy": 0.7408834203798337,
+        "by_kind": {
+          "choice": {
+            "questions": 1151,
+            "modal_accuracy": 0.7176368375325803,
+            "brier": 0.34765282650467216,
+            "cross_entropy": 1.0022488366720779
+          },
+          "noul": {
+            "questions": 649,
+            "modal_accuracy": 0.9090909090909091,
+            "brier": 0.1294082456235435,
+            "cross_entropy": 0.3089760935689536
+          },
+          "score": {
+            "questions": 248,
+            "modal_accuracy": 0.8588709677419355,
+            "brier": 0.1295168598982062,
+            "cross_entropy": 0.6581264080729309
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.84375,
+            "brier": 0.1956377254181323,
+            "cross_entropy": 0.5228371535175946
+          },
+          "arithmetic": {
+            "questions": 30,
+            "modal_accuracy": 0.8,
+            "brier": 0.365987202749238,
+            "cross_entropy": 0.874987503758623
+          },
+          "babyai": {
+            "questions": 128,
+            "modal_accuracy": 0.4140625,
+            "brier": 0.7052731307744691,
+            "cross_entropy": 1.4919952201710827
+          },
+          "community-diversity-v2/browser_tools/goal_completion": {
+            "questions": 24,
+            "modal_accuracy": 0.9583333333333334,
+            "brier": 0.0757580121004977,
+            "cross_entropy": 0.10499604055191543
+          },
+          "community-diversity-v2/contracts/exclusivity_exception": {
+            "questions": 23,
+            "modal_accuracy": 0.8695652173913043,
+            "brier": 0.17625150922502755,
+            "cross_entropy": 0.3819911750055886
+          },
+          "community-diversity-v2/games/pit_stop": {
+            "questions": 13,
+            "modal_accuracy": 0.46153846153846156,
+            "brier": 0.8058966314340112,
+            "cross_entropy": 1.7033569197410379
+          },
+          "community-diversity-v2/ir/warranty_transfer": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.13580214677968178,
+            "cross_entropy": 0.36657722104104
+          },
+          "community-diversity-v2/rag/independent_sources": {
+            "questions": 21,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5103346094744698,
+            "cross_entropy": 0.9877849897459301
+          },
+          "community-diversity-v2/rubric_judge/balanced_tradeoff": {
+            "questions": 12,
+            "modal_accuracy": 0.8333333333333334,
+            "brier": 0.31827468282287835,
+            "cross_entropy": 0.7584622273231553
+          },
+          "community-diversity-v2/shell_history/incremental_copy": {
+            "questions": 17,
+            "modal_accuracy": 0.8823529411764706,
+            "brier": 0.23477636259692025,
+            "cross_entropy": 0.5888464829632447
+          },
+          "community-diversity-v2/support/outage_compensation": {
+            "questions": 3,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.46034493119811687,
+            "cross_entropy": 1.0544000105768865
+          },
+          "connect_four": {
+            "questions": 128,
+            "modal_accuracy": 0.5234375,
+            "brier": 0.2605442517143562,
+            "cross_entropy": 1.8198101893170386
+          },
+          "customer-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003903831787449739,
+            "cross_entropy": 0.01599108917033031
+          },
+          "devign": {
+            "questions": 46,
+            "modal_accuracy": 0.5434782608695652,
+            "brier": 0.5038973081034425,
+            "cross_entropy": 0.6986062066022048
+          },
+          "entity_belief_tracking": {
+            "questions": 19,
+            "modal_accuracy": 1.0,
+            "brier": 0.003996155280999412,
+            "cross_entropy": 0.018453305162397043
+          },
+          "event_state_reconstruction": {
+            "questions": 30,
+            "modal_accuracy": 1.0,
+            "brier": 0.000632084906836889,
+            "cross_entropy": 0.0070833207931613705
+          },
+          "event_triggered_execution": {
+            "questions": 18,
+            "modal_accuracy": 0.8888888888888888,
+            "brier": 0.20065607784122677,
+            "cross_entropy": 0.3866436913361192
+          },
+          "evidence_sufficiency": {
+            "questions": 13,
+            "modal_accuracy": 0.7692307692307693,
+            "brier": 0.3885272040680112,
+            "cross_entropy": 0.7354956714769301
+          },
+          "goal_directed_planning": {
+            "questions": 19,
+            "modal_accuracy": 0.8421052631578947,
+            "brier": 0.24661119154460764,
+            "cross_entropy": 0.80410976975599
+          },
+          "graph_navigation": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5177916242601774,
+            "cross_entropy": 1.158343409907232
+          },
+          "gsm8k": {
+            "questions": 106,
+            "modal_accuracy": 0.7547169811320755,
+            "brier": 0.38874049122073956,
+            "cross_entropy": 0.7555982932600139
+          },
+          "instruction_execution": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.16643936427800027,
+            "cross_entropy": 0.5910813947667087
+          },
+          "multi_view_adjudication": {
+            "questions": 20,
+            "modal_accuracy": 0.95,
+            "brier": 0.07032918266188046,
+            "cross_entropy": 0.09175669834651938
+          },
+          "needle_retrieval": {
+            "questions": 5,
+            "modal_accuracy": 1.0,
+            "brier": 1.0762412043050325e-08,
+            "cross_entropy": 4.8760052363298926e-05
+          },
+          "partial_observation_calibration": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.02878926642066244,
+            "cross_entropy": 0.5423652505807739
+          },
+          "policy_applicability": {
+            "questions": 17,
+            "modal_accuracy": 1.0,
+            "brier": 0.000332556851194119,
+            "cross_entropy": 0.006342650317050527
+          },
+          "policy_under_uncertainty": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 0.055895947994874184,
+            "cross_entropy": 0.5045127678250978
+          },
+          "race": {
+            "questions": 104,
+            "modal_accuracy": 0.9134615384615384,
+            "brier": 0.15016846124700772,
+            "cross_entropy": 0.3446495798522507
+          },
+          "ragtruth": {
+            "questions": 128,
+            "modal_accuracy": 0.84375,
+            "brier": 0.2673585181473687,
+            "cross_entropy": 0.4660759647283446
+          },
+          "reasoning-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9111111111111111,
+            "brier": 0.11092651727785302,
+            "cross_entropy": 0.2669770296039986
+          },
+          "record_aggregation": {
+            "questions": 14,
+            "modal_accuracy": 0.7857142857142857,
+            "brier": 0.29174772725112047,
+            "cross_entropy": 0.45793008149173803
+          },
+          "record_eligibility": {
+            "questions": 8,
+            "modal_accuracy": 0.875,
+            "brier": 0.10770378005071689,
+            "cross_entropy": 0.13562242943700845
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.6006833831415513,
+            "cross_entropy": 1.7746254397586039
+          },
+          "snake-v1": {
+            "questions": 30,
+            "modal_accuracy": 0.9666666666666667,
+            "brier": 0.04547834255423711,
+            "cross_entropy": 0.25335150127501904
+          },
+          "state_perturbation": {
+            "questions": 11,
+            "modal_accuracy": 0.8181818181818182,
+            "brier": 0.3589651830631727,
+            "cross_entropy": 0.9294419729230835
+          },
+          "table_lookup": {
+            "questions": 13,
+            "modal_accuracy": 0.8461538461538461,
+            "brier": 0.1995349559267647,
+            "cross_entropy": 0.7888754577823849
+          },
+          "taxonomy_routing": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 4.5835879900615156e-05,
+            "cross_entropy": 0.002082040171618654
+          },
+          "tic_tac_toe-v1": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.2455337950398313,
+            "cross_entropy": 1.1492408686263713
+          },
+          "tile_platformer-v1": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 0.023323220046764518,
+            "cross_entropy": 0.08534179201524755
+          },
+          "toolace": {
+            "questions": 128,
+            "modal_accuracy": 0.9921875,
+            "brier": 0.015580959620615913,
+            "cross_entropy": 0.043558856167122205
+          },
+          "trex_runner-v1": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.01809983422643395,
+            "cross_entropy": 0.11356731402761971
+          },
+          "vizdoom-basic-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9555555555555556,
+            "brier": 0.08476731050032603,
+            "cross_entropy": 0.18352009286167528
+          },
+          "wanli-decisions-v1": {
+            "questions": 128,
+            "modal_accuracy": 0.6953125,
+            "brier": 0.5342434192250224,
+            "cross_entropy": 1.027889201322489
+          },
+          "workflow-controls-v1/agent_trace_observability": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.13236608755793375,
+            "cross_entropy": 0.37446550855959687
+          },
+          "workflow-controls-v1/customer_service": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 9.801834989730725e-07,
+            "cross_entropy": 0.00023351213246618595
+          },
+          "workflow-controls-v1/invoice_processing": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 4.168477226094833e-07,
+            "cross_entropy": 0.00024165434698863794
+          },
+          "workflow-controls-v1/security_incidents": {
+            "questions": 14,
+            "modal_accuracy": 1.0,
+            "brier": 2.5913148618252294e-09,
+            "cross_entropy": 1.9618643712043932e-05
+          },
+          "yuri_v3": {
+            "questions": 256,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.03336715127934959,
+            "cross_entropy": 0.8210925298273019
+          }
+        }
+      },
+      "transfer": {
+        "questions": 89,
+        "modal_accuracy": 0.6404494382022472,
+        "brier": 0.49972145070841484,
+        "cross_entropy": 1.069735656698559,
+        "by_kind": {
+          "choice": {
+            "questions": 33,
+            "modal_accuracy": 0.6060606060606061,
+            "brier": 0.5839105237267139,
+            "cross_entropy": 1.3151504789912494
+          },
+          "noul": {
+            "questions": 41,
+            "modal_accuracy": 0.7560975609756098,
+            "brier": 0.34728857025217713,
+            "cross_entropy": 0.6781416340204177
+          },
+          "score": {
+            "questions": 15,
+            "modal_accuracy": 0.4,
+            "brier": 0.7311553633152064,
+            "cross_entropy": 1.6001800429748936
+          }
+        },
+        "by_source": {
+          "basic_cls": {
+            "questions": 12,
+            "modal_accuracy": 0.5833333333333334,
+            "brier": 0.4769190960406326,
+            "cross_entropy": 0.7929537222990599
+          },
+          "code": {
+            "questions": 6,
+            "modal_accuracy": 0.5,
+            "brier": 0.6355124607251713,
+            "cross_entropy": 0.9367448318704151
+          },
+          "dialog": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.4233092995278946,
+            "cross_entropy": 1.4220052117820898
+          },
+          "judge": {
+            "questions": 8,
+            "modal_accuracy": 0.75,
+            "brier": 0.4294109009179358,
+            "cross_entropy": 1.1813213005452772
+          },
+          "know": {
+            "questions": 10,
+            "modal_accuracy": 0.8,
+            "brier": 0.22602505333734763,
+            "cross_entropy": 0.39225738115288933
+          },
+          "ret": {
+            "questions": 10,
+            "modal_accuracy": 0.4,
+            "brier": 0.9045065751061194,
+            "cross_entropy": 1.655086462390568
+          },
+          "route": {
+            "questions": 9,
+            "modal_accuracy": 0.4444444444444444,
+            "brier": 0.6393115918944736,
+            "cross_entropy": 1.1139742544661488
+          },
+          "safety": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.17156815175055062,
+            "cross_entropy": 0.4272056920862004
+          },
+          "step": {
+            "questions": 8,
+            "modal_accuracy": 0.625,
+            "brier": 0.710965126411655,
+            "cross_entropy": 1.9533480392831426
+          },
+          "struct": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.8092234753736499,
+            "cross_entropy": 1.1517337483418988
+          }
+        }
+      }
+    },
+    "state-32k-nvfp4": {
+      "new_validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.76171875,
+        "brier": 0.27786017825450965,
+        "cross_entropy": 0.9927829125866763,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.7654135338345864,
+            "brier": 0.3189465668614991,
+            "cross_entropy": 1.0534065611001757
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.8958333333333334,
+            "brier": 0.16471210037099535,
+            "cross_entropy": 0.39482479972079665
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.6604651162790698,
+            "brier": 0.22656192147115203,
+            "cross_entropy": 1.2057654334760237
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1790825849971539,
+            "cross_entropy": 0.6080971490404414
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5263100827359132,
+            "cross_entropy": 1.403012675529207
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.5517241379310345,
+            "brier": 0.684833327087956,
+            "cross_entropy": 1.5818612127101548
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 8.422208826818114e-06,
+            "cross_entropy": 0.0016334122276107624
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.39072800981047207,
+            "cross_entropy": 0.8926558622999824
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.8571428571428571,
+            "brier": 0.28439123582360026,
+            "cross_entropy": 0.7928804085918381
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6286826358235,
+            "cross_entropy": 2.0004531092707
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.44538602593083176,
+            "cross_entropy": 1.117689014223367
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.25,
+            "brier": 1.4342464951649092,
+            "cross_entropy": 3.381933827755728
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5535714285714286,
+            "brier": 0.2622696208064482,
+            "cross_entropy": 1.8219529101323455
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.417104630113183e-06,
+            "cross_entropy": 0.0005900334430327367
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.43780762834608883,
+            "cross_entropy": 0.6306076256145883
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003744313065983422,
+            "cross_entropy": 0.011988960485765076
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.20697473621325052,
+            "cross_entropy": 1.5034032587243362
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.9635416666666666,
+            "brier": 0.11802009440200961,
+            "cross_entropy": 0.6579051066101222
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.7315329429054658,
+            "cross_entropy": 2.0647249708287245
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.609860991064251,
+            "cross_entropy": 0.9741778978250341
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.703125,
+            "brier": 0.44470820194957994,
+            "cross_entropy": 1.2102900099661185
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.796875,
+            "brier": 0.29338767561802165,
+            "cross_entropy": 0.6140028830315581
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.1804417353162663e-07,
+            "cross_entropy": 0.0003234946827488933
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7169811320754716,
+            "brier": 0.056977519481840835,
+            "cross_entropy": 0.860752644211149
+          }
+        }
+      },
+      "new_test": {
+        "questions": 512,
+        "modal_accuracy": 0.755859375,
+        "brier": 0.2705710538872297,
+        "cross_entropy": 1.0354930422480668,
+        "by_kind": {
+          "choice": {
+            "questions": 241,
+            "modal_accuracy": 0.8215767634854771,
+            "brier": 0.30593746240810665,
+            "cross_entropy": 1.0288481446263855
+          },
+          "noul": {
+            "questions": 94,
+            "modal_accuracy": 0.8936170212765957,
+            "brier": 0.1853453372075174,
+            "cross_entropy": 0.4718470608079094
+          },
+          "score": {
+            "questions": 177,
+            "modal_accuracy": 0.5932203389830508,
+            "brier": 0.26767790651074164,
+            "cross_entropy": 1.34387802858818
+          }
+        },
+        "by_source": {
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5078125,
+            "brier": 0.22898458428820745,
+            "cross_entropy": 1.5346420909767997
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.921875,
+            "brier": 0.1909474730913934,
+            "cross_entropy": 0.8915435249464682
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.7552083333333334,
+            "brier": 0.3779189477490808,
+            "cross_entropy": 0.8466765270638437
+          }
+        }
+      },
+      "test": {
+        "questions": 2048,
+        "modal_accuracy": 0.79345703125,
+        "brier": 0.2641515323526975,
+        "cross_entropy": 0.784624370615209,
+        "by_kind": {
+          "choice": {
+            "questions": 1151,
+            "modal_accuracy": 0.7193744569939183,
+            "brier": 0.36420584491384933,
+            "cross_entropy": 1.0683460554623134
+          },
+          "noul": {
+            "questions": 649,
+            "modal_accuracy": 0.9090909090909091,
+            "brier": 0.12795487547550322,
+            "cross_entropy": 0.3201729560342784
+          },
+          "score": {
+            "questions": 248,
+            "modal_accuracy": 0.8346774193548387,
+            "brier": 0.1562044216890414,
+            "cross_entropy": 0.6832748093410428
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1662207666853679,
+            "cross_entropy": 0.4678064926902018
+          },
+          "arithmetic": {
+            "questions": 30,
+            "modal_accuracy": 0.7333333333333333,
+            "brier": 0.5001721787123533,
+            "cross_entropy": 1.2885683809129926
+          },
+          "babyai": {
+            "questions": 128,
+            "modal_accuracy": 0.484375,
+            "brier": 0.7067526312424577,
+            "cross_entropy": 1.6676958019283898
+          },
+          "community-diversity-v2/browser_tools/goal_completion": {
+            "questions": 24,
+            "modal_accuracy": 0.9583333333333334,
+            "brier": 0.06921012297338076,
+            "cross_entropy": 0.10122484092336533
+          },
+          "community-diversity-v2/contracts/exclusivity_exception": {
+            "questions": 23,
+            "modal_accuracy": 0.8695652173913043,
+            "brier": 0.24976225264440965,
+            "cross_entropy": 0.5909170782174602
+          },
+          "community-diversity-v2/games/pit_stop": {
+            "questions": 13,
+            "modal_accuracy": 0.7692307692307693,
+            "brier": 0.4785595907160497,
+            "cross_entropy": 1.2971969110502473
+          },
+          "community-diversity-v2/ir/warranty_transfer": {
+            "questions": 15,
+            "modal_accuracy": 0.8666666666666667,
+            "brier": 0.14068735459076726,
+            "cross_entropy": 0.17808536247316048
+          },
+          "community-diversity-v2/rag/independent_sources": {
+            "questions": 21,
+            "modal_accuracy": 0.47619047619047616,
+            "brier": 0.9915299974666058,
+            "cross_entropy": 2.310007596806081
+          },
+          "community-diversity-v2/rubric_judge/balanced_tradeoff": {
+            "questions": 12,
+            "modal_accuracy": 0.75,
+            "brier": 0.4814089394948157,
+            "cross_entropy": 1.102602184903888
+          },
+          "community-diversity-v2/shell_history/incremental_copy": {
+            "questions": 17,
+            "modal_accuracy": 0.8823529411764706,
+            "brier": 0.20441354684881466,
+            "cross_entropy": 0.5009642986702474
+          },
+          "community-diversity-v2/support/outage_compensation": {
+            "questions": 3,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.4884574890213706,
+            "cross_entropy": 1.3886592311282289
+          },
+          "connect_four": {
+            "questions": 128,
+            "modal_accuracy": 0.59375,
+            "brier": 0.2561712647462938,
+            "cross_entropy": 1.802544508268942
+          },
+          "customer-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 1.0,
+            "brier": 0.00019936847019355491,
+            "cross_entropy": 0.015971454914726523
+          },
+          "devign": {
+            "questions": 46,
+            "modal_accuracy": 0.5217391304347826,
+            "brier": 0.5225763998633566,
+            "cross_entropy": 0.722933623300632
+          },
+          "entity_belief_tracking": {
+            "questions": 19,
+            "modal_accuracy": 0.8421052631578947,
+            "brier": 0.259415578060587,
+            "cross_entropy": 0.6433967485445069
+          },
+          "event_state_reconstruction": {
+            "questions": 30,
+            "modal_accuracy": 1.0,
+            "brier": 0.015897166327449284,
+            "cross_entropy": 0.02589146516518676
+          },
+          "event_triggered_execution": {
+            "questions": 18,
+            "modal_accuracy": 0.8888888888888888,
+            "brier": 0.16562186759207273,
+            "cross_entropy": 0.2757516689442136
+          },
+          "evidence_sufficiency": {
+            "questions": 13,
+            "modal_accuracy": 0.6923076923076923,
+            "brier": 0.5880021120988826,
+            "cross_entropy": 1.4889127617030355
+          },
+          "goal_directed_planning": {
+            "questions": 19,
+            "modal_accuracy": 0.8421052631578947,
+            "brier": 0.2509895082794504,
+            "cross_entropy": 0.4540842796343451
+          },
+          "graph_navigation": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.44283093090962034,
+            "cross_entropy": 1.3194420415465578
+          },
+          "gsm8k": {
+            "questions": 106,
+            "modal_accuracy": 0.7075471698113207,
+            "brier": 0.43472841661725453,
+            "cross_entropy": 0.8890682888838121
+          },
+          "instruction_execution": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.16555489236402793,
+            "cross_entropy": 0.49206138807591987
+          },
+          "multi_view_adjudication": {
+            "questions": 20,
+            "modal_accuracy": 1.0,
+            "brier": 5.2326944475445606e-05,
+            "cross_entropy": 0.0019525076494628021
+          },
+          "needle_retrieval": {
+            "questions": 5,
+            "modal_accuracy": 1.0,
+            "brier": 3.199473123353177e-08,
+            "cross_entropy": 8.974482553742138e-05
+          },
+          "partial_observation_calibration": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.02661607292190903,
+            "cross_entropy": 0.5399351074049796
+          },
+          "policy_applicability": {
+            "questions": 17,
+            "modal_accuracy": 1.0,
+            "brier": 6.106492112719508e-05,
+            "cross_entropy": 0.003675418740395231
+          },
+          "policy_under_uncertainty": {
+            "questions": 8,
+            "modal_accuracy": 0.875,
+            "brier": 0.10177113114450002,
+            "cross_entropy": 0.560772453887988
+          },
+          "race": {
+            "questions": 104,
+            "modal_accuracy": 0.9134615384615384,
+            "brier": 0.16990537457587326,
+            "cross_entropy": 0.3822661135973287
+          },
+          "ragtruth": {
+            "questions": 128,
+            "modal_accuracy": 0.8515625,
+            "brier": 0.2317892915096781,
+            "cross_entropy": 0.39549112820971377
+          },
+          "reasoning-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9111111111111111,
+            "brier": 0.1285355661042637,
+            "cross_entropy": 0.30974376864438463
+          },
+          "record_aggregation": {
+            "questions": 14,
+            "modal_accuracy": 0.7857142857142857,
+            "brier": 0.35331876706309845,
+            "cross_entropy": 0.696535349171323
+          },
+          "record_eligibility": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 0.00021253785775029812,
+            "cross_entropy": 0.0043067743592292455
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.5625,
+            "brier": 0.6406304488569265,
+            "cross_entropy": 1.8806145121011755
+          },
+          "snake-v1": {
+            "questions": 30,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.04564391684280246,
+            "cross_entropy": 0.2431433189091872
+          },
+          "state_perturbation": {
+            "questions": 11,
+            "modal_accuracy": 1.0,
+            "brier": 0.0002344655717187281,
+            "cross_entropy": 0.005994309073216685
+          },
+          "table_lookup": {
+            "questions": 13,
+            "modal_accuracy": 0.9230769230769231,
+            "brier": 0.1498702753732909,
+            "cross_entropy": 0.3551783229099786
+          },
+          "taxonomy_routing": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 0.045424091680658744,
+            "cross_entropy": 0.07014486556652395
+          },
+          "tic_tac_toe-v1": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.26386162320957124,
+            "cross_entropy": 1.1837003824426573
+          },
+          "tile_platformer-v1": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.10164899419557688,
+            "cross_entropy": 0.220570822391115
+          },
+          "toolace": {
+            "questions": 128,
+            "modal_accuracy": 0.9921875,
+            "brier": 0.01562046345535589,
+            "cross_entropy": 0.04367168294967396
+          },
+          "trex_runner-v1": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.36032773502188475,
+            "cross_entropy": 0.5817032440936637
+          },
+          "vizdoom-basic-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9555555555555556,
+            "brier": 0.08693939517676995,
+            "cross_entropy": 0.20638551041670192
+          },
+          "wanli-decisions-v1": {
+            "questions": 128,
+            "modal_accuracy": 0.6875,
+            "brier": 0.5529247926341857,
+            "cross_entropy": 1.0802996209215674
+          },
+          "workflow-controls-v1/agent_trace_observability": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.13168080686035147,
+            "cross_entropy": 0.3395590286213042
+          },
+          "workflow-controls-v1/customer_service": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 7.55085699524451e-06,
+            "cross_entropy": 0.0007668466946998275
+          },
+          "workflow-controls-v1/invoice_processing": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 1.1169422484519878e-06,
+            "cross_entropy": 0.0002747598457579768
+          },
+          "workflow-controls-v1/security_incidents": {
+            "questions": 14,
+            "modal_accuracy": 1.0,
+            "brier": 2.4204779837314167e-09,
+            "cross_entropy": 2.2446460647594408e-05
+          },
+          "yuri_v3": {
+            "questions": 256,
+            "modal_accuracy": 0.83203125,
+            "brier": 0.03526590762889966,
+            "cross_entropy": 0.8245288048118414
+          }
+        }
+      },
+      "transfer": {
+        "questions": 89,
+        "modal_accuracy": 0.651685393258427,
+        "brier": 0.47914605790680204,
+        "cross_entropy": 1.0043952783379058,
+        "by_kind": {
+          "choice": {
+            "questions": 33,
+            "modal_accuracy": 0.6363636363636364,
+            "brier": 0.5172571933930417,
+            "cross_entropy": 1.2048737574594803
+          },
+          "noul": {
+            "questions": 41,
+            "modal_accuracy": 0.7317073170731707,
+            "brier": 0.3731759473027374,
+            "cross_entropy": 0.7052377430670229
+          },
+          "score": {
+            "questions": 15,
+            "modal_accuracy": 0.4666666666666667,
+            "brier": 0.6849531954881848,
+            "cross_entropy": 1.381039887344188
+          }
+        },
+        "by_source": {
+          "basic_cls": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.4038371458185728,
+            "cross_entropy": 0.7355992645221435
+          },
+          "code": {
+            "questions": 6,
+            "modal_accuracy": 0.3333333333333333,
+            "brier": 0.8287228425681598,
+            "cross_entropy": 1.262918501594028
+          },
+          "dialog": {
+            "questions": 12,
+            "modal_accuracy": 0.5833333333333334,
+            "brier": 0.30448197088852874,
+            "cross_entropy": 1.1186203509870534
+          },
+          "judge": {
+            "questions": 8,
+            "modal_accuracy": 0.875,
+            "brier": 0.3292730875017608,
+            "cross_entropy": 0.7769559856794094
+          },
+          "know": {
+            "questions": 10,
+            "modal_accuracy": 0.8,
+            "brier": 0.26539280354851125,
+            "cross_entropy": 0.4839723017070307
+          },
+          "ret": {
+            "questions": 10,
+            "modal_accuracy": 0.5,
+            "brier": 0.814367705138209,
+            "cross_entropy": 1.6245878852220568
+          },
+          "route": {
+            "questions": 9,
+            "modal_accuracy": 0.4444444444444444,
+            "brier": 0.7471958010258476,
+            "cross_entropy": 1.3369414656592875
+          },
+          "safety": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.1672822055997213,
+            "cross_entropy": 0.43712494222128856
+          },
+          "step": {
+            "questions": 8,
+            "modal_accuracy": 0.625,
+            "brier": 0.6510657904776266,
+            "cross_entropy": 1.5015904290656892
+          },
+          "struct": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.8996839553398099,
+            "cross_entropy": 1.4855438457791856
+          }
+        }
+      }
+    }
+  },
+  "scope": {
+    "new_validation": "1024 selection questions: 512 new and 512 previous; all four candidate checkpoints scored",
+    "new_test": "512 reserved new questions; first scored after checkpoint selection",
+    "test": "2048 earlier questions; reused to check retention of earlier abilities",
+    "transfer": "89 general questions; small reused development check",
+    "parent": "Previous actions-48k adapter, re-exported to byte-identical NVFP4 and rescored with the same Eider binary",
+    "calibration": "All scores are uncalibrated. No fitted profile for the new weights.",
+    "bf16": "Matching merged BF16 weights are included but were not independently scored in this native evaluation.",
+    "base_model": "The unfine-tuned NVIDIA model was not rescored in this evaluation."
+  }
+}

evaluation/runtime.json

@@ -0,0 +1,36 @@
+{
+  "server_sha256": "935335c9df451b4b9108d65fc46882abbfd7a0b6cb20b3ba76f4e8c42fa7ab0c",
+  "training_recipe_server_sha256": "2119b000902c32d33d57b2be9fc7630ff57b830818edc09a0a768a6e6dc8ebbd",
+  "reason": "Use one frozen copy of the current Eider binary for all candidates after unrelated module development.",
+  "scripts_sha256": {
+    "dory_lora.py": "cdcd877f30e8b2c6c9892a22281770125261423d22eec9890246496cb3d3ad26",
+    "dory-decision-corpus.py": "f81de0dc66b7c433a13b71d705d5650ed813c31fb600ba4f73e459191387c4c0",
+    "dory_nvfp4.py": "baf5ad4237ff4390c41696a32ba97cb3b82ec87fe0bb78ea8878909f0a1b3772",
+    "dory-decision-state.py": "3324e067b8e03913bcceff4cbe8a35675604f4f6a861ccb9f17750f6ef1a689d",
+    "dory-decision-lora.py": "6fdf5ba67a09b7ac75dbec8450a1044c277437b14340b7cd31e65d99015f309c",
+    "dory-decision-verda.py": "5c347955220c157d8fa4fb2bc157569245f6ab70419f98f5c141645d75ef9f1c",
+    "dory-decision-nvfp4.py": "cbd6c833654b1fff319abf1d8f3cb491af5d96b78d541e593a283b684268da04",
+    "sha256.json": "f0e48a77a313f739023f0ee23dd5c287c7197bb31ef61facc8c9eb3f5b4c1b49",
+    "dory-decision-targeted.py": "f78f2633a35fa96967a956ee8060cafb5861e4d65e449c3edb5e7dcbc84ded82",
+    "dory-decision-continuation.py": "b044299772c69184e256c593aa97cb33fc2d79a8f0bb1bab4c6230aedd46ba40",
+    "decision-eval.py": "15cf0b44d2d21ae2484e97dc298b4b3e25e541bedf653436b0c5f7431a982956",
+    "run-dory-decision-overnight.sh": "f12d624177c78a28a255b4b910e24b92d4883662076ddaf2afe3561288dd76f7",
+    "dory-decision-overnight.py": "0ffa9a9353edf2cb77ce8b9254182a1753a88a036517b1b9f199f8d4981a9c1a",
+    "dory_pytorch.py": "51582e13b8a8742f0597bf1cb6c3aaf2b1fa03b3d3f8a2e02a1577b0b58f6d0d",
+    "dory-decision-actions.py": "37e23a2d72b5738b5a977bd1625a385cdee9b014173ca6165d47d7b250c1b735"
+  },
+  "validation_split": "new_validation",
+  "test_splits": [
+    "new_test",
+    "test",
+    "transfer"
+  ],
+  "calibration": "uncalibrated",
+  "candidate_steps": [
+    0,
+    2000,
+    4000,
+    8000
+  ],
+  "parent_comparison_step": 0
+}

formats.json

@@ -0,0 +1,54 @@
+{
+  "release": "state-32k",
+  "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+  "files": {
+    "model-00001-of-00008.safetensors": {
+      "bytes": 1064423904,
+      "sha256": "f8ae89948adc35f42b2d735abc460bf20df89fe27063009e7023af236421943e"
+    },
+    "model-00002-of-00008.safetensors": {
+      "bytes": 1038413752,
+      "sha256": "09c16bdfe0f5fb160048a35f08ca998fa1e704efdcb4b3d504e46b2870c68e2d"
+    },
+    "model-00003-of-00008.safetensors": {
+      "bytes": 1064623416,
+      "sha256": "978f4d0f66dce924a81ee2e57aaaa4d0431bf5a163eb3445f32414637846b74c"
+    },
+    "model-00004-of-00008.safetensors": {
+      "bytes": 1064650776,
+      "sha256": "056ce5b367b2d955ce08ec5795d12ba6475640df7175c9f3eb7d5d35e18ddd86"
+    },
+    "model-00005-of-00008.safetensors": {
+      "bytes": 1054137760,
+      "sha256": "dbc45d99c137991456115cd9b71e8fe69dcb7d5b923cee93835e34646af9da0d"
+    },
+    "model-00006-of-00008.safetensors": {
+      "bytes": 1038413752,
+      "sha256": "bd559f8be05597b9915e91c8bd449d7c5194a2205c5a02e4ec750971a3f2814e"
+    },
+    "model-00007-of-00008.safetensors": {
+      "bytes": 954762648,
+      "sha256": "6c1ebd7492594617d7e0be46cce8b05d57f27930dac431bff1c23a3d496782a3"
+    },
+    "model-00008-of-00008.safetensors": {
+      "bytes": 671088768,
+      "sha256": "09dde4e45f4cc217f00222f3c57d8b8656d12a8f215e09d300a6f98d434fe80e"
+    },
+    "adapter/adapter.safetensors": {
+      "bytes": 64941920,
+      "sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d"
+    },
+    "nvfp4/model.safetensors": {
+      "bytes": 3202498032,
+      "sha256": "9bc834f4dce29a48499b6614c11772d189710138b29c637a384f753b1326b4c0"
+    },
+    "nvfp4/adapter/adapter.safetensors": {
+      "bytes": 64941920,
+      "sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d"
+    },
+    "dory-decisions-Q8_0.gguf": {
+      "bytes": 4232655616,
+      "sha256": "bb36b8ed263ce633649ccfad9c6e1dc065e9c3d333624fc27028f496089a38b8"
+    }
+  }
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": [
+    2,
+    11
+  ],
+  "pad_token_id": 11,
+  "top_p": 0.95,
+  "temperature": 1.0,
+  "transformers_version": "4.57.6"
+}

gguf/gpu-boundaries.json

@@ -0,0 +1,86 @@
+{
+  "alternate_answers": {
+    "model": "eider-dory-decisions-gguf",
+    "answers": {
+      "colour": {
+        "type": "choice",
+        "choice": "blue",
+        "probabilities": {
+          "red": 6.588990763500258e-05,
+          "blue": 0.999834129171347,
+          "green": 9.998092101819787e-05
+        },
+        "confidence": 0.9997511937570203
+      },
+      "locked": {
+        "type": "noul",
+        "noul": 0.000527313024721127
+      },
+      "progress": {
+        "type": "score",
+        "score": 1.997173953179149,
+        "legend": {
+          "0": "Not started",
+          "1": "Partly complete",
+          "2": "Complete"
+        },
+        "probabilities": {
+          "0": 0.0010355160597244543,
+          "1": 0.0007550147014019543,
+          "2": 0.9982094692388735
+        },
+        "confidence": 0.9957609297687237
+      }
+    },
+    "usage": {
+      "input_tokens": 333,
+      "output_tokens": 0
+    }
+  },
+  "invalid_inputs": [
+    {
+      "case": "one_choice",
+      "status": 500,
+      "response": {
+        "error": {
+          "code": 500,
+          "message": "\n------------\nWhile executing CallExpression at line 8, column 23 in source:\n...| length > 64 -%}\u21b5    {{ raise_exception('Dory decisions require 2 to 64 options...\n                                           ^\nError: Jinja Exception: Dory decisions require 2 to 64 options",
+          "type": "server_error"
+        }
+      }
+    },
+    {
+      "case": "choices65",
+      "status": 500,
+      "response": {
+        "error": {
+          "code": 500,
+          "message": "\n------------\nWhile executing CallExpression at line 8, column 23 in source:\n...| length > 64 -%}\u21b5    {{ raise_exception('Dory decisions require 2 to 64 options...\n                                           ^\nError: Jinja Exception: Dory decisions require 2 to 64 options",
+          "type": "server_error"
+        }
+      }
+    },
+    {
+      "case": "scores11",
+      "status": 400,
+      "response": {
+        "error": {
+          "code": 400,
+          "message": "questions.q: \"criteria\" must be an array of 2 to 10 levels",
+          "type": "invalid_request_error"
+        }
+      }
+    },
+    {
+      "case": "object_description",
+      "status": 500,
+      "response": {
+        "error": {
+          "code": 500,
+          "message": "\n------------\nWhile executing CallExpression at line 54, column 27 in source:\n... is none) -%}\u21b5        {{ raise_exception('Option descriptions must be text, or n...\n                                           ^\nError: Jinja Exception: Option descriptions must be text, or null for a choice",
+          "type": "server_error"
+        }
+      }
+    }
+  ]
+}

gguf/gpu-smoke.json

@@ -0,0 +1,879 @@
+{
+  "server_pid": 1986089,
+  "device": "NVIDIA GB10",
+  "quantization": "Q8_0",
+  "gpu_layers": "all",
+  "rows": [
+    {
+      "mode": "repeated_state",
+      "questions": 3,
+      "repeat": 0,
+      "elapsed_ms": 199.72225278615952,
+      "max_probability_difference_from_cpu": 2.600803527597062e-05,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9992566868317317
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0016332087335073193,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9986680909317902,
+              "1": 0.0010306094029122327,
+              "2": 0.00030129966529754334
+            },
+            "confidence": 0.9975501868997391
+          }
+        },
+        "usage": {
+          "input_tokens": 336,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 3,
+      "repeat": 1,
+      "elapsed_ms": 191.04570196941495,
+      "max_probability_difference_from_cpu": 2.600803527597062e-05,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9992566868317317
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0016332087335073193,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9986680909317902,
+              "1": 0.0010306094029122327,
+              "2": 0.00030129966529754334
+            },
+            "confidence": 0.9975501868997391
+          }
+        },
+        "usage": {
+          "input_tokens": 336,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 3,
+      "repeat": 2,
+      "elapsed_ms": 190.74565405026078,
+      "max_probability_difference_from_cpu": 2.600803527597062e-05,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9992566868317317
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0016332087335073193,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9986680909317902,
+              "1": 0.0010306094029122327,
+              "2": 0.00030129966529754334
+            },
+            "confidence": 0.9975501868997391
+          }
+        },
+        "usage": {
+          "input_tokens": 336,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 3,
+      "repeat": 3,
+      "elapsed_ms": 191.78658397868276,
+      "max_probability_difference_from_cpu": 2.600803527597062e-05,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9992566868317317
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0016332087335073193,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9986680909317902,
+              "1": 0.0010306094029122327,
+              "2": 0.00030129966529754334
+            },
+            "confidence": 0.9975501868997391
+          }
+        },
+        "usage": {
+          "input_tokens": 336,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 3,
+      "repeat": 4,
+      "elapsed_ms": 191.52610376477242,
+      "max_probability_difference_from_cpu": 2.600803527597062e-05,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9992566868317317
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0016332087335073193,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9986680909317902,
+              "1": 0.0010306094029122327,
+              "2": 0.00030129966529754334
+            },
+            "confidence": 0.9975501868997391
+          }
+        },
+        "usage": {
+          "input_tokens": 336,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 3,
+      "repeat": 5,
+      "elapsed_ms": 190.9664859995246,
+      "max_probability_difference_from_cpu": 2.600803527597062e-05,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9992566868317317
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0016332087335073193,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9986680909317902,
+              "1": 0.0010306094029122327,
+              "2": 0.00030129966529754334
+            },
+            "confidence": 0.9975501868997391
+          }
+        },
+        "usage": {
+          "input_tokens": 336,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 1,
+      "repeat": 0,
+      "elapsed_ms": 65.5279322527349,
+      "max_probability_difference_from_cpu": 6.176286291650234e-06,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          }
+        },
+        "usage": {
+          "input_tokens": 115,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 1,
+      "repeat": 1,
+      "elapsed_ms": 81.84757502749562,
+      "max_probability_difference_from_cpu": 6.176286291650234e-06,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          }
+        },
+        "usage": {
+          "input_tokens": 115,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 1,
+      "repeat": 2,
+      "elapsed_ms": 60.70876680314541,
+      "max_probability_difference_from_cpu": 6.176286291650234e-06,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          }
+        },
+        "usage": {
+          "input_tokens": 115,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 1,
+      "repeat": 3,
+      "elapsed_ms": 59.67060476541519,
+      "max_probability_difference_from_cpu": 6.176286291650234e-06,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          }
+        },
+        "usage": {
+          "input_tokens": 115,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 1,
+      "repeat": 4,
+      "elapsed_ms": 59.59377298131585,
+      "max_probability_difference_from_cpu": 6.176286291650234e-06,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          }
+        },
+        "usage": {
+          "input_tokens": 115,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "repeated_state",
+      "questions": 1,
+      "repeat": 5,
+      "elapsed_ms": 62.094338703900576,
+      "max_probability_difference_from_cpu": 6.176286291650234e-06,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999704513197298,
+              "blue": 0.00015332330877705514,
+              "green": 0.00014216349392486737
+            },
+            "confidence": 0.9995567697959469
+          }
+        },
+        "usage": {
+          "input_tokens": 115,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 1,
+      "repeat": 0,
+      "elapsed_ms": 66.91363174468279,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995695545086536,
+              "blue": 0.0002143776232400631,
+              "green": 0.0002160678681065346
+            },
+            "confidence": 0.9993543317629804
+          }
+        },
+        "usage": {
+          "input_tokens": 121,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 1,
+      "repeat": 1,
+      "elapsed_ms": 72.32489390298724,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9997458575149862,
+              "blue": 0.0001332114240093469,
+              "green": 0.00012093106100444944
+            },
+            "confidence": 0.9996187862724794
+          }
+        },
+        "usage": {
+          "input_tokens": 121,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 1,
+      "repeat": 2,
+      "elapsed_ms": 61.78825814276934,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9996117671871932,
+              "blue": 0.0002076205382546569,
+              "green": 0.0001806122745522041
+            },
+            "confidence": 0.9994176507807897
+          }
+        },
+        "usage": {
+          "input_tokens": 121,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 1,
+      "repeat": 3,
+      "elapsed_ms": 61.55131384730339,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995588430391672,
+              "blue": 0.0002289865969598523,
+              "green": 0.00021217036387290148
+            },
+            "confidence": 0.9993382645587506
+          }
+        },
+        "usage": {
+          "input_tokens": 121,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 1,
+      "repeat": 4,
+      "elapsed_ms": 61.43116904422641,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995519861141342,
+              "blue": 0.00022869062477868097,
+              "green": 0.00021932326108723308
+            },
+            "confidence": 0.9993279791712013
+          }
+        },
+        "usage": {
+          "input_tokens": 121,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 1,
+      "repeat": 5,
+      "elapsed_ms": 61.040688306093216,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999539281617701,
+              "blue": 0.0002283309773867298,
+              "green": 0.0002323874049123223
+            },
+            "confidence": 0.9993089224265513
+          }
+        },
+        "usage": {
+          "input_tokens": 121,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 3,
+      "repeat": 0,
+      "elapsed_ms": 188.38033499196172,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999437078477227,
+              "blue": 0.0002762680579587071,
+              "green": 0.0002866534648141884
+            },
+            "confidence": 0.9991556177158403
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9987147361755384
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.004406003763708459,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9965169654679571,
+              "1": 0.0025600653003774155,
+              "2": 0.0009229692316655217
+            },
+            "confidence": 0.9933909943544373
+          }
+        },
+        "usage": {
+          "input_tokens": 354,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 3,
+      "repeat": 1,
+      "elapsed_ms": 190.96131809055805,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995368788686774,
+              "blue": 0.0002353511629641391,
+              "green": 0.00022776996835864894
+            },
+            "confidence": 0.999305318303016
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9989745477968868
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.002145674128292655,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.998301550520478,
+              "1": 0.001251224830751178,
+              "2": 0.0004472246487707385
+            },
+            "confidence": 0.996781488807561
+          }
+        },
+        "usage": {
+          "input_tokens": 354,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 3,
+      "repeat": 2,
+      "elapsed_ms": 194.13367891684175,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995370027976712,
+              "blue": 0.00022356741035164737,
+              "green": 0.0002394297919770094
+            },
+            "confidence": 0.9993055041965067
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9988616838788668
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0026604382123893754,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.997831678475456,
+              "1": 0.0016762048366985361,
+              "2": 0.0004921166878454197
+            },
+            "confidence": 0.9960093426814159
+          }
+        },
+        "usage": {
+          "input_tokens": 354,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 3,
+      "repeat": 3,
+      "elapsed_ms": 190.9974138252437,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995825700020634,
+              "blue": 0.00021005535720785526,
+              "green": 0.00020737464072878524
+            },
+            "confidence": 0.999373855003095
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9990223692866393
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0018735515686515714,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9984904455523493,
+              "1": 0.0011455573266497875,
+              "2": 0.0003639971210008919
+            },
+            "confidence": 0.9971896726470226
+          }
+        },
+        "usage": {
+          "input_tokens": 354,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 3,
+      "repeat": 4,
+      "elapsed_ms": 191.36837497353554,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.999511656501715,
+              "blue": 0.0002471284650324335,
+              "green": 0.00024121503325257176
+            },
+            "confidence": 0.9992674847525725
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9988051595905004
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.001991323914600817,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9984062479408564,
+              "1": 0.0011961802036863066,
+              "2": 0.0003975718554572552
+            },
+            "confidence": 0.9970130141280987
+          }
+        },
+        "usage": {
+          "input_tokens": 354,
+          "output_tokens": 0
+        }
+      }
+    },
+    {
+      "mode": "changed_state",
+      "questions": 3,
+      "repeat": 5,
+      "elapsed_ms": 194.05430229380727,
+      "max_probability_difference_from_cpu": null,
+      "response": {
+        "model": "eider-dory-decisions-gguf",
+        "answers": {
+          "colour": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9995076738557905,
+              "blue": 0.00023638462231822404,
+              "green": 0.0002559415218912954
+            },
+            "confidence": 0.9992615107836856
+          },
+          "locked": {
+            "type": "noul",
+            "noul": 0.9987609200486905
+          },
+          "progress": {
+            "type": "score",
+            "score": 0.0022313493409761936,
+            "legend": {
+              "0": "Not started",
+              "1": "Partly complete",
+              "2": "Complete"
+            },
+            "probabilities": {
+              "0": 0.9982206221748126,
+              "1": 0.0013274063093986507,
+              "2": 0.00045197151578877135
+            },
+            "confidence": 0.9966529759885357
+          }
+        },
+        "usage": {
+          "input_tokens": 354,
+          "output_tokens": 0
+        }
+      }
+    }
+  ],
+  "summary": {
+    "repeated_state_1": {
+      "first_ms": 65.5279322527349,
+      "warm_median_ms": 60.70876680314541
+    },
+    "repeated_state_3": {
+      "first_ms": 199.72225278615952,
+      "warm_median_ms": 191.04570196941495
+    },
+    "changed_state_1": {
+      "first_ms": 66.91363174468279,
+      "warm_median_ms": 61.55131384730339
+    },
+    "changed_state_3": {
+      "first_ms": 188.38033499196172,
+      "warm_median_ms": 191.36837497353554
+    }
+  },
+  "cpu_three_question_ms": 15091.274152044207
+}

gguf/prompt-parity.json

@@ -0,0 +1,4492 @@
+{
+  "rows": [
+    {
+      "id": "plain",
+      "state": "The key is red.",
+      "kind": "choice",
+      "instructions": "Which colour?",
+      "options": [
+        [
+          "red",
+          "Red"
+        ],
+        [
+          "blue",
+          "Blue"
+        ]
+      ]
+    },
+    {
+      "id": "json",
+      "state": {
+        "z": 2,
+        "a": {
+          "b": true
+        }
+      },
+      "kind": "noul",
+      "instructions": {
+        "z": "Is z two?",
+        "a": "Answer from state."
+      },
+      "options": [
+        [
+          "true",
+          "Yes"
+        ],
+        [
+          "false",
+          "No"
+        ]
+      ]
+    },
+    {
+      "id": "chat",
+      "state": [
+        {
+          "role": "system",
+          "content": "You are helpful."
+        },
+        {
+          "role": "user",
+          "content": "Pay for the red item."
+        },
+        {
+          "role": "assistant",
+          "content": "<think>Private</think>Paid."
+        }
+      ],
+      "kind": "score",
+      "instructions": "How complete?",
+      "options": [
+        [
+          "0",
+          "No progress"
+        ],
+        [
+          "1",
+          "Done"
+        ]
+      ]
+    },
+    {
+      "id": "escaping",
+      "state": {
+        "text": "caf\u00e9 <tag> 'quoted' & more\n"
+      },
+      "kind": "choice",
+      "instructions": "Choose \"a\".",
+      "options": [
+        [
+          "first",
+          null
+        ],
+        [
+          "second",
+          "\u00e9 \"ok\" <here>"
+        ]
+      ]
+    },
+    {
+      "id": "labels64",
+      "state": "Choose option 30.",
+      "kind": "choice",
+      "instructions": "Which option?",
+      "options": [
+        [
+          "0",
+          "0"
+        ],
+        [
+          "1",
+          "1"
+        ],
+        [
+          "2",
+          "2"
+        ],
+        [
+          "3",
+          "3"
+        ],
+        [
+          "4",
+          "4"
+        ],
+        [
+          "5",
+          "5"
+        ],
+        [
+          "6",
+          "6"
+        ],
+        [
+          "7",
+          "7"
+        ],
+        [
+          "8",
+          "8"
+        ],
+        [
+          "9",
+          "9"
+        ],
+        [
+          "10",
+          "10"
+        ],
+        [
+          "11",
+          "11"
+        ],
+        [
+          "12",
+          "12"
+        ],
+        [
+          "13",
+          "13"
+        ],
+        [
+          "14",
+          "14"
+        ],
+        [
+          "15",
+          "15"
+        ],
+        [
+          "16",
+          "16"
+        ],
+        [
+          "17",
+          "17"
+        ],
+        [
+          "18",
+          "18"
+        ],
+        [
+          "19",
+          "19"
+        ],
+        [
+          "20",
+          "20"
+        ],
+        [
+          "21",
+          "21"
+        ],
+        [
+          "22",
+          "22"
+        ],
+        [
+          "23",
+          "23"
+        ],
+        [
+          "24",
+          "24"
+        ],
+        [
+          "25",
+          "25"
+        ],
+        [
+          "26",
+          "26"
+        ],
+        [
+          "27",
+          "27"
+        ],
+        [
+          "28",
+          "28"
+        ],
+        [
+          "29",
+          "29"
+        ],
+        [
+          "30",
+          "30"
+        ],
+        [
+          "31",
+          "31"
+        ],
+        [
+          "32",
+          "32"
+        ],
+        [
+          "33",
+          "33"
+        ],
+        [
+          "34",
+          "34"
+        ],
+        [
+          "35",
+          "35"
+        ],
+        [
+          "36",
+          "36"
+        ],
+        [
+          "37",
+          "37"
+        ],
+        [
+          "38",
+          "38"
+        ],
+        [
+          "39",
+          "39"
+        ],
+        [
+          "40",
+          "40"
+        ],
+        [
+          "41",
+          "41"
+        ],
+        [
+          "42",
+          "42"
+        ],
+        [
+          "43",
+          "43"
+        ],
+        [
+          "44",
+          "44"
+        ],
+        [
+          "45",
+          "45"
+        ],
+        [
+          "46",
+          "46"
+        ],
+        [
+          "47",
+          "47"
+        ],
+        [
+          "48",
+          "48"
+        ],
+        [
+          "49",
+          "49"
+        ],
+        [
+          "50",
+          "50"
+        ],
+        [
+          "51",
+          "51"
+        ],
+        [
+          "52",
+          "52"
+        ],
+        [
+          "53",
+          "53"
+        ],
+        [
+          "54",
+          "54"
+        ],
+        [
+          "55",
+          "55"
+        ],
+        [
+          "56",
+          "56"
+        ],
+        [
+          "57",
+          "57"
+        ],
+        [
+          "58",
+          "58"
+        ],
+        [
+          "59",
+          "59"
+        ],
+        [
+          "60",
+          "60"
+        ],
+        [
+          "61",
+          "61"
+        ],
+        [
+          "62",
+          "62"
+        ],
+        [
+          "63",
+          "63"
+        ]
+      ]
+    },
+    {
+      "id": "text_parts",
+      "state": {
+        "messages": [
+          {
+            "role": "user",
+            "content": [
+              {
+                "type": "text",
+                "text": "First."
+              },
+              {
+                "type": "text",
+                "text": "Second."
+              }
+            ]
+          }
+        ]
+      },
+      "kind": "noul",
+      "instructions": "Is Second mentioned?",
+      "options": [
+        [
+          "true",
+          "Yes"
+        ],
+        [
+          "false",
+          "No"
+        ]
+      ]
+    },
+    {
+      "id": "noul_defaults",
+      "state": {
+        "z": 2,
+        "a": {
+          "b": true
+        }
+      },
+      "kind": "noul",
+      "instructions": {
+        "z": "Is z two?",
+        "a": "Answer from state."
+      },
+      "options": [
+        [
+          "true",
+          "Yes"
+        ],
+        [
+          "false",
+          "No"
+        ]
+      ]
+    },
+    {
+      "id": "score10",
+      "state": "The task is complete.",
+      "kind": "score",
+      "instructions": "How complete?",
+      "options": [
+        [
+          "0",
+          "0"
+        ],
+        [
+          "1",
+          "1"
+        ],
+        [
+          "2",
+          "2"
+        ],
+        [
+          "3",
+          "3"
+        ],
+        [
+          "4",
+          "4"
+        ],
+        [
+          "5",
+          "5"
+        ],
+        [
+          "6",
+          "6"
+        ],
+        [
+          "7",
+          "7"
+        ],
+        [
+          "8",
+          "8"
+        ],
+        [
+          "9",
+          "9"
+        ]
+      ]
+    }
+  ],
+  "requests": [
+    {
+      "state": "The key is red.",
+      "questions": {
+        "plain": {
+          "type": "choice",
+          "instructions": "Which colour?",
+          "criteria": {
+            "red": "Red",
+            "blue": "Blue"
+          }
+        }
+      }
+    },
+    {
+      "state": {
+        "z": 2,
+        "a": {
+          "b": true
+        }
+      },
+      "questions": {
+        "json": {
+          "type": "noul",
+          "instructions": {
+            "z": "Is z two?",
+            "a": "Answer from state."
+          },
+          "criteria": {
+            "true": "Yes",
+            "false": "No"
+          }
+        }
+      }
+    },
+    {
+      "state": [
+        {
+          "role": "system",
+          "content": "You are helpful."
+        },
+        {
+          "role": "user",
+          "content": "Pay for the red item."
+        },
+        {
+          "role": "assistant",
+          "content": "<think>Private</think>Paid."
+        }
+      ],
+      "questions": {
+        "chat": {
+          "type": "score",
+          "instructions": "How complete?",
+          "criteria": [
+            "No progress",
+            "Done"
+          ]
+        }
+      }
+    },
+    {
+      "state": {
+        "text": "caf\u00e9 <tag> 'quoted' & more\n"
+      },
+      "questions": {
+        "escaping": {
+          "type": "choice",
+          "instructions": "Choose \"a\".",
+          "criteria": {
+            "first": null,
+            "second": "\u00e9 \"ok\" <here>"
+          }
+        }
+      }
+    },
+    {
+      "state": "Choose option 30.",
+      "questions": {
+        "labels64": {
+          "type": "choice",
+          "instructions": "Which option?",
+          "criteria": {
+            "0": "0",
+            "1": "1",
+            "2": "2",
+            "3": "3",
+            "4": "4",
+            "5": "5",
+            "6": "6",
+            "7": "7",
+            "8": "8",
+            "9": "9",
+            "10": "10",
+            "11": "11",
+            "12": "12",
+            "13": "13",
+            "14": "14",
+            "15": "15",
+            "16": "16",
+            "17": "17",
+            "18": "18",
+            "19": "19",
+            "20": "20",
+            "21": "21",
+            "22": "22",
+            "23": "23",
+            "24": "24",
+            "25": "25",
+            "26": "26",
+            "27": "27",
+            "28": "28",
+            "29": "29",
+            "30": "30",
+            "31": "31",
+            "32": "32",
+            "33": "33",
+            "34": "34",
+            "35": "35",
+            "36": "36",
+            "37": "37",
+            "38": "38",
+            "39": "39",
+            "40": "40",
+            "41": "41",
+            "42": "42",
+            "43": "43",
+            "44": "44",
+            "45": "45",
+            "46": "46",
+            "47": "47",
+            "48": "48",
+            "49": "49",
+            "50": "50",
+            "51": "51",
+            "52": "52",
+            "53": "53",
+            "54": "54",
+            "55": "55",
+            "56": "56",
+            "57": "57",
+            "58": "58",
+            "59": "59",
+            "60": "60",
+            "61": "61",
+            "62": "62",
+            "63": "63"
+          }
+        }
+      }
+    },
+    {
+      "state": {
+        "messages": [
+          {
+            "role": "user",
+            "content": [
+              {
+                "type": "text",
+                "text": "First."
+              },
+              {
+                "type": "text",
+                "text": "Second."
+              }
+            ]
+          }
+        ]
+      },
+      "questions": {
+        "text_parts": {
+          "type": "noul",
+          "instructions": "Is Second mentioned?",
+          "criteria": {
+            "true": "Yes",
+            "false": "No"
+          }
+        }
+      }
+    },
+    {
+      "state": {
+        "z": 2,
+        "a": {
+          "b": true
+        }
+      },
+      "questions": {
+        "noul_defaults": {
+          "type": "noul",
+          "instructions": {
+            "z": "Is z two?",
+            "a": "Answer from state."
+          }
+        }
+      }
+    },
+    {
+      "state": "The task is complete.",
+      "questions": {
+        "score10": {
+          "type": "score",
+          "instructions": "How complete?",
+          "criteria": [
+            "0",
+            "1",
+            "2",
+            "3",
+            "4",
+            "5",
+            "6",
+            "7",
+            "8",
+            "9"
+          ]
+        }
+      }
+    }
+  ],
+  "eider": [
+    {
+      "id": "plain",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1784,
+        3844,
+        1395,
+        4804,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        27923,
+        22526,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        2338,
+        8011,
+        14653,
+        12592,
+        12846,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        23493,
+        8011,
+        14653,
+        12592,
+        24851,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ]
+      ]
+    },
+    {
+      "id": "json",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        19227,
+        1122,
+        2811,
+        1050,
+        4225,
+        1097,
+        90610,
+        1098,
+        2811,
+        5876,
+        2821,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        16753,
+        1122,
+        12592,
+        5356,
+        1554,
+        2295,
+        1063,
+        8011,
+        1097,
+        12592,
+        31106,
+        1562,
+        3468,
+        2613,
+        2092,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        5876,
+        8011,
+        14653,
+        12592,
+        16860,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        11339,
+        8011,
+        14653,
+        12592,
+        4753,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ]
+      ]
+    },
+    {
+      "id": "chat",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        4568,
+        1584,
+        20351,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        36807,
+        1394,
+        1278,
+        4804,
+        5610,
+        1046,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        124973,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        3075,
+        7662,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        1048,
+        8011,
+        14653,
+        12592,
+        4753,
+        8417,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        8011,
+        14653,
+        12592,
+        42617,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ]
+      ]
+    },
+    {
+      "id": "escaping",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        19227,
+        2149,
+        12592,
+        3173,
+        1102,
+        1337,
+        1534,
+        16593,
+        1062,
+        1576,
+        1348,
+        14744,
+        1039,
+        1748,
+        2081,
+        6250,
+        46005,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        64561,
+        1429,
+        1097,
+        7223,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        11517,
+        8011,
+        14653,
+        2811,
+        10267,
+        2002,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        13023,
+        8011,
+        14653,
+        12592,
+        1337,
+        25994,
+        1662,
+        17931,
+        1534,
+        18262,
+        30486,
+        1125,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ]
+      ]
+    },
+    {
+      "id": "labels64",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        64798,
+        7091,
+        1032,
+        1051,
+        1048,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        27923,
+        7091,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        1048,
+        8011,
+        14653,
+        12592,
+        1048,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        8011,
+        14653,
+        12592,
+        1049,
+        36745,
+        1067,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        8011,
+        14653,
+        12592,
+        1050,
+        36745,
+        1068,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        8011,
+        14653,
+        12592,
+        1051,
+        36745,
+        1069,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        8011,
+        14653,
+        12592,
+        1052,
+        36745,
+        1070,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        8011,
+        14653,
+        12592,
+        1053,
+        36745,
+        1071,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        8011,
+        14653,
+        12592,
+        1054,
+        36745,
+        1072,
+        1058,
+        16753,
+        6827,
+        12592,
+        1055,
+        8011,
+        14653,
+        12592,
+        1055,
+        36745,
+        1073,
+        1058,
+        16753,
+        6827,
+        12592,
+        1056,
+        8011,
+        14653,
+        12592,
+        1056,
+        36745,
+        1074,
+        1058,
+        16753,
+        6827,
+        12592,
+        1057,
+        8011,
+        14653,
+        12592,
+        1057,
+        36745,
+        1075,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1048,
+        8011,
+        14653,
+        12592,
+        1049,
+        1048,
+        36745,
+        1076,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1049,
+        8011,
+        14653,
+        12592,
+        1049,
+        1049,
+        36745,
+        1077,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1050,
+        8011,
+        14653,
+        12592,
+        1049,
+        1050,
+        36745,
+        1078,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1051,
+        8011,
+        14653,
+        12592,
+        1049,
+        1051,
+        36745,
+        1079,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1052,
+        8011,
+        14653,
+        12592,
+        1049,
+        1052,
+        36745,
+        1080,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1053,
+        8011,
+        14653,
+        12592,
+        1049,
+        1053,
+        36745,
+        1081,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1054,
+        8011,
+        14653,
+        12592,
+        1049,
+        1054,
+        36745,
+        1082,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1055,
+        8011,
+        14653,
+        12592,
+        1049,
+        1055,
+        36745,
+        1083,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1056,
+        8011,
+        14653,
+        12592,
+        1049,
+        1056,
+        36745,
+        1084,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1057,
+        8011,
+        14653,
+        12592,
+        1049,
+        1057,
+        36745,
+        1085,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1048,
+        8011,
+        14653,
+        12592,
+        1050,
+        1048,
+        36745,
+        1086,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1049,
+        8011,
+        14653,
+        12592,
+        1050,
+        1049,
+        36745,
+        1087,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1050,
+        8011,
+        14653,
+        12592,
+        1050,
+        1050,
+        36745,
+        1088,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1051,
+        8011,
+        14653,
+        12592,
+        1050,
+        1051,
+        36745,
+        1089,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1052,
+        8011,
+        14653,
+        12592,
+        1050,
+        1052,
+        36745,
+        1090,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1053,
+        8011,
+        14653,
+        12592,
+        1050,
+        1053,
+        36745,
+        14229,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1054,
+        8011,
+        14653,
+        12592,
+        1050,
+        1054,
+        36745,
+        8062,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1055,
+        8011,
+        14653,
+        12592,
+        1050,
+        1055,
+        36745,
+        5193,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1056,
+        8011,
+        14653,
+        12592,
+        1050,
+        1056,
+        36745,
+        5516,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1057,
+        8011,
+        14653,
+        12592,
+        1050,
+        1057,
+        36745,
+        32484,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1048,
+        8011,
+        14653,
+        12592,
+        1051,
+        1048,
+        36745,
+        12763,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1049,
+        8011,
+        14653,
+        12592,
+        1051,
+        1049,
+        36745,
+        6970,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1050,
+        8011,
+        14653,
+        12592,
+        1051,
+        1050,
+        36745,
+        91939,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1051,
+        8011,
+        14653,
+        12592,
+        1051,
+        1051,
+        36745,
+        33613,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1052,
+        8011,
+        14653,
+        12592,
+        1051,
+        1052,
+        36745,
+        26039,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1053,
+        8011,
+        14653,
+        12592,
+        1051,
+        1053,
+        36745,
+        4286,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1054,
+        8011,
+        14653,
+        12592,
+        1051,
+        1054,
+        36745,
+        6758,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1055,
+        8011,
+        14653,
+        12592,
+        1051,
+        1055,
+        36745,
+        5010,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1056,
+        8011,
+        14653,
+        12592,
+        1051,
+        1056,
+        36745,
+        106550,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1057,
+        8011,
+        14653,
+        12592,
+        1051,
+        1057,
+        36745,
+        6292,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1048,
+        8011,
+        14653,
+        12592,
+        1052,
+        1048,
+        36745,
+        3905,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1049,
+        8011,
+        14653,
+        12592,
+        1052,
+        1049,
+        36745,
+        6003,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1050,
+        8011,
+        14653,
+        12592,
+        1052,
+        1050,
+        36745,
+        3313,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1051,
+        8011,
+        14653,
+        12592,
+        1052,
+        1051,
+        36745,
+        117778,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1052,
+        8011,
+        14653,
+        12592,
+        1052,
+        1052,
+        36745,
+        29970,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1053,
+        8011,
+        14653,
+        12592,
+        1052,
+        1053,
+        36745,
+        13339,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1054,
+        8011,
+        14653,
+        12592,
+        1052,
+        1054,
+        36745,
+        25698,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1055,
+        8011,
+        14653,
+        12592,
+        1052,
+        1055,
+        36745,
+        75317,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1056,
+        8011,
+        14653,
+        12592,
+        1052,
+        1056,
+        36745,
+        10574,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1057,
+        8011,
+        14653,
+        12592,
+        1052,
+        1057,
+        36745,
+        28235,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1048,
+        8011,
+        14653,
+        12592,
+        1053,
+        1048,
+        36745,
+        6535,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1049,
+        8011,
+        14653,
+        12592,
+        1053,
+        1049,
+        36745,
+        18704,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1050,
+        8011,
+        14653,
+        12592,
+        1053,
+        1050,
+        36745,
+        34817,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1051,
+        8011,
+        14653,
+        12592,
+        1053,
+        1051,
+        36745,
+        71042,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1052,
+        8011,
+        14653,
+        12592,
+        1053,
+        1052,
+        36745,
+        87478,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1053,
+        8011,
+        14653,
+        12592,
+        1053,
+        1053,
+        36745,
+        17782,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1054,
+        8011,
+        14653,
+        12592,
+        1053,
+        1054,
+        36745,
+        119492,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1055,
+        8011,
+        14653,
+        12592,
+        1053,
+        1055,
+        36745,
+        30631,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1056,
+        8011,
+        14653,
+        12592,
+        1053,
+        1056,
+        36745,
+        20195,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1057,
+        8011,
+        14653,
+        12592,
+        1053,
+        1057,
+        36745,
+        90805,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1048,
+        8011,
+        14653,
+        12592,
+        1054,
+        1048,
+        36745,
+        18412,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1049,
+        8011,
+        14653,
+        12592,
+        1054,
+        1049,
+        36745,
+        22133,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1050,
+        8011,
+        14653,
+        12592,
+        1054,
+        1050,
+        36745,
+        35846,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1051,
+        8011,
+        14653,
+        12592,
+        1054,
+        1051,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ],
+        [
+          1067
+        ],
+        [
+          1068
+        ],
+        [
+          1069
+        ],
+        [
+          1070
+        ],
+        [
+          1071
+        ],
+        [
+          1072
+        ],
+        [
+          1073
+        ],
+        [
+          1074
+        ],
+        [
+          1075
+        ],
+        [
+          1076
+        ],
+        [
+          1077
+        ],
+        [
+          1078
+        ],
+        [
+          1079
+        ],
+        [
+          1080
+        ],
+        [
+          1081
+        ],
+        [
+          1082
+        ],
+        [
+          1083
+        ],
+        [
+          1084
+        ],
+        [
+          1085
+        ],
+        [
+          1086
+        ],
+        [
+          1087
+        ],
+        [
+          1088
+        ],
+        [
+          1089
+        ],
+        [
+          1090
+        ],
+        [
+          14229
+        ],
+        [
+          8062
+        ],
+        [
+          5193
+        ],
+        [
+          5516
+        ],
+        [
+          32484
+        ],
+        [
+          12763
+        ],
+        [
+          6970
+        ],
+        [
+          91939
+        ],
+        [
+          33613
+        ],
+        [
+          26039
+        ],
+        [
+          4286
+        ],
+        [
+          6758
+        ],
+        [
+          5010
+        ],
+        [
+          106550
+        ],
+        [
+          6292
+        ],
+        [
+          3905
+        ],
+        [
+          6003
+        ],
+        [
+          3313
+        ],
+        [
+          117778
+        ],
+        [
+          29970
+        ],
+        [
+          13339
+        ],
+        [
+          25698
+        ],
+        [
+          75317
+        ],
+        [
+          10574
+        ],
+        [
+          28235
+        ],
+        [
+          6535
+        ],
+        [
+          18704
+        ],
+        [
+          34817
+        ],
+        [
+          71042
+        ],
+        [
+          87478
+        ],
+        [
+          17782
+        ],
+        [
+          119492
+        ],
+        [
+          30631
+        ],
+        [
+          20195
+        ],
+        [
+          90805
+        ],
+        [
+          18412
+        ],
+        [
+          22133
+        ],
+        [
+          35846
+        ]
+      ]
+    },
+    {
+      "id": "text_parts",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        10107,
+        112283,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        3316,
+        10322,
+        10910,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        5876,
+        8011,
+        14653,
+        12592,
+        16860,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        11339,
+        8011,
+        14653,
+        12592,
+        4753,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ]
+      ]
+    },
+    {
+      "id": "noul_defaults",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        19227,
+        1122,
+        2811,
+        1050,
+        4225,
+        1097,
+        90610,
+        1098,
+        2811,
+        5876,
+        2821,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        16753,
+        1122,
+        12592,
+        5356,
+        1554,
+        2295,
+        1063,
+        8011,
+        1097,
+        12592,
+        31106,
+        1562,
+        3468,
+        2613,
+        2092,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        5876,
+        8011,
+        14653,
+        12592,
+        16860,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        11339,
+        8011,
+        14653,
+        12592,
+        4753,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ]
+      ]
+    },
+    {
+      "id": "score10",
+      "prefix_tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1784,
+        6255,
+        1395,
+        7662,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338
+      ],
+      "suffix_tokens": [
+        25229,
+        1058,
+        3075,
+        7662,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        1048,
+        8011,
+        14653,
+        12592,
+        1048,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        8011,
+        14653,
+        12592,
+        1049,
+        36745,
+        1067,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        8011,
+        14653,
+        12592,
+        1050,
+        36745,
+        1068,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        8011,
+        14653,
+        12592,
+        1051,
+        36745,
+        1069,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        8011,
+        14653,
+        12592,
+        1052,
+        36745,
+        1070,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        8011,
+        14653,
+        12592,
+        1053,
+        36745,
+        1071,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        8011,
+        14653,
+        12592,
+        1054,
+        36745,
+        1072,
+        1058,
+        16753,
+        6827,
+        12592,
+        1055,
+        8011,
+        14653,
+        12592,
+        1055,
+        36745,
+        1073,
+        1058,
+        16753,
+        6827,
+        12592,
+        1056,
+        8011,
+        14653,
+        12592,
+        1056,
+        36745,
+        1074,
+        1058,
+        16753,
+        6827,
+        12592,
+        1057,
+        8011,
+        14653,
+        12592,
+        1057,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        [
+          1065
+        ],
+        [
+          1066
+        ],
+        [
+          1067
+        ],
+        [
+          1068
+        ],
+        [
+          1069
+        ],
+        [
+          1070
+        ],
+        [
+          1071
+        ],
+        [
+          1072
+        ],
+        [
+          1073
+        ],
+        [
+          1074
+        ]
+      ]
+    }
+  ],
+  "llama": [
+    {
+      "id": "plain",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1784,
+        3844,
+        1395,
+        4804,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        27923,
+        22526,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        2338,
+        8011,
+        14653,
+        12592,
+        12846,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        23493,
+        8011,
+        14653,
+        12592,
+        24851,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066
+      ]
+    },
+    {
+      "id": "json",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        19227,
+        1122,
+        2811,
+        1050,
+        4225,
+        1097,
+        90610,
+        1098,
+        2811,
+        5876,
+        2821,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        16753,
+        1122,
+        12592,
+        5356,
+        1554,
+        2295,
+        1063,
+        8011,
+        1097,
+        12592,
+        31106,
+        1562,
+        3468,
+        2613,
+        2092,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        5876,
+        8011,
+        14653,
+        12592,
+        16860,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        11339,
+        8011,
+        14653,
+        12592,
+        4753,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066
+      ]
+    },
+    {
+      "id": "chat",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        4568,
+        1584,
+        20351,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        36807,
+        1394,
+        1278,
+        4804,
+        5610,
+        1046,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        124973,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        3075,
+        7662,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        1048,
+        8011,
+        14653,
+        12592,
+        4753,
+        8417,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        8011,
+        14653,
+        12592,
+        42617,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066
+      ]
+    },
+    {
+      "id": "escaping",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        19227,
+        2149,
+        12592,
+        3173,
+        1102,
+        1337,
+        1534,
+        16593,
+        1062,
+        1576,
+        1348,
+        14744,
+        1039,
+        1748,
+        2081,
+        6250,
+        46005,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        64561,
+        1429,
+        1097,
+        7223,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        11517,
+        8011,
+        14653,
+        2811,
+        10267,
+        2002,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        13023,
+        8011,
+        14653,
+        12592,
+        1337,
+        25994,
+        1662,
+        17931,
+        1534,
+        18262,
+        30486,
+        1125,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066
+      ]
+    },
+    {
+      "id": "labels64",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        64798,
+        7091,
+        1032,
+        1051,
+        1048,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        27923,
+        7091,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        1048,
+        8011,
+        14653,
+        12592,
+        1048,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        8011,
+        14653,
+        12592,
+        1049,
+        36745,
+        1067,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        8011,
+        14653,
+        12592,
+        1050,
+        36745,
+        1068,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        8011,
+        14653,
+        12592,
+        1051,
+        36745,
+        1069,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        8011,
+        14653,
+        12592,
+        1052,
+        36745,
+        1070,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        8011,
+        14653,
+        12592,
+        1053,
+        36745,
+        1071,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        8011,
+        14653,
+        12592,
+        1054,
+        36745,
+        1072,
+        1058,
+        16753,
+        6827,
+        12592,
+        1055,
+        8011,
+        14653,
+        12592,
+        1055,
+        36745,
+        1073,
+        1058,
+        16753,
+        6827,
+        12592,
+        1056,
+        8011,
+        14653,
+        12592,
+        1056,
+        36745,
+        1074,
+        1058,
+        16753,
+        6827,
+        12592,
+        1057,
+        8011,
+        14653,
+        12592,
+        1057,
+        36745,
+        1075,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1048,
+        8011,
+        14653,
+        12592,
+        1049,
+        1048,
+        36745,
+        1076,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1049,
+        8011,
+        14653,
+        12592,
+        1049,
+        1049,
+        36745,
+        1077,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1050,
+        8011,
+        14653,
+        12592,
+        1049,
+        1050,
+        36745,
+        1078,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1051,
+        8011,
+        14653,
+        12592,
+        1049,
+        1051,
+        36745,
+        1079,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1052,
+        8011,
+        14653,
+        12592,
+        1049,
+        1052,
+        36745,
+        1080,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1053,
+        8011,
+        14653,
+        12592,
+        1049,
+        1053,
+        36745,
+        1081,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1054,
+        8011,
+        14653,
+        12592,
+        1049,
+        1054,
+        36745,
+        1082,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1055,
+        8011,
+        14653,
+        12592,
+        1049,
+        1055,
+        36745,
+        1083,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1056,
+        8011,
+        14653,
+        12592,
+        1049,
+        1056,
+        36745,
+        1084,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        1057,
+        8011,
+        14653,
+        12592,
+        1049,
+        1057,
+        36745,
+        1085,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1048,
+        8011,
+        14653,
+        12592,
+        1050,
+        1048,
+        36745,
+        1086,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1049,
+        8011,
+        14653,
+        12592,
+        1050,
+        1049,
+        36745,
+        1087,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1050,
+        8011,
+        14653,
+        12592,
+        1050,
+        1050,
+        36745,
+        1088,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1051,
+        8011,
+        14653,
+        12592,
+        1050,
+        1051,
+        36745,
+        1089,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1052,
+        8011,
+        14653,
+        12592,
+        1050,
+        1052,
+        36745,
+        1090,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1053,
+        8011,
+        14653,
+        12592,
+        1050,
+        1053,
+        36745,
+        14229,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1054,
+        8011,
+        14653,
+        12592,
+        1050,
+        1054,
+        36745,
+        8062,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1055,
+        8011,
+        14653,
+        12592,
+        1050,
+        1055,
+        36745,
+        5193,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1056,
+        8011,
+        14653,
+        12592,
+        1050,
+        1056,
+        36745,
+        5516,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        1057,
+        8011,
+        14653,
+        12592,
+        1050,
+        1057,
+        36745,
+        32484,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1048,
+        8011,
+        14653,
+        12592,
+        1051,
+        1048,
+        36745,
+        12763,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1049,
+        8011,
+        14653,
+        12592,
+        1051,
+        1049,
+        36745,
+        6970,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1050,
+        8011,
+        14653,
+        12592,
+        1051,
+        1050,
+        36745,
+        91939,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1051,
+        8011,
+        14653,
+        12592,
+        1051,
+        1051,
+        36745,
+        33613,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1052,
+        8011,
+        14653,
+        12592,
+        1051,
+        1052,
+        36745,
+        26039,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1053,
+        8011,
+        14653,
+        12592,
+        1051,
+        1053,
+        36745,
+        4286,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1054,
+        8011,
+        14653,
+        12592,
+        1051,
+        1054,
+        36745,
+        6758,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1055,
+        8011,
+        14653,
+        12592,
+        1051,
+        1055,
+        36745,
+        5010,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1056,
+        8011,
+        14653,
+        12592,
+        1051,
+        1056,
+        36745,
+        106550,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        1057,
+        8011,
+        14653,
+        12592,
+        1051,
+        1057,
+        36745,
+        6292,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1048,
+        8011,
+        14653,
+        12592,
+        1052,
+        1048,
+        36745,
+        3905,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1049,
+        8011,
+        14653,
+        12592,
+        1052,
+        1049,
+        36745,
+        6003,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1050,
+        8011,
+        14653,
+        12592,
+        1052,
+        1050,
+        36745,
+        3313,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1051,
+        8011,
+        14653,
+        12592,
+        1052,
+        1051,
+        36745,
+        117778,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1052,
+        8011,
+        14653,
+        12592,
+        1052,
+        1052,
+        36745,
+        29970,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1053,
+        8011,
+        14653,
+        12592,
+        1052,
+        1053,
+        36745,
+        13339,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1054,
+        8011,
+        14653,
+        12592,
+        1052,
+        1054,
+        36745,
+        25698,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1055,
+        8011,
+        14653,
+        12592,
+        1052,
+        1055,
+        36745,
+        75317,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1056,
+        8011,
+        14653,
+        12592,
+        1052,
+        1056,
+        36745,
+        10574,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        1057,
+        8011,
+        14653,
+        12592,
+        1052,
+        1057,
+        36745,
+        28235,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1048,
+        8011,
+        14653,
+        12592,
+        1053,
+        1048,
+        36745,
+        6535,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1049,
+        8011,
+        14653,
+        12592,
+        1053,
+        1049,
+        36745,
+        18704,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1050,
+        8011,
+        14653,
+        12592,
+        1053,
+        1050,
+        36745,
+        34817,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1051,
+        8011,
+        14653,
+        12592,
+        1053,
+        1051,
+        36745,
+        71042,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1052,
+        8011,
+        14653,
+        12592,
+        1053,
+        1052,
+        36745,
+        87478,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1053,
+        8011,
+        14653,
+        12592,
+        1053,
+        1053,
+        36745,
+        17782,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1054,
+        8011,
+        14653,
+        12592,
+        1053,
+        1054,
+        36745,
+        119492,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1055,
+        8011,
+        14653,
+        12592,
+        1053,
+        1055,
+        36745,
+        30631,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1056,
+        8011,
+        14653,
+        12592,
+        1053,
+        1056,
+        36745,
+        20195,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        1057,
+        8011,
+        14653,
+        12592,
+        1053,
+        1057,
+        36745,
+        90805,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1048,
+        8011,
+        14653,
+        12592,
+        1054,
+        1048,
+        36745,
+        18412,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1049,
+        8011,
+        14653,
+        12592,
+        1054,
+        1049,
+        36745,
+        22133,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1050,
+        8011,
+        14653,
+        12592,
+        1054,
+        1050,
+        36745,
+        35846,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        1051,
+        8011,
+        14653,
+        12592,
+        1054,
+        1051,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066,
+        1067,
+        1068,
+        1069,
+        1070,
+        1071,
+        1072,
+        1073,
+        1074,
+        1075,
+        1076,
+        1077,
+        1078,
+        1079,
+        1080,
+        1081,
+        1082,
+        1083,
+        1084,
+        1085,
+        1086,
+        1087,
+        1088,
+        1089,
+        1090,
+        14229,
+        8062,
+        5193,
+        5516,
+        32484,
+        12763,
+        6970,
+        91939,
+        33613,
+        26039,
+        4286,
+        6758,
+        5010,
+        106550,
+        6292,
+        3905,
+        6003,
+        3313,
+        117778,
+        29970,
+        13339,
+        25698,
+        75317,
+        10574,
+        28235,
+        6535,
+        18704,
+        34817,
+        71042,
+        87478,
+        17782,
+        119492,
+        30631,
+        20195,
+        90805,
+        18412,
+        22133,
+        35846
+      ]
+    },
+    {
+      "id": "text_parts",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        10107,
+        112283,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        3316,
+        10322,
+        10910,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        5876,
+        8011,
+        14653,
+        12592,
+        16860,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        11339,
+        8011,
+        14653,
+        12592,
+        4753,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066
+      ]
+    },
+    {
+      "id": "noul_defaults",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        19227,
+        1122,
+        2811,
+        1050,
+        4225,
+        1097,
+        90610,
+        1098,
+        2811,
+        5876,
+        2821,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        16753,
+        1122,
+        12592,
+        5356,
+        1554,
+        2295,
+        1063,
+        8011,
+        1097,
+        12592,
+        31106,
+        1562,
+        3468,
+        2613,
+        2092,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        5876,
+        8011,
+        14653,
+        12592,
+        16860,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        11339,
+        8011,
+        14653,
+        12592,
+        4753,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066
+      ]
+    },
+    {
+      "id": "score10",
+      "tokens": [
+        10,
+        25708,
+        1010,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1784,
+        6255,
+        1395,
+        7662,
+        1046,
+        11,
+        1010,
+        10,
+        3263,
+        1010,
+        1069,
+        43807,
+        1278,
+        30869,
+        18106,
+        1505,
+        3468,
+        2505,
+        1278,
+        4098,
+        5956,
+        1046,
+        35280,
+        15776,
+        1294,
+        1278,
+        3468,
+        1435,
+        4318,
+        1317,
+        15013,
+        1046,
+        64561,
+        10693,
+        1925,
+        7091,
+        1321,
+        4832,
+        1454,
+        2342,
+        2246,
+        6765,
+        1338,
+        25229,
+        1058,
+        3075,
+        7662,
+        2634,
+        10140,
+        1877,
+        1065,
+        1058,
+        16753,
+        6827,
+        12592,
+        1048,
+        8011,
+        14653,
+        12592,
+        1048,
+        36745,
+        1066,
+        1058,
+        16753,
+        6827,
+        12592,
+        1049,
+        8011,
+        14653,
+        12592,
+        1049,
+        36745,
+        1067,
+        1058,
+        16753,
+        6827,
+        12592,
+        1050,
+        8011,
+        14653,
+        12592,
+        1050,
+        36745,
+        1068,
+        1058,
+        16753,
+        6827,
+        12592,
+        1051,
+        8011,
+        14653,
+        12592,
+        1051,
+        36745,
+        1069,
+        1058,
+        16753,
+        6827,
+        12592,
+        1052,
+        8011,
+        14653,
+        12592,
+        1052,
+        36745,
+        1070,
+        1058,
+        16753,
+        6827,
+        12592,
+        1053,
+        8011,
+        14653,
+        12592,
+        1053,
+        36745,
+        1071,
+        1058,
+        16753,
+        6827,
+        12592,
+        1054,
+        8011,
+        14653,
+        12592,
+        1054,
+        36745,
+        1072,
+        1058,
+        16753,
+        6827,
+        12592,
+        1055,
+        8011,
+        14653,
+        12592,
+        1055,
+        36745,
+        1073,
+        1058,
+        16753,
+        6827,
+        12592,
+        1056,
+        8011,
+        14653,
+        12592,
+        1056,
+        36745,
+        1074,
+        1058,
+        16753,
+        6827,
+        12592,
+        1057,
+        8011,
+        14653,
+        12592,
+        1057,
+        46005,
+        11,
+        1010,
+        10,
+        1503,
+        19464,
+        1010,
+        12,
+        13,
+        31106,
+        1877
+      ],
+      "labels": [
+        1065,
+        1066,
+        1067,
+        1068,
+        1069,
+        1070,
+        1071,
+        1072,
+        1073,
+        1074
+      ]
+    }
+  ]
+}

gguf/provenance.json

@@ -0,0 +1,55 @@
+{
+  "source_repo": "rdaum/eider-dory-1.0-4b-decisions-bf16",
+  "source_weights_revision": "8f75c6c706c8e487793688d7b81833c9032e188d",
+  "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+  "source_weight_sha256": {
+    "model-00001-of-00008.safetensors": "f8ae89948adc35f42b2d735abc460bf20df89fe27063009e7023af236421943e",
+    "model-00002-of-00008.safetensors": "09c16bdfe0f5fb160048a35f08ca998fa1e704efdcb4b3d504e46b2870c68e2d",
+    "model-00003-of-00008.safetensors": "978f4d0f66dce924a81ee2e57aaaa4d0431bf5a163eb3445f32414637846b74c",
+    "model-00004-of-00008.safetensors": "056ce5b367b2d955ce08ec5795d12ba6475640df7175c9f3eb7d5d35e18ddd86",
+    "model-00005-of-00008.safetensors": "dbc45d99c137991456115cd9b71e8fe69dcb7d5b923cee93835e34646af9da0d",
+    "model-00006-of-00008.safetensors": "bd559f8be05597b9915e91c8bd449d7c5194a2205c5a02e4ec750971a3f2814e",
+    "model-00007-of-00008.safetensors": "6c1ebd7492594617d7e0be46cce8b05d57f27930dac431bff1c23a3d496782a3",
+    "model-00008-of-00008.safetensors": "09dde4e45f4cc217f00222f3c57d8b8656d12a8f215e09d300a6f98d434fe80e",
+    "adapter/adapter.safetensors": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d"
+  },
+  "llama_dory_pr": 30306,
+  "llama_revision": "3834b4ae1ee6f63b8bf314a07dfb9c45407975b9",
+  "decision_format": "eider-decision-v1",
+  "loops": 2,
+  "recurrent_kv_cache_mode": "per_loop",
+  "calibration": null,
+  "verified": {
+    "native_prompt_token_and_label_parity": 8,
+    "native_prompt_evidence": "prompt-parity.json",
+    "unchanged_weight_tensors_per_file": 507,
+    "gpu": "NVIDIA GB10",
+    "gpu_offloaded_layers": "73/73",
+    "gpu_smoke_types": [
+      "choice",
+      "noul",
+      "score"
+    ],
+    "gpu_smoke_requests": 24,
+    "gpu_smoke_evidence": "gpu-smoke.json",
+    "generated_output_tokens": 0,
+    "alternate_answer_and_boundary_evidence": "gpu-boundaries.json"
+  },
+  "limitations": [
+    "Dory architecture support is still an unmerged llama.cpp PR",
+    "No full accuracy evaluation for GGUF",
+    "The llama.cpp decision convention accepts at most 10 score levels",
+    "Tool-call chat must be serialized as state text",
+    "Image and audio inputs are unsupported",
+    "llama.cpp reports template validation errors as HTTP 500"
+  ],
+  "files": {
+    "dory-decisions-Q8_0.gguf": {
+      "bytes": 4232655616,
+      "sha256": "bb36b8ed263ce633649ccfad9c6e1dc065e9c3d333624fc27028f496089a38b8"
+    }
+  },
+  "llama_decision_type": "granite-switch",
+  "llama_local_source_changes": false,
+  "llama_revision_role": "Tested source revision, not a required version pin."
+}

model-00001-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8ae89948adc35f42b2d735abc460bf20df89fe27063009e7023af236421943e
+size 1064423904

model-00002-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09c16bdfe0f5fb160048a35f08ca998fa1e704efdcb4b3d504e46b2870c68e2d
+size 1038413752

model-00003-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:978f4d0f66dce924a81ee2e57aaaa4d0431bf5a163eb3445f32414637846b74c
+size 1064623416

model-00004-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056ce5b367b2d955ce08ec5795d12ba6475640df7175c9f3eb7d5d35e18ddd86
+size 1064650776

model-00005-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbc45d99c137991456115cd9b71e8fe69dcb7d5b923cee93835e34646af9da0d
+size 1054137760

model-00006-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd559f8be05597b9915e91c8bd449d7c5194a2205c5a02e4ec750971a3f2814e
+size 1038413752

model-00007-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c1ebd7492594617d7e0be46cce8b05d57f27930dac431bff1c23a3d496782a3
+size 954762648

model-00008-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09dde4e45f4cc217f00222f3c57d8b8656d12a8f215e09d300a6f98d434fe80e
+size 671088768

model.safetensors.index.json

@@ -0,0 +1,514 @@
+{
+  "metadata": {
+    "total_size": 7950458880
+  },
+  "weight_map": {
+    "backbone.embeddings.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.attn_alpha": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.g_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.k_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.o_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.q_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.s_gate": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.sqk": "model-00001-of-00008.safetensors",
+    "backbone.layers.0.mixer.v_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.1.mixer.down_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.1.mixer.gate_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.1.mixer.suv_gate": "model-00001-of-00008.safetensors",
+    "backbone.layers.1.mixer.suv_up": "model-00001-of-00008.safetensors",
+    "backbone.layers.1.mixer.up_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.1.mlp_alpha": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.attn_alpha": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.g_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.k_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.o_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.q_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.s_gate": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.sqk": "model-00001-of-00008.safetensors",
+    "backbone.layers.10.mixer.v_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.11.mixer.down_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.11.mixer.gate_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.11.mixer.suv_gate": "model-00001-of-00008.safetensors",
+    "backbone.layers.11.mixer.suv_up": "model-00001-of-00008.safetensors",
+    "backbone.layers.11.mixer.up_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.11.mlp_alpha": "model-00001-of-00008.safetensors",
+    "backbone.layers.12.attn_alpha": "model-00001-of-00008.safetensors",
+    "backbone.layers.12.mixer.g_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.12.mixer.k_proj.weight": "model-00001-of-00008.safetensors",
+    "backbone.layers.12.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.12.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.12.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.12.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.12.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.13.mixer.down_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.13.mixer.gate_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.13.mixer.suv_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.13.mixer.suv_up": "model-00002-of-00008.safetensors",
+    "backbone.layers.13.mixer.up_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.13.mlp_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.attn_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.g_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.k_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.14.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.15.mixer.down_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.15.mixer.gate_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.15.mixer.suv_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.15.mixer.suv_up": "model-00002-of-00008.safetensors",
+    "backbone.layers.15.mixer.up_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.15.mlp_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.attn_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.g_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.k_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.16.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.17.mixer.down_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.17.mixer.gate_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.17.mixer.suv_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.17.mixer.suv_up": "model-00002-of-00008.safetensors",
+    "backbone.layers.17.mixer.up_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.17.mlp_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.attn_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.g_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.k_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.18.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.19.mixer.down_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.19.mixer.gate_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.19.mixer.suv_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.19.mixer.suv_up": "model-00002-of-00008.safetensors",
+    "backbone.layers.19.mixer.up_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.19.mlp_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.attn_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.g_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.k_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.2.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.attn_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.g_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.k_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.20.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.21.mixer.down_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.21.mixer.gate_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.21.mixer.suv_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.21.mixer.suv_up": "model-00002-of-00008.safetensors",
+    "backbone.layers.21.mixer.up_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.21.mlp_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.attn_alpha": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.g_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.k_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.o_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.q_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.s_gate": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.sqk": "model-00002-of-00008.safetensors",
+    "backbone.layers.22.mixer.v_proj.weight": "model-00002-of-00008.safetensors",
+    "backbone.layers.23.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.23.mixer.gate_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.23.mixer.suv_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.23.mixer.suv_up": "model-00003-of-00008.safetensors",
+    "backbone.layers.23.mixer.up_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.23.mlp_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.attn_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.g_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.k_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.o_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.q_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.s_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.sqk": "model-00003-of-00008.safetensors",
+    "backbone.layers.24.mixer.v_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.25.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.25.mixer.gate_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.25.mixer.suv_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.25.mixer.suv_up": "model-00003-of-00008.safetensors",
+    "backbone.layers.25.mixer.up_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.25.mlp_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.attn_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.g_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.k_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.o_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.q_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.s_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.sqk": "model-00003-of-00008.safetensors",
+    "backbone.layers.26.mixer.v_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.27.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.27.mixer.gate_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.27.mixer.suv_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.27.mixer.suv_up": "model-00003-of-00008.safetensors",
+    "backbone.layers.27.mixer.up_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.27.mlp_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.attn_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.g_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.k_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.o_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.q_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.s_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.sqk": "model-00003-of-00008.safetensors",
+    "backbone.layers.28.mixer.v_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.29.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.29.mixer.gate_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.29.mixer.suv_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.29.mixer.suv_up": "model-00003-of-00008.safetensors",
+    "backbone.layers.29.mixer.up_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.29.mlp_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.3.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.3.mixer.gate_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.3.mixer.suv_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.3.mixer.suv_up": "model-00003-of-00008.safetensors",
+    "backbone.layers.3.mixer.up_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.3.mlp_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.attn_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.g_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.k_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.o_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.q_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.s_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.sqk": "model-00003-of-00008.safetensors",
+    "backbone.layers.30.mixer.v_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.31.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.31.mixer.gate_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.31.mixer.suv_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.31.mixer.suv_up": "model-00003-of-00008.safetensors",
+    "backbone.layers.31.mixer.up_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.31.mlp_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.attn_alpha": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.g_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.k_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.o_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.q_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.s_gate": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.sqk": "model-00003-of-00008.safetensors",
+    "backbone.layers.32.mixer.v_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.33.mixer.down_proj.weight": "model-00003-of-00008.safetensors",
+    "backbone.layers.33.mixer.gate_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.33.mixer.suv_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.33.mixer.suv_up": "model-00004-of-00008.safetensors",
+    "backbone.layers.33.mixer.up_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.33.mlp_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.g_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.k_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.o_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.q_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.s_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.sqk": "model-00004-of-00008.safetensors",
+    "backbone.layers.34.mixer.v_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.35.mixer.down_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.35.mixer.gate_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.35.mixer.suv_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.35.mixer.suv_up": "model-00004-of-00008.safetensors",
+    "backbone.layers.35.mixer.up_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.35.mlp_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.g_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.k_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.o_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.q_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.s_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.sqk": "model-00004-of-00008.safetensors",
+    "backbone.layers.36.mixer.v_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.37.mixer.down_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.37.mixer.gate_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.37.mixer.suv_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.37.mixer.suv_up": "model-00004-of-00008.safetensors",
+    "backbone.layers.37.mixer.up_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.37.mlp_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.g_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.k_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.o_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.q_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.s_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.sqk": "model-00004-of-00008.safetensors",
+    "backbone.layers.38.mixer.v_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.39.mixer.down_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.39.mixer.gate_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.39.mixer.suv_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.39.mixer.suv_up": "model-00004-of-00008.safetensors",
+    "backbone.layers.39.mixer.up_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.39.mlp_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.g_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.k_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.o_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.q_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.s_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.sqk": "model-00004-of-00008.safetensors",
+    "backbone.layers.4.mixer.v_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.g_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.k_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.o_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.q_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.s_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.sqk": "model-00004-of-00008.safetensors",
+    "backbone.layers.40.mixer.v_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.41.mixer.down_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.41.mixer.gate_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.41.mixer.suv_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.41.mixer.suv_up": "model-00004-of-00008.safetensors",
+    "backbone.layers.41.mixer.up_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.41.mlp_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.g_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.k_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.o_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.q_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.s_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.sqk": "model-00004-of-00008.safetensors",
+    "backbone.layers.42.mixer.v_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.43.mixer.down_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.43.mixer.gate_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.43.mixer.suv_gate": "model-00004-of-00008.safetensors",
+    "backbone.layers.43.mixer.suv_up": "model-00004-of-00008.safetensors",
+    "backbone.layers.43.mixer.up_proj.weight": "model-00004-of-00008.safetensors",
+    "backbone.layers.43.mlp_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.44.attn_alpha": "model-00004-of-00008.safetensors",
+    "backbone.layers.44.mixer.g_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.44.mixer.k_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.44.mixer.o_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.44.mixer.q_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.44.mixer.s_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.44.mixer.sqk": "model-00005-of-00008.safetensors",
+    "backbone.layers.44.mixer.v_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.45.mixer.down_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.45.mixer.gate_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.45.mixer.suv_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.45.mixer.suv_up": "model-00005-of-00008.safetensors",
+    "backbone.layers.45.mixer.up_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.45.mlp_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.attn_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.g_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.k_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.o_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.q_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.s_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.sqk": "model-00005-of-00008.safetensors",
+    "backbone.layers.46.mixer.v_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.47.mixer.down_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.47.mixer.gate_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.47.mixer.suv_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.47.mixer.suv_up": "model-00005-of-00008.safetensors",
+    "backbone.layers.47.mixer.up_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.47.mlp_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.attn_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.g_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.k_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.o_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.q_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.s_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.sqk": "model-00005-of-00008.safetensors",
+    "backbone.layers.48.mixer.v_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.49.mixer.down_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.49.mixer.gate_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.49.mixer.suv_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.49.mixer.suv_up": "model-00005-of-00008.safetensors",
+    "backbone.layers.49.mixer.up_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.49.mlp_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.5.mixer.down_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.5.mixer.gate_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.5.mixer.suv_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.5.mixer.suv_up": "model-00005-of-00008.safetensors",
+    "backbone.layers.5.mixer.up_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.5.mlp_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.attn_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.g_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.k_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.o_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.q_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.s_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.sqk": "model-00005-of-00008.safetensors",
+    "backbone.layers.50.mixer.v_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.51.mixer.down_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.51.mixer.gate_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.51.mixer.suv_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.51.mixer.suv_up": "model-00005-of-00008.safetensors",
+    "backbone.layers.51.mixer.up_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.51.mlp_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.attn_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.g_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.k_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.o_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.q_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.s_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.sqk": "model-00005-of-00008.safetensors",
+    "backbone.layers.52.mixer.v_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.53.mixer.down_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.53.mixer.gate_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.53.mixer.suv_gate": "model-00005-of-00008.safetensors",
+    "backbone.layers.53.mixer.suv_up": "model-00005-of-00008.safetensors",
+    "backbone.layers.53.mixer.up_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.53.mlp_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.54.attn_alpha": "model-00005-of-00008.safetensors",
+    "backbone.layers.54.mixer.g_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.54.mixer.k_proj.weight": "model-00005-of-00008.safetensors",
+    "backbone.layers.54.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.54.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.54.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.54.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.54.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.55.mixer.down_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.55.mixer.gate_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.55.mixer.suv_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.55.mixer.suv_up": "model-00006-of-00008.safetensors",
+    "backbone.layers.55.mixer.up_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.55.mlp_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.attn_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.g_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.k_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.56.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.57.mixer.down_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.57.mixer.gate_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.57.mixer.suv_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.57.mixer.suv_up": "model-00006-of-00008.safetensors",
+    "backbone.layers.57.mixer.up_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.57.mlp_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.attn_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.g_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.k_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.58.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.59.mixer.down_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.59.mixer.gate_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.59.mixer.suv_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.59.mixer.suv_up": "model-00006-of-00008.safetensors",
+    "backbone.layers.59.mixer.up_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.59.mlp_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.attn_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.g_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.k_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.6.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.attn_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.g_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.k_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.60.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.61.mixer.down_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.61.mixer.gate_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.61.mixer.suv_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.61.mixer.suv_up": "model-00006-of-00008.safetensors",
+    "backbone.layers.61.mixer.up_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.61.mlp_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.attn_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.g_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.k_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.62.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.63.mixer.down_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.63.mixer.gate_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.63.mixer.suv_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.63.mixer.suv_up": "model-00006-of-00008.safetensors",
+    "backbone.layers.63.mixer.up_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.63.mlp_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.attn_alpha": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.g_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.k_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.o_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.q_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.s_gate": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.sqk": "model-00006-of-00008.safetensors",
+    "backbone.layers.64.mixer.v_proj.weight": "model-00006-of-00008.safetensors",
+    "backbone.layers.65.mixer.down_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.65.mixer.gate_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.65.mixer.suv_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.65.mixer.suv_up": "model-00007-of-00008.safetensors",
+    "backbone.layers.65.mixer.up_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.65.mlp_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.attn_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.g_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.k_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.o_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.q_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.s_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.sqk": "model-00007-of-00008.safetensors",
+    "backbone.layers.66.mixer.v_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.67.mixer.down_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.67.mixer.gate_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.67.mixer.suv_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.67.mixer.suv_up": "model-00007-of-00008.safetensors",
+    "backbone.layers.67.mixer.up_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.67.mlp_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.attn_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.g_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.k_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.o_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.q_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.s_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.sqk": "model-00007-of-00008.safetensors",
+    "backbone.layers.68.mixer.v_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.69.mixer.down_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.69.mixer.gate_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.69.mixer.suv_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.69.mixer.suv_up": "model-00007-of-00008.safetensors",
+    "backbone.layers.69.mixer.up_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.69.mlp_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.7.mixer.down_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.7.mixer.gate_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.7.mixer.suv_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.7.mixer.suv_up": "model-00007-of-00008.safetensors",
+    "backbone.layers.7.mixer.up_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.7.mlp_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.attn_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.g_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.k_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.o_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.q_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.s_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.sqk": "model-00007-of-00008.safetensors",
+    "backbone.layers.70.mixer.v_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.71.mixer.down_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.71.mixer.gate_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.71.mixer.suv_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.71.mixer.suv_up": "model-00007-of-00008.safetensors",
+    "backbone.layers.71.mixer.up_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.71.mlp_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.attn_alpha": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.g_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.k_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.o_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.q_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.s_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.sqk": "model-00007-of-00008.safetensors",
+    "backbone.layers.8.mixer.v_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.9.mixer.down_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.9.mixer.gate_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.9.mixer.suv_gate": "model-00007-of-00008.safetensors",
+    "backbone.layers.9.mixer.suv_up": "model-00007-of-00008.safetensors",
+    "backbone.layers.9.mixer.up_proj.weight": "model-00007-of-00008.safetensors",
+    "backbone.layers.9.mlp_alpha": "model-00007-of-00008.safetensors",
+    "lm_head.logit_scale": "model-00007-of-00008.safetensors",
+    "lm_head.weight": "model-00008-of-00008.safetensors"
+  }
+}

nvfp4/LICENSE

@@ -0,0 +1,49 @@
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

nvfp4/NOTICE

@@ -0,0 +1,5 @@
+Derived from nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16, revision b366b87f762ea334ba3eb55912031baba1515810.
+Original model: NVIDIA. Base model licence: OpenMDW-1.1.
+Modifications: typed-decision LoRA fine-tuning, action selection, state tracking, planning, human-judgement training, BF16 merging, and NVFP4 quantization.
+Training source attribution, pinned versions, and recorded source licences appear in README.md and training-data.json.
+SCONE attribution: Long, Pasupat and Liang (2016). HelpSteer2 attribution: NVIDIA and Scale AI.

nvfp4/adapter/README.md

@@ -0,0 +1,15 @@
+# Eider Dory adapter
+
+This adapter uses Eider tensor names. It is not a PEFT adapter.
+The complete model weights already include these changes. Do not apply the adapter twice.
+
+To load the adapter with the supplied Python module, use the original BF16 snapshot:
+
+```python
+from pathlib import Path
+from dory_lora import Dory
+
+model = Dory.load(Path("/path/to/original-bf16"), rank=8, alpha=16, loops=2)
+model.load_adapter(Path("adapter.safetensors"))
+model.eval()
+```

nvfp4/adapter/adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d
+size 64941920

nvfp4/adapter/dory_lora.py

@@ -0,0 +1,274 @@
+"""BF16 Dory with frozen checkpoint weights and trainable linear LoRA adapters.
+
+The nGPT equations follow the independent dory-checkpoint-reference.py oracle
+and vLLM revision 55076950b6c155f384d6766d63563c3bc20a0e3f. This is a training
+experiment, not a replacement for Eider's CUDA serving implementation.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+from pathlib import Path
+
+import torch
+from safetensors import safe_open
+from safetensors.torch import load_file, save_file
+from torch import nn
+from torch.nn import functional as F
+from torch.utils.checkpoint import checkpoint
+
+
+def norm(x):
+    value = x.float()
+    return (value * torch.rsqrt(value.square().sum(-1, keepdim=True) + 1e-6)).to(
+        x.dtype
+    )
+
+
+class Linear(nn.Module):
+    def __init__(self, weight, rank, alpha):
+        super().__init__()
+        self.register_buffer("weight", weight)
+        self.scale = alpha / rank if rank else 0
+        if rank:
+            self.lora_a = nn.Parameter(
+                torch.empty(rank, weight.shape[1], device=weight.device)
+            )
+            self.lora_b = nn.Parameter(
+                torch.zeros(weight.shape[0], rank, device=weight.device)
+            )
+            nn.init.kaiming_uniform_(self.lora_a, a=math.sqrt(5))
+        else:
+            self.register_parameter("lora_a", None)
+            self.register_parameter("lora_b", None)
+
+    def forward(self, x):
+        base = F.linear(x, self.weight)
+        if self.lora_a is None:
+            return base
+        delta = F.linear(F.linear(x, self.lora_a.to(x.dtype)), self.lora_b.to(x.dtype))
+        return (base.float() + self.scale * delta.float()).to(x.dtype)
+
+
+class Layer(nn.Module):
+    def __init__(self, config, index, read, rank, alpha):
+        super().__init__()
+        self.config = config
+        self.kind = config["layer_types"][index]
+        prefix = f"backbone.layers.{index}"
+        mixer = prefix + ".mixer"
+        std = config["initializer_range"]
+        if self.kind == "mlp":
+            names = ("gate_proj", "up_proj", "down_proj")
+            scale = math.sqrt(config["hidden_size"])
+            self.register_buffer("suv_gate", read(mixer + ".suv_gate").float() * scale)
+            self.register_buffer("suv_up", read(mixer + ".suv_up").float() * scale)
+            rate = read(prefix + ".mlp_alpha")
+        else:
+            names = ("q_proj", "k_proj", "v_proj", "g_proj", "o_proj")
+            dim = config["head_dim"]
+            self.register_buffer(
+                "sqk", (read(mixer + ".sqk").float() * (math.sqrt(dim) / std)).square()
+            )
+            g0 = config.get("attention_output_gate_logit_scale_init") or math.sqrt(
+                config["hidden_size"]
+            )
+            self.register_buffer(
+                "s_gate",
+                (read(mixer + ".s_gate").float() * (math.sqrt(g0) / std)).square(),
+            )
+            rate = read(prefix + ".attn_alpha")
+        self.register_buffer(
+            "rate", (rate.float() * config["ngpt_alpha_init"] / std).abs()
+        )
+        self.projections = nn.ModuleDict(
+            {
+                name: Linear(read(mixer + "." + name + ".weight"), rank, alpha)
+                for name in names
+            }
+        )
+
+    def forward(self, h, rotary, sliding_mask, reference=False):
+        p, c, dtype = self.projections, self.config, h.dtype
+        if self.kind == "mlp":
+            gate = (p["gate_proj"](h).float() * self.suv_gate).to(dtype)
+            up = (p["up_proj"](h).float() * self.suv_up).to(dtype)
+            out = p["down_proj"](norm((F.silu(gate.float()) * up.float()).to(dtype)))
+        else:
+            nq, nk, dim = (
+                c["num_attention_heads"],
+                c["num_key_value_heads"],
+                c["head_dim"],
+            )
+            cos, sin = rotary[self.kind]
+
+            def rope(x):
+                a, b = x.float().chunk(2, -1)
+                return torch.cat((a * cos - b * sin, b * cos + a * sin), -1).to(dtype)
+
+            q = norm(rope(p["q_proj"](h).reshape(*h.shape[:-1], nq, dim)))
+            k = norm(rope(p["k_proj"](h).reshape(*h.shape[:-1], nk, dim)))
+            q = (q.float() * self.sqk.reshape(nq, dim)).to(dtype)
+            v = p["v_proj"](h).reshape(*h.shape[:-1], nk, dim)
+            # Repeat GQA heads explicitly so both SDPA backends and the oracle
+            # use the same layout. Batch rows retain independent attention.
+            q = q.transpose(1, 2)
+            k = k.repeat_interleave(nq // nk, 2).transpose(1, 2)
+            v = v.repeat_interleave(nq // nk, 2).transpose(1, 2)
+            mask = sliding_mask if self.kind == "sliding_attention" else None
+            if reference:
+                if mask is None:
+                    mask = torch.ones(
+                        h.shape[1], h.shape[1], device=h.device, dtype=torch.bool
+                    ).tril()
+                scores = (q.float() @ k.float().transpose(-1, -2)) / math.sqrt(dim)
+                out = (
+                    scores.masked_fill(~mask, -torch.inf).softmax(-1) @ v.float()
+                ).to(dtype)
+            else:
+                out = F.scaled_dot_product_attention(
+                    q, k, v, attn_mask=mask, is_causal=mask is None
+                )
+            out = out.transpose(1, 2).reshape(*h.shape[:-1], nq * dim)
+            gate = p["g_proj"](h)
+            out = norm(
+                (out.float() * torch.sigmoid(gate.float() * self.s_gate)).to(dtype)
+            )
+            out = p["o_proj"](out)
+        a, b = norm(h).float(), norm(out).float()
+        return norm(a + self.rate * (b - a)).to(dtype)
+
+
+class Dory(nn.Module):
+    def __init__(self, config, read, *, rank=8, alpha=16, loops=2, checkpointing=False):
+        super().__init__()
+        if config.get("model_type") != "dory" or config.get("quantization_config"):
+            raise ValueError("requires an unquantized Dory checkpoint")
+        if loops < 1 or rank < 0 or alpha <= 0:
+            raise ValueError("invalid loop count or adapter dimensions")
+        self.config, self.loops, self.checkpointing = config, loops, checkpointing
+        self.register_buffer("embedding", read("backbone.embeddings.weight"))
+        self.register_buffer("head", read("lm_head.weight"))
+        self.register_buffer(
+            "head_scale",
+            read("lm_head.logit_scale").float() / config["initializer_range"],
+        )
+        self.layers = nn.ModuleList(
+            Layer(config, i, read, rank, alpha)
+            for i in range(config["num_hidden_layers"])
+        )
+
+    @classmethod
+    def load(cls, path: Path, *, device="cuda", **kwargs):
+        config = json.loads((path / "config.json").read_text())
+        with safe_open(
+            path / "model.safetensors", framework="pt", device="cpu"
+        ) as weights:
+            consumed = set()
+
+            def read(name):
+                value = weights.get_tensor(name)
+                if value.dtype != torch.bfloat16:
+                    raise ValueError(f"expected BF16 weight: {name}")
+                consumed.add(name)
+                return value.to(device)
+
+            model = cls(config, read, **kwargs)
+            if consumed != set(weights.keys()):
+                raise ValueError(
+                    f"unconsumed checkpoint tensors: {set(weights.keys()) - consumed}"
+                )
+        return model
+
+    def forward(self, ids, labels=None, *, lengths=None, reference=False):
+        c = self.config
+        single = ids.ndim == 1
+        if single:
+            ids = ids.unsqueeze(0)
+        positions = torch.arange(ids.shape[1], device=ids.device)
+        dim = c["head_dim"]
+        rotary = {}
+        for kind, params in c["rope_parameters"].items():
+            angles = positions.float()[:, None] * params["rope_theta"] ** (
+                -torch.arange(dim // 2, device=ids.device).float() / (dim // 2)
+            )
+            rotary[kind] = (angles.cos()[:, None, :], angles.sin()[:, None, :])
+        # For short training rows the sliding mask equals the causal mask.
+        sliding_mask = None
+        if ids.shape[1] > c["sliding_window"]:
+            distance = positions[:, None] - positions[None, :]
+            sliding_mask = (distance >= 0) & (distance < c["sliding_window"])
+
+        def apply(h, start, end):
+            for layer in self.layers[start:end]:
+                if self.training and self.checkpointing and not reference:
+                    h = checkpoint(layer, h, rotary, sliding_mask, use_reentrant=False)
+                else:
+                    # Keep the serial evaluator and oracle eager. Their batch-one,
+                    # inference-only graphs otherwise compete with training graphs.
+                    forward = (
+                        layer if self.training and not reference else layer.forward
+                    )
+                    h = forward(h, rotary, sliding_mask, reference)
+            return h
+
+        ni, nr = c["n_input_layers"], c["n_recurrent_layers"]
+        h = apply(F.embedding(ids, self.embedding), 0, ni)
+        input_h = h
+        for _ in range(self.loops):
+            h = norm(h + input_h)
+            h = norm(apply(h, ni, ni + nr))
+        h = apply(h, ni + nr, len(self.layers))
+        # Right-padding is always after each real query, so causal attention
+        # excludes it without a padding mask. Read only each row's final token.
+        last = (
+            h[:, -1]
+            if lengths is None
+            else h[torch.arange(ids.shape[0], device=ids.device), lengths - 1]
+        )
+        weight = self.head if labels is None else self.head[labels]
+        scale = self.head_scale if labels is None else self.head_scale[labels]
+        logits = (
+            torch.bmm(weight, last.unsqueeze(-1)).squeeze(-1)
+            if labels is not None and labels.ndim == 2
+            else F.linear(last, weight)
+        ).float() * scale
+        return logits.squeeze(0) if single else logits
+
+    def compile_layers(self):
+        # Compile individual layers so recurrent traversals reuse their graphs.
+        # Keep the BF16 rounding points used by the eager nGPT equations.
+        # All layer kinds share Layer.forward's code cache. Attention masks,
+        # input gradients and shape guards produce more than eight variants when
+        # training batches cross the sliding-window boundary. Keep a bounded
+        # cache large enough for those variants; fullgraph raises on overflow.
+        torch._dynamo.config.recompile_limit = max(
+            torch._dynamo.config.recompile_limit, 32
+        )
+        for layer in self.layers:
+            layer.compile(
+                fullgraph=True,
+                dynamic=True,
+                options={"emulate_precision_casts": True},
+            )
+
+    def save_adapter(self, path):
+        save_file(
+            {
+                name: p.detach().cpu().contiguous()
+                for name, p in self.named_parameters()
+            },
+            str(path),
+        )
+
+    def load_adapter(self, path):
+        values = load_file(str(path))
+        params = dict(self.named_parameters())
+        if values.keys() != params.keys():
+            raise ValueError("adapter does not match the model's projection modules")
+        with torch.no_grad():
+            for name, p in params.items():
+                if p.shape != values[name].shape:
+                    raise ValueError(f"adapter shape mismatch: {name}")
+                p.copy_(values[name])

nvfp4/adapter/eider_adapter_config.json

@@ -0,0 +1,12 @@
+{
+  "format": "eider.dory.lora/v1",
+  "base_model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16",
+  "base_revision": "b366b87f762ea334ba3eb55912031baba1515810",
+  "rank": 8,
+  "alpha": 16.0,
+  "loops": 2,
+  "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+  "tensor_names": "layers.{i}.projections.{name}.lora_a|lora_b",
+  "merge": "BF16(W + (alpha/rank) * F32(BF16(B)) @ F32(BF16(A)))",
+  "peft_compatible": false
+}

nvfp4/chat_template.jinja

@@ -0,0 +1,195 @@
+{% macro render_extra_keys(json_dict, handled_keys) %}
+    {%- if json_dict is mapping %}
+        {%- for json_key in json_dict if json_key not in handled_keys %}
+            {%- if json_dict[json_key] is mapping or (json_dict[json_key] is sequence and json_dict[json_key] is not string) %}
+                {{- '\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | tojson | safe) ~ '</' ~ json_key ~ '>' }}
+            {%- else %}
+                {{-'\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | string) ~ '</' ~ json_key ~ '>' }}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+{% endmacro %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else True %}
+{%- set medium_effort = medium_effort if medium_effort is defined else False %}
+{%- set truncate_history_thinking = truncate_history_thinking if truncate_history_thinking is defined else True %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- set loop_messages = messages %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = [] %}
+{%- endif %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if system_message is defined %}
+    {{- "<|im_start|>system\n" + system_message }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- "<|im_start|>system\n" }}
+    {%- endif %}
+{%- endif %}
+{%- if tools is iterable and tools | length > 0 %}
+    {%- if system_message is defined and system_message | length > 0 %}
+        {{- "\n\n" }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n" }}
+    {{- "<tools>" }}
+    {%- for tool in tools %}
+        {%- if tool.function is defined %}
+            {%- set tool = tool.function %}
+        {%- endif %}
+        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
+        {%- if tool.description is defined %}
+            {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
+        {%- endif %}
+        {{- '\n<parameters>' }}
+        {%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
+            {%- for param_name, param_fields in tool.parameters.properties|items %}
+                {{- '\n<parameter>' }}
+                {{- '\n<name>' ~ param_name ~ '</name>' }}
+                {%- if param_fields.type is defined %}
+                    {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
+                {%- endif %}
+                {%- if param_fields.description is defined %}
+                    {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
+                {%- endif %}
+                {%- if param_fields.enum is defined %}
+                    {{- '\n<enum>' ~ (param_fields.enum | tojson | safe) ~ '</enum>' }}
+                {%- endif %}
+                {%- set handled_keys = ['name', 'type', 'description', 'enum'] %}
+                {{- render_extra_keys(param_fields, handled_keys) }}
+                {{- '\n</parameter>' }}
+            {%- endfor %}
+        {%- endif %}
+        {% set handled_keys = ['type', 'properties', 'required'] %}
+        {{- render_extra_keys(tool.parameters, handled_keys) }}
+        {%- if tool.parameters is defined and tool.parameters.required is defined %}
+            {{- '\n<required>' ~ (tool.parameters.required | tojson | safe) ~ '</required>' }}
+        {%- endif %}
+        {{- '\n</parameters>' }}
+        {%- set handled_keys = ['type', 'name', 'description', 'parameters'] %}
+        {{- render_extra_keys(tool, handled_keys) }}
+        {{- '\n</function>' }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+{%- endif %}
+{%- if system_message is defined %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in loop_messages %}
+    {%- if message.role == "assistant" %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
+            {%- set content = "<think>\n" ~ message.reasoning_content ~ "</think>" ~ (message.content | default('', true)) %}
+        {%- else %}
+            {%- set content = message.content | default('', true) %}
+            {%- if content is string -%}
+                {%- if '<think>' not in content and '</think>' not in content -%}
+                    {%- set content = "<think></think>" ~ content -%}
+                {%- endif -%}
+            {%- else -%}
+                {%- set content = content -%}
+            {%- endif -%}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
+            {{- '<|im_start|>assistant\n' }}
+                {%- set include_content = not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {%- if content is string and content | trim | length > 0 %}
+                    {%- if include_content %}
+                        {{- (content | trim) ~ '\n' -}}
+                    {%- else %}
+                        {%- set c = (content | string) %}
+                        {%- if '</think>' in c %}
+                            {%- set c = c.split('</think>')[-1] %}
+                        {%- elif '<think>' in c %}
+                            {%- set c = c.split('<think>')[0] %}
+                        {%- endif %}
+                        {%- set c = "<think></think>" ~ c %}
+                        {%- if c | length > 0 %}
+                            {{- c ~ '\n' -}}
+                        {%- endif %}
+                    {%- endif %}
+                {%- else %}
+                    {{- "<think></think>" -}}
+                {%- endif %}
+                {%- for tool_call in message.tool_calls %}
+                    {%- if tool_call.function is defined %}
+                        {%- set tool_call = tool_call.function %}
+                    {%- endif %}
+                    {{- '<tool_call>\n<function=' ~ tool_call.name ~ '>\n' -}}
+                        {%- if tool_call.arguments is defined %}
+                            {%- for args_name, args_value in tool_call.arguments|items %}
+                                {{- '<parameter=' ~ args_name ~ '>\n' -}}
+                                    {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                                {{- args_value ~ '\n</parameter>\n' -}}
+                            {%- endfor %}
+                        {%- endif %}
+                    {{- '</function>\n</tool_call>\n' -}}
+                {%- endfor %}
+                {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- if not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {{- '<|im_start|>assistant\n' ~ (content | default('', true) | string | trim) ~ '<|im_end|>\n' }}
+            {%- else %}
+                {%- set c = (content | default('', true) | string) %}
+                {%- if '<think>' in c and '</think>' in c %}
+                    {%- set c = "<think></think>" ~ c.split('</think>')[-1] %}
+                {%- endif %}
+                {%- set c = c | trim %}
+                {%- if c | length > 0 %}
+                    {{- '<|im_start|>assistant\n' ~ c ~ '<|im_end|>\n' }}
+                {%- else %}
+                    {{- '<|im_start|>assistant\n<|im_end|>\n' }}
+                {%- endif %}
+            {%- endif %}
+        {%- endif %}
+    {%- elif message.role == "user" or message.role == "system" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- set content = message.content | string %}
+        {%- if message.role == "user" and loop.index0 == ns.last_user_idx and medium_effort %}
+            {{- content + '\n\n{reasoning effort: efficient}' }}
+        {%- else %}
+            {{- content }}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user\n' }}
+        {%- endif %}
+        {{- '<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>\n' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {%- if enable_thinking %}
+        {{- '<|im_start|>assistant\n<think>\n' }}
+    {%- else %}
+        {{- '<|im_start|>assistant\n<think></think>' }}
+    {%- endif %}
+{%- endif %}

nvfp4/config.json

@@ -0,0 +1,138 @@
+{
+  "architectures": [
+    "DoryForCausalLM"
+  ],
+  "model_type": "dory",
+  "vocab_size": 131072,
+  "hidden_size": 2560,
+  "num_hidden_layers": 72,
+  "intermediate_size": 7168,
+  "num_attention_heads": 16,
+  "num_key_value_heads": 4,
+  "max_position_embeddings": 262144,
+  "tie_word_embeddings": false,
+  "ngpt_alpha_init": 0.05,
+  "attention_output_gate_logit_scale_init": null,
+  "mlp_bias": false,
+  "attention_bias": false,
+  "n_input_layers": 16,
+  "n_recurrent_layers": 40,
+  "n_output_layers": 16,
+  "n_recurrent_loops": 2,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "initializer_range": 0.0177,
+  "dtype": "bfloat16",
+  "sliding_window": 1024,
+  "rope_parameters": {
+    "sliding_attention": {
+      "rope_type": "default",
+      "rope_theta": 10000,
+      "partial_rotary_factor": 1.0
+    },
+    "full_attention": {
+      "rope_type": "default",
+      "rope_theta": 10000000,
+      "partial_rotary_factor": 1.0
+    }
+  },
+  "layer_types": [
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "sliding_attention",
+    "mlp",
+    "full_attention",
+    "mlp"
+  ],
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "weights": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "targets": [
+          "Linear"
+        ]
+      }
+    },
+    "ignore": [
+      "backbone.embeddings",
+      "lm_head"
+    ],
+    "quant_algo": "W4A16_NVFP4",
+    "quant_method": "modelopt",
+    "producer": {
+      "name": "eider-mse",
+      "version": "1"
+    }
+  }
+}

nvfp4/evaluation/native-serving.json

@@ -0,0 +1,38 @@
+{
+  "command": [
+    "eider-serve",
+    "--model-dir",
+    "nvfp4-merged",
+    "--offline",
+    "--artifact-dir",
+    "nvfp4-merged",
+    "--served-model-name",
+    "dory-quant-eval",
+    "--dory-loops",
+    "2",
+    "--listen",
+    "127.0.0.1:18089",
+    "--max-active-sequences",
+    "1",
+    "--decode-capacity",
+    "1",
+    "--prefill-sequence-capacity",
+    "1",
+    "--prefill-token-capacity",
+    "2048",
+    "--decision-branch-capacity",
+    "8",
+    "--max-context-tokens",
+    "4096",
+    "--retained-prefix-gib",
+    "0"
+  ],
+  "binary_sha256": "935335c9df451b4b9108d65fc46882abbfd7a0b6cb20b3ba76f4e8c42fa7ab0c",
+  "script_sha256": "cbd6c833654b1fff319abf1d8f3cb491af5d96b78d541e593a283b684268da04",
+  "evaluation_data_dir": "data",
+  "evaluation_data_sha256": {
+    "new_test": "e2ca7413a306fc2b0e9e044aa685c94d5aa8e0def7cfcec6448941b7369ae625",
+    "test": "ace4bd84b914cee4c9ad8658d934386b045b7c643d490283d4f178f037acdba5",
+    "transfer": "3c5a77ed25fc7d0c2d1419151aa816bab9d12975b1e59de681934c114834d8e7"
+  }
+}

nvfp4/evaluation/nvfp4-comparison.json

@@ -0,0 +1,2033 @@
+{
+  "schema": "eider.dory.state-continuation-evaluation/v1",
+  "selection": {
+    "criterion": "NVFP4 validation Brier, cross-entropy, earliest step",
+    "candidates": [
+      {
+        "step": 0,
+        "examples_seen": 0,
+        "checkpoint": "step-000000",
+        "export": "step-000000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.6005859375,
+          "brier": 0.5027092950491365,
+          "cross_entropy": 1.6357855528374599,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.6030075187969924,
+              "brier": 0.4978527031414729,
+              "cross_entropy": 1.564796901472784
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.7361111111111112,
+              "brier": 0.4178836877295325,
+              "cross_entropy": 0.9008511909421006
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.5023255813953489,
+              "brier": 0.5745442767822497,
+              "cross_entropy": 2.347590209909279
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8359375,
+              "brier": 0.19931059120575503,
+              "cross_entropy": 0.7444107758516407
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.7083333333333334,
+              "brier": 0.428241328947602,
+              "cross_entropy": 1.050083253299498
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.39655172413793105,
+              "brier": 0.7305371209519513,
+              "cross_entropy": 1.4973851876562216
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 4.986938069711992e-05,
+              "cross_entropy": 0.003831617325445817
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.6,
+              "brier": 0.7805844658893524,
+              "cross_entropy": 1.7801497473019683
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.7142857142857143,
+              "brier": 0.4790055889595874,
+              "cross_entropy": 1.1121963454176031
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.6101325803838552,
+              "cross_entropy": 2.0033591272403792
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.7,
+              "brier": 0.6003855550573081,
+              "cross_entropy": 1.611779150841159
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.75,
+              "brier": 0.42100449664852624,
+              "cross_entropy": 1.9128633478153523
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5357142857142857,
+              "brier": 0.2741592752209492,
+              "cross_entropy": 1.860460665385586
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 2.0002025677443218e-06,
+              "cross_entropy": 0.0004476462766485558
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 0.5,
+              "brier": 0.41023068518803085,
+              "cross_entropy": 0.6006732852714238
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0013119361198151522,
+              "cross_entropy": 0.02163855006976333
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.4296875,
+              "brier": 0.5887451086780798,
+              "cross_entropy": 2.801737422463417
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.6041666666666666,
+              "brier": 0.5293956848932907,
+              "cross_entropy": 1.6646787560472525
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.8826525379691635,
+              "cross_entropy": 2.2968173663260414
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.31120510035647425,
+              "cross_entropy": 0.4402550888479215
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6953125,
+              "brier": 0.4588827899272394,
+              "cross_entropy": 1.3847298212396886
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.4739583333333333,
+              "brier": 0.8022954836937664,
+              "cross_entropy": 2.0042890400598963
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 2.2949296518888497e-08,
+              "cross_entropy": 9.582785411797238e-05
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7358490566037735,
+              "brier": 0.0564251534824245,
+              "cross_entropy": 0.8602018822761758
+            }
+          }
+        }
+      },
+      {
+        "step": 2000,
+        "examples_seen": 8000,
+        "checkpoint": "step-002000",
+        "export": "step-002000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.7275390625,
+          "brier": 0.31872868889688055,
+          "cross_entropy": 1.116584448395983,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7398496240601504,
+              "brier": 0.3486314543321744,
+              "cross_entropy": 1.1847615210242326
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8472222222222222,
+              "brier": 0.24375927242920462,
+              "cross_entropy": 0.5036143361404846
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6093023255813953,
+              "brier": 0.27645081427769413,
+              "cross_entropy": 1.3162586012657769
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8515625,
+              "brier": 0.20112173562767713,
+              "cross_entropy": 0.7568937670273396
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.75,
+              "brier": 0.3778614792863498,
+              "cross_entropy": 1.0070228429929422
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.5517241379310345,
+              "brier": 0.6998687564446888,
+              "cross_entropy": 1.449144847710849
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 8.502797038864707e-06,
+              "cross_entropy": 0.0017491505823356403
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.6,
+              "brier": 0.64090513735759,
+              "cross_entropy": 1.2842974245060323
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.7142857142857143,
+              "brier": 0.5732229366022509,
+              "cross_entropy": 1.3498112142920367
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.75,
+              "brier": 0.45820962916227576,
+              "cross_entropy": 1.7893452530760972
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.6,
+              "brier": 0.7878584523917267,
+              "cross_entropy": 2.649907510922513
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.8062445012416534,
+              "cross_entropy": 1.8181549807503623
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5714285714285714,
+              "brier": 0.268346840698743,
+              "cross_entropy": 1.8491581077937622
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 4.014360934536871e-05,
+              "cross_entropy": 0.0018805145026462434
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 0.43785613217853164,
+              "cross_entropy": 0.6306845393838758
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0023963047501729595,
+              "cross_entropy": 0.02626187987425209
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.546875,
+              "brier": 0.20973734203719346,
+              "cross_entropy": 1.5167245986871056
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9114583333333334,
+              "brier": 0.1816467064252445,
+              "cross_entropy": 0.8230249970733392
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.7315860423730358,
+              "cross_entropy": 2.0558892146843153
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.5,
+              "brier": 0.6525781536661275,
+              "cross_entropy": 0.9808095278526777
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6953125,
+              "brier": 0.430457617431226,
+              "cross_entropy": 1.3267025517668618
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.6979166666666666,
+              "brier": 0.44132333714721916,
+              "cross_entropy": 0.939129528604241
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 1.4302336637850583e-07,
+              "cross_entropy": 0.00026036870363595557
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7358490566037735,
+              "brier": 0.05366664986435064,
+              "cross_entropy": 0.8647152527990684
+            }
+          }
+        }
+      },
+      {
+        "step": 4000,
+        "examples_seen": 16000,
+        "checkpoint": "step-004000",
+        "export": "step-004000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.740234375,
+          "brier": 0.31259603426070215,
+          "cross_entropy": 1.0641685621781927,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7413533834586467,
+              "brier": 0.35520899641938364,
+              "cross_entropy": 1.150404749554445
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8333333333333334,
+              "brier": 0.2587362169867197,
+              "cross_entropy": 0.527162702406588
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6744186046511628,
+              "brier": 0.2168667033394475,
+              "cross_entropy": 1.157107070094022
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.859375,
+              "brier": 0.1875144540034567,
+              "cross_entropy": 0.6708904667559872
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.7083333333333334,
+              "brier": 0.4071248120462074,
+              "cross_entropy": 1.072867837361791
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.4827586206896552,
+              "brier": 0.7226287558174715,
+              "cross_entropy": 1.4270151869584489
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 6.599285085529234e-06,
+              "cross_entropy": 0.0016185379703032327
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.8,
+              "brier": 0.39998067070790544,
+              "cross_entropy": 1.08609476980073
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.8571428571428571,
+              "brier": 0.28687468365522234,
+              "cross_entropy": 0.7248149597143485
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.75,
+              "brier": 0.477438586205062,
+              "cross_entropy": 1.9109389754190291
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.6,
+              "brier": 0.7693115419877218,
+              "cross_entropy": 2.051549980800976
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.25,
+              "brier": 1.1149087692116901,
+              "cross_entropy": 2.466424832470031
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.6428571428571429,
+              "brier": 0.26805081416207754,
+              "cross_entropy": 1.8479820614570504
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 1.6518035601985627e-05,
+              "cross_entropy": 0.0012598198888345246
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 0.5,
+              "brier": 0.43338634881169164,
+              "cross_entropy": 0.6242116836662074
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0005199031989910008,
+              "cross_entropy": 0.013970904245707016
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.5625,
+              "brier": 0.2123470190045601,
+              "cross_entropy": 1.5161004021555795
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9010416666666666,
+              "brier": 0.18652079389211781,
+              "cross_entropy": 0.7723674627217165
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.6806357215652038,
+              "cross_entropy": 1.9339496668179939
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.5780328053534197,
+              "cross_entropy": 0.9064917649110132
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6796875,
+              "brier": 0.49342345206712573,
+              "cross_entropy": 1.4302781643246851
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.765625,
+              "brier": 0.3713316872108095,
+              "cross_entropy": 0.7410961018920553
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 3.664008183371336e-07,
+              "cross_entropy": 0.00041208643515311584
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7547169811320755,
+              "brier": 0.056719285382028715,
+              "cross_entropy": 0.8641864337643139
+            }
+          }
+        }
+      },
+      {
+        "step": 8000,
+        "examples_seen": 32000,
+        "checkpoint": "step-008000",
+        "export": "step-008000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.76171875,
+          "brier": 0.27786017825450965,
+          "cross_entropy": 0.9927829125866763,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7654135338345864,
+              "brier": 0.3189465668614991,
+              "cross_entropy": 1.0534065611001757
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8958333333333334,
+              "brier": 0.16471210037099535,
+              "cross_entropy": 0.39482479972079665
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6604651162790698,
+              "brier": 0.22656192147115203,
+              "cross_entropy": 1.2057654334760237
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8671875,
+              "brier": 0.1790825849971539,
+              "cross_entropy": 0.6080971490404414
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.5263100827359132,
+              "cross_entropy": 1.403012675529207
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.5517241379310345,
+              "brier": 0.684833327087956,
+              "cross_entropy": 1.5818612127101548
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 8.422208826818114e-06,
+              "cross_entropy": 0.0016334122276107624
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.8,
+              "brier": 0.39072800981047207,
+              "cross_entropy": 0.8926558622999824
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.8571428571428571,
+              "brier": 0.28439123582360026,
+              "cross_entropy": 0.7928804085918381
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.6286826358235,
+              "cross_entropy": 2.0004531092707
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.7,
+              "brier": 0.44538602593083176,
+              "cross_entropy": 1.117689014223367
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.25,
+              "brier": 1.4342464951649092,
+              "cross_entropy": 3.381933827755728
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5535714285714286,
+              "brier": 0.2622696208064482,
+              "cross_entropy": 1.8219529101323455
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 2.417104630113183e-06,
+              "cross_entropy": 0.0005900334430327367
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 0.43780762834608883,
+              "cross_entropy": 0.6306076256145883
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0003744313065983422,
+              "cross_entropy": 0.011988960485765076
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.5859375,
+              "brier": 0.20697473621325052,
+              "cross_entropy": 1.5034032587243362
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9635416666666666,
+              "brier": 0.11802009440200961,
+              "cross_entropy": 0.6579051066101222
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.7315329429054658,
+              "cross_entropy": 2.0647249708287245
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.609860991064251,
+              "cross_entropy": 0.9741778978250341
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.703125,
+              "brier": 0.44470820194957994,
+              "cross_entropy": 1.2102900099661185
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.796875,
+              "brier": 0.29338767561802165,
+              "cross_entropy": 0.6140028830315581
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 2.1804417353162663e-07,
+              "cross_entropy": 0.0003234946827488933
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7169811320754716,
+              "brier": 0.056977519481840835,
+              "cross_entropy": 0.860752644211149
+            }
+          }
+        }
+      }
+    ],
+    "winner": {
+      "step": 8000,
+      "examples_seen": 32000,
+      "checkpoint": "step-008000",
+      "export": "step-008000",
+      "validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.76171875,
+        "brier": 0.27786017825450965,
+        "cross_entropy": 0.9927829125866763,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.7654135338345864,
+            "brier": 0.3189465668614991,
+            "cross_entropy": 1.0534065611001757
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.8958333333333334,
+            "brier": 0.16471210037099535,
+            "cross_entropy": 0.39482479972079665
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.6604651162790698,
+            "brier": 0.22656192147115203,
+            "cross_entropy": 1.2057654334760237
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1790825849971539,
+            "cross_entropy": 0.6080971490404414
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5263100827359132,
+            "cross_entropy": 1.403012675529207
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.5517241379310345,
+            "brier": 0.684833327087956,
+            "cross_entropy": 1.5818612127101548
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 8.422208826818114e-06,
+            "cross_entropy": 0.0016334122276107624
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.39072800981047207,
+            "cross_entropy": 0.8926558622999824
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.8571428571428571,
+            "brier": 0.28439123582360026,
+            "cross_entropy": 0.7928804085918381
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6286826358235,
+            "cross_entropy": 2.0004531092707
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.44538602593083176,
+            "cross_entropy": 1.117689014223367
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.25,
+            "brier": 1.4342464951649092,
+            "cross_entropy": 3.381933827755728
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5535714285714286,
+            "brier": 0.2622696208064482,
+            "cross_entropy": 1.8219529101323455
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.417104630113183e-06,
+            "cross_entropy": 0.0005900334430327367
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.43780762834608883,
+            "cross_entropy": 0.6306076256145883
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003744313065983422,
+            "cross_entropy": 0.011988960485765076
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.20697473621325052,
+            "cross_entropy": 1.5034032587243362
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.9635416666666666,
+            "brier": 0.11802009440200961,
+            "cross_entropy": 0.6579051066101222
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.7315329429054658,
+            "cross_entropy": 2.0647249708287245
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.609860991064251,
+            "cross_entropy": 0.9741778978250341
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.703125,
+            "brier": 0.44470820194957994,
+            "cross_entropy": 1.2102900099661185
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.796875,
+            "brier": 0.29338767561802165,
+            "cross_entropy": 0.6140028830315581
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.1804417353162663e-07,
+            "cross_entropy": 0.0003234946827488933
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7169811320754716,
+            "brier": 0.056977519481840835,
+            "cross_entropy": 0.860752644211149
+          }
+        }
+      }
+    },
+    "validation_split": "new_validation"
+  },
+  "results": {
+    "actions-48k-parent-nvfp4": {
+      "new_validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.6005859375,
+        "brier": 0.5027092950491365,
+        "cross_entropy": 1.6357855528374599,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.6030075187969924,
+            "brier": 0.4978527031414729,
+            "cross_entropy": 1.564796901472784
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.7361111111111112,
+            "brier": 0.4178836877295325,
+            "cross_entropy": 0.9008511909421006
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.5023255813953489,
+            "brier": 0.5745442767822497,
+            "cross_entropy": 2.347590209909279
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8359375,
+            "brier": 0.19931059120575503,
+            "cross_entropy": 0.7444107758516407
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.7083333333333334,
+            "brier": 0.428241328947602,
+            "cross_entropy": 1.050083253299498
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.39655172413793105,
+            "brier": 0.7305371209519513,
+            "cross_entropy": 1.4973851876562216
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 4.986938069711992e-05,
+            "cross_entropy": 0.003831617325445817
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.6,
+            "brier": 0.7805844658893524,
+            "cross_entropy": 1.7801497473019683
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.7142857142857143,
+            "brier": 0.4790055889595874,
+            "cross_entropy": 1.1121963454176031
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6101325803838552,
+            "cross_entropy": 2.0033591272403792
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.6003855550573081,
+            "cross_entropy": 1.611779150841159
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.75,
+            "brier": 0.42100449664852624,
+            "cross_entropy": 1.9128633478153523
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5357142857142857,
+            "brier": 0.2741592752209492,
+            "cross_entropy": 1.860460665385586
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.0002025677443218e-06,
+            "cross_entropy": 0.0004476462766485558
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.41023068518803085,
+            "cross_entropy": 0.6006732852714238
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0013119361198151522,
+            "cross_entropy": 0.02163855006976333
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.4296875,
+            "brier": 0.5887451086780798,
+            "cross_entropy": 2.801737422463417
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.6041666666666666,
+            "brier": 0.5293956848932907,
+            "cross_entropy": 1.6646787560472525
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.8826525379691635,
+            "cross_entropy": 2.2968173663260414
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.31120510035647425,
+            "cross_entropy": 0.4402550888479215
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.6953125,
+            "brier": 0.4588827899272394,
+            "cross_entropy": 1.3847298212396886
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.4739583333333333,
+            "brier": 0.8022954836937664,
+            "cross_entropy": 2.0042890400598963
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.2949296518888497e-08,
+            "cross_entropy": 9.582785411797238e-05
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7358490566037735,
+            "brier": 0.0564251534824245,
+            "cross_entropy": 0.8602018822761758
+          }
+        }
+      },
+      "new_test": {
+        "questions": 512,
+        "modal_accuracy": 0.46484375,
+        "brier": 0.7260609345715332,
+        "cross_entropy": 2.283754099438618,
+        "by_kind": {
+          "choice": {
+            "questions": 241,
+            "modal_accuracy": 0.3941908713692946,
+            "brier": 0.814820900706993,
+            "cross_entropy": 2.425967860593386
+          },
+          "noul": {
+            "questions": 94,
+            "modal_accuracy": 0.5851063829787234,
+            "brier": 0.7046781668962778,
+            "cross_entropy": 1.348769508075005
+          },
+          "score": {
+            "questions": 177,
+            "modal_accuracy": 0.4971751412429379,
+            "brier": 0.6165627895027659,
+            "cross_entropy": 2.586663902545287
+          }
+        },
+        "by_source": {
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5,
+            "brier": 0.5252889060517718,
+            "cross_entropy": 2.7138956389335567
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.484375,
+            "brier": 0.695581487251716,
+            "cross_entropy": 2.128417640206426
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.421875,
+            "brier": 0.8903884009045245,
+            "cross_entropy": 2.1523295323408504
+          }
+        }
+      },
+      "test": {
+        "questions": 2048,
+        "modal_accuracy": 0.79541015625,
+        "brier": 0.25207741014224244,
+        "cross_entropy": 0.7408834203798337,
+        "by_kind": {
+          "choice": {
+            "questions": 1151,
+            "modal_accuracy": 0.7176368375325803,
+            "brier": 0.34765282650467216,
+            "cross_entropy": 1.0022488366720779
+          },
+          "noul": {
+            "questions": 649,
+            "modal_accuracy": 0.9090909090909091,
+            "brier": 0.1294082456235435,
+            "cross_entropy": 0.3089760935689536
+          },
+          "score": {
+            "questions": 248,
+            "modal_accuracy": 0.8588709677419355,
+            "brier": 0.1295168598982062,
+            "cross_entropy": 0.6581264080729309
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.84375,
+            "brier": 0.1956377254181323,
+            "cross_entropy": 0.5228371535175946
+          },
+          "arithmetic": {
+            "questions": 30,
+            "modal_accuracy": 0.8,
+            "brier": 0.365987202749238,
+            "cross_entropy": 0.874987503758623
+          },
+          "babyai": {
+            "questions": 128,
+            "modal_accuracy": 0.4140625,
+            "brier": 0.7052731307744691,
+            "cross_entropy": 1.4919952201710827
+          },
+          "community-diversity-v2/browser_tools/goal_completion": {
+            "questions": 24,
+            "modal_accuracy": 0.9583333333333334,
+            "brier": 0.0757580121004977,
+            "cross_entropy": 0.10499604055191543
+          },
+          "community-diversity-v2/contracts/exclusivity_exception": {
+            "questions": 23,
+            "modal_accuracy": 0.8695652173913043,
+            "brier": 0.17625150922502755,
+            "cross_entropy": 0.3819911750055886
+          },
+          "community-diversity-v2/games/pit_stop": {
+            "questions": 13,
+            "modal_accuracy": 0.46153846153846156,
+            "brier": 0.8058966314340112,
+            "cross_entropy": 1.7033569197410379
+          },
+          "community-diversity-v2/ir/warranty_transfer": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.13580214677968178,
+            "cross_entropy": 0.36657722104104
+          },
+          "community-diversity-v2/rag/independent_sources": {
+            "questions": 21,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5103346094744698,
+            "cross_entropy": 0.9877849897459301
+          },
+          "community-diversity-v2/rubric_judge/balanced_tradeoff": {
+            "questions": 12,
+            "modal_accuracy": 0.8333333333333334,
+            "brier": 0.31827468282287835,
+            "cross_entropy": 0.7584622273231553
+          },
+          "community-diversity-v2/shell_history/incremental_copy": {
+            "questions": 17,
+            "modal_accuracy": 0.8823529411764706,
+            "brier": 0.23477636259692025,
+            "cross_entropy": 0.5888464829632447
+          },
+          "community-diversity-v2/support/outage_compensation": {
+            "questions": 3,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.46034493119811687,
+            "cross_entropy": 1.0544000105768865
+          },
+          "connect_four": {
+            "questions": 128,
+            "modal_accuracy": 0.5234375,
+            "brier": 0.2605442517143562,
+            "cross_entropy": 1.8198101893170386
+          },
+          "customer-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003903831787449739,
+            "cross_entropy": 0.01599108917033031
+          },
+          "devign": {
+            "questions": 46,
+            "modal_accuracy": 0.5434782608695652,
+            "brier": 0.5038973081034425,
+            "cross_entropy": 0.6986062066022048
+          },
+          "entity_belief_tracking": {
+            "questions": 19,
+            "modal_accuracy": 1.0,
+            "brier": 0.003996155280999412,
+            "cross_entropy": 0.018453305162397043
+          },
+          "event_state_reconstruction": {
+            "questions": 30,
+            "modal_accuracy": 1.0,
+            "brier": 0.000632084906836889,
+            "cross_entropy": 0.0070833207931613705
+          },
+          "event_triggered_execution": {
+            "questions": 18,
+            "modal_accuracy": 0.8888888888888888,
+            "brier": 0.20065607784122677,
+            "cross_entropy": 0.3866436913361192
+          },
+          "evidence_sufficiency": {
+            "questions": 13,
+            "modal_accuracy": 0.7692307692307693,
+            "brier": 0.3885272040680112,
+            "cross_entropy": 0.7354956714769301
+          },
+          "goal_directed_planning": {
+            "questions": 19,
+            "modal_accuracy": 0.8421052631578947,
+            "brier": 0.24661119154460764,
+            "cross_entropy": 0.80410976975599
+          },
+          "graph_navigation": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5177916242601774,
+            "cross_entropy": 1.158343409907232
+          },
+          "gsm8k": {
+            "questions": 106,
+            "modal_accuracy": 0.7547169811320755,
+            "brier": 0.38874049122073956,
+            "cross_entropy": 0.7555982932600139
+          },
+          "instruction_execution": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.16643936427800027,
+            "cross_entropy": 0.5910813947667087
+          },
+          "multi_view_adjudication": {
+            "questions": 20,
+            "modal_accuracy": 0.95,
+            "brier": 0.07032918266188046,
+            "cross_entropy": 0.09175669834651938
+          },
+          "needle_retrieval": {
+            "questions": 5,
+            "modal_accuracy": 1.0,
+            "brier": 1.0762412043050325e-08,
+            "cross_entropy": 4.8760052363298926e-05
+          },
+          "partial_observation_calibration": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.02878926642066244,
+            "cross_entropy": 0.5423652505807739
+          },
+          "policy_applicability": {
+            "questions": 17,
+            "modal_accuracy": 1.0,
+            "brier": 0.000332556851194119,
+            "cross_entropy": 0.006342650317050527
+          },
+          "policy_under_uncertainty": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 0.055895947994874184,
+            "cross_entropy": 0.5045127678250978
+          },
+          "race": {
+            "questions": 104,
+            "modal_accuracy": 0.9134615384615384,
+            "brier": 0.15016846124700772,
+            "cross_entropy": 0.3446495798522507
+          },
+          "ragtruth": {
+            "questions": 128,
+            "modal_accuracy": 0.84375,
+            "brier": 0.2673585181473687,
+            "cross_entropy": 0.4660759647283446
+          },
+          "reasoning-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9111111111111111,
+            "brier": 0.11092651727785302,
+            "cross_entropy": 0.2669770296039986
+          },
+          "record_aggregation": {
+            "questions": 14,
+            "modal_accuracy": 0.7857142857142857,
+            "brier": 0.29174772725112047,
+            "cross_entropy": 0.45793008149173803
+          },
+          "record_eligibility": {
+            "questions": 8,
+            "modal_accuracy": 0.875,
+            "brier": 0.10770378005071689,
+            "cross_entropy": 0.13562242943700845
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.6006833831415513,
+            "cross_entropy": 1.7746254397586039
+          },
+          "snake-v1": {
+            "questions": 30,
+            "modal_accuracy": 0.9666666666666667,
+            "brier": 0.04547834255423711,
+            "cross_entropy": 0.25335150127501904
+          },
+          "state_perturbation": {
+            "questions": 11,
+            "modal_accuracy": 0.8181818181818182,
+            "brier": 0.3589651830631727,
+            "cross_entropy": 0.9294419729230835
+          },
+          "table_lookup": {
+            "questions": 13,
+            "modal_accuracy": 0.8461538461538461,
+            "brier": 0.1995349559267647,
+            "cross_entropy": 0.7888754577823849
+          },
+          "taxonomy_routing": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 4.5835879900615156e-05,
+            "cross_entropy": 0.002082040171618654
+          },
+          "tic_tac_toe-v1": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.2455337950398313,
+            "cross_entropy": 1.1492408686263713
+          },
+          "tile_platformer-v1": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 0.023323220046764518,
+            "cross_entropy": 0.08534179201524755
+          },
+          "toolace": {
+            "questions": 128,
+            "modal_accuracy": 0.9921875,
+            "brier": 0.015580959620615913,
+            "cross_entropy": 0.043558856167122205
+          },
+          "trex_runner-v1": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.01809983422643395,
+            "cross_entropy": 0.11356731402761971
+          },
+          "vizdoom-basic-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9555555555555556,
+            "brier": 0.08476731050032603,
+            "cross_entropy": 0.18352009286167528
+          },
+          "wanli-decisions-v1": {
+            "questions": 128,
+            "modal_accuracy": 0.6953125,
+            "brier": 0.5342434192250224,
+            "cross_entropy": 1.027889201322489
+          },
+          "workflow-controls-v1/agent_trace_observability": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.13236608755793375,
+            "cross_entropy": 0.37446550855959687
+          },
+          "workflow-controls-v1/customer_service": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 9.801834989730725e-07,
+            "cross_entropy": 0.00023351213246618595
+          },
+          "workflow-controls-v1/invoice_processing": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 4.168477226094833e-07,
+            "cross_entropy": 0.00024165434698863794
+          },
+          "workflow-controls-v1/security_incidents": {
+            "questions": 14,
+            "modal_accuracy": 1.0,
+            "brier": 2.5913148618252294e-09,
+            "cross_entropy": 1.9618643712043932e-05
+          },
+          "yuri_v3": {
+            "questions": 256,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.03336715127934959,
+            "cross_entropy": 0.8210925298273019
+          }
+        }
+      },
+      "transfer": {
+        "questions": 89,
+        "modal_accuracy": 0.6404494382022472,
+        "brier": 0.49972145070841484,
+        "cross_entropy": 1.069735656698559,
+        "by_kind": {
+          "choice": {
+            "questions": 33,
+            "modal_accuracy": 0.6060606060606061,
+            "brier": 0.5839105237267139,
+            "cross_entropy": 1.3151504789912494
+          },
+          "noul": {
+            "questions": 41,
+            "modal_accuracy": 0.7560975609756098,
+            "brier": 0.34728857025217713,
+            "cross_entropy": 0.6781416340204177
+          },
+          "score": {
+            "questions": 15,
+            "modal_accuracy": 0.4,
+            "brier": 0.7311553633152064,
+            "cross_entropy": 1.6001800429748936
+          }
+        },
+        "by_source": {
+          "basic_cls": {
+            "questions": 12,
+            "modal_accuracy": 0.5833333333333334,
+            "brier": 0.4769190960406326,
+            "cross_entropy": 0.7929537222990599
+          },
+          "code": {
+            "questions": 6,
+            "modal_accuracy": 0.5,
+            "brier": 0.6355124607251713,
+            "cross_entropy": 0.9367448318704151
+          },
+          "dialog": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.4233092995278946,
+            "cross_entropy": 1.4220052117820898
+          },
+          "judge": {
+            "questions": 8,
+            "modal_accuracy": 0.75,
+            "brier": 0.4294109009179358,
+            "cross_entropy": 1.1813213005452772
+          },
+          "know": {
+            "questions": 10,
+            "modal_accuracy": 0.8,
+            "brier": 0.22602505333734763,
+            "cross_entropy": 0.39225738115288933
+          },
+          "ret": {
+            "questions": 10,
+            "modal_accuracy": 0.4,
+            "brier": 0.9045065751061194,
+            "cross_entropy": 1.655086462390568
+          },
+          "route": {
+            "questions": 9,
+            "modal_accuracy": 0.4444444444444444,
+            "brier": 0.6393115918944736,
+            "cross_entropy": 1.1139742544661488
+          },
+          "safety": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.17156815175055062,
+            "cross_entropy": 0.4272056920862004
+          },
+          "step": {
+            "questions": 8,
+            "modal_accuracy": 0.625,
+            "brier": 0.710965126411655,
+            "cross_entropy": 1.9533480392831426
+          },
+          "struct": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.8092234753736499,
+            "cross_entropy": 1.1517337483418988
+          }
+        }
+      }
+    },
+    "state-32k-nvfp4": {
+      "new_validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.76171875,
+        "brier": 0.27786017825450965,
+        "cross_entropy": 0.9927829125866763,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.7654135338345864,
+            "brier": 0.3189465668614991,
+            "cross_entropy": 1.0534065611001757
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.8958333333333334,
+            "brier": 0.16471210037099535,
+            "cross_entropy": 0.39482479972079665
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.6604651162790698,
+            "brier": 0.22656192147115203,
+            "cross_entropy": 1.2057654334760237
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1790825849971539,
+            "cross_entropy": 0.6080971490404414
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5263100827359132,
+            "cross_entropy": 1.403012675529207
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.5517241379310345,
+            "brier": 0.684833327087956,
+            "cross_entropy": 1.5818612127101548
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 8.422208826818114e-06,
+            "cross_entropy": 0.0016334122276107624
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.39072800981047207,
+            "cross_entropy": 0.8926558622999824
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.8571428571428571,
+            "brier": 0.28439123582360026,
+            "cross_entropy": 0.7928804085918381
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6286826358235,
+            "cross_entropy": 2.0004531092707
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.44538602593083176,
+            "cross_entropy": 1.117689014223367
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.25,
+            "brier": 1.4342464951649092,
+            "cross_entropy": 3.381933827755728
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5535714285714286,
+            "brier": 0.2622696208064482,
+            "cross_entropy": 1.8219529101323455
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.417104630113183e-06,
+            "cross_entropy": 0.0005900334430327367
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.43780762834608883,
+            "cross_entropy": 0.6306076256145883
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003744313065983422,
+            "cross_entropy": 0.011988960485765076
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.20697473621325052,
+            "cross_entropy": 1.5034032587243362
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.9635416666666666,
+            "brier": 0.11802009440200961,
+            "cross_entropy": 0.6579051066101222
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.7315329429054658,
+            "cross_entropy": 2.0647249708287245
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.609860991064251,
+            "cross_entropy": 0.9741778978250341
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.703125,
+            "brier": 0.44470820194957994,
+            "cross_entropy": 1.2102900099661185
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.796875,
+            "brier": 0.29338767561802165,
+            "cross_entropy": 0.6140028830315581
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.1804417353162663e-07,
+            "cross_entropy": 0.0003234946827488933
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7169811320754716,
+            "brier": 0.056977519481840835,
+            "cross_entropy": 0.860752644211149
+          }
+        }
+      },
+      "new_test": {
+        "questions": 512,
+        "modal_accuracy": 0.755859375,
+        "brier": 0.2705710538872297,
+        "cross_entropy": 1.0354930422480668,
+        "by_kind": {
+          "choice": {
+            "questions": 241,
+            "modal_accuracy": 0.8215767634854771,
+            "brier": 0.30593746240810665,
+            "cross_entropy": 1.0288481446263855
+          },
+          "noul": {
+            "questions": 94,
+            "modal_accuracy": 0.8936170212765957,
+            "brier": 0.1853453372075174,
+            "cross_entropy": 0.4718470608079094
+          },
+          "score": {
+            "questions": 177,
+            "modal_accuracy": 0.5932203389830508,
+            "brier": 0.26767790651074164,
+            "cross_entropy": 1.34387802858818
+          }
+        },
+        "by_source": {
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5078125,
+            "brier": 0.22898458428820745,
+            "cross_entropy": 1.5346420909767997
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.921875,
+            "brier": 0.1909474730913934,
+            "cross_entropy": 0.8915435249464682
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.7552083333333334,
+            "brier": 0.3779189477490808,
+            "cross_entropy": 0.8466765270638437
+          }
+        }
+      },
+      "test": {
+        "questions": 2048,
+        "modal_accuracy": 0.79345703125,
+        "brier": 0.2641515323526975,
+        "cross_entropy": 0.784624370615209,
+        "by_kind": {
+          "choice": {
+            "questions": 1151,
+            "modal_accuracy": 0.7193744569939183,
+            "brier": 0.36420584491384933,
+            "cross_entropy": 1.0683460554623134
+          },
+          "noul": {
+            "questions": 649,
+            "modal_accuracy": 0.9090909090909091,
+            "brier": 0.12795487547550322,
+            "cross_entropy": 0.3201729560342784
+          },
+          "score": {
+            "questions": 248,
+            "modal_accuracy": 0.8346774193548387,
+            "brier": 0.1562044216890414,
+            "cross_entropy": 0.6832748093410428
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1662207666853679,
+            "cross_entropy": 0.4678064926902018
+          },
+          "arithmetic": {
+            "questions": 30,
+            "modal_accuracy": 0.7333333333333333,
+            "brier": 0.5001721787123533,
+            "cross_entropy": 1.2885683809129926
+          },
+          "babyai": {
+            "questions": 128,
+            "modal_accuracy": 0.484375,
+            "brier": 0.7067526312424577,
+            "cross_entropy": 1.6676958019283898
+          },
+          "community-diversity-v2/browser_tools/goal_completion": {
+            "questions": 24,
+            "modal_accuracy": 0.9583333333333334,
+            "brier": 0.06921012297338076,
+            "cross_entropy": 0.10122484092336533
+          },
+          "community-diversity-v2/contracts/exclusivity_exception": {
+            "questions": 23,
+            "modal_accuracy": 0.8695652173913043,
+            "brier": 0.24976225264440965,
+            "cross_entropy": 0.5909170782174602
+          },
+          "community-diversity-v2/games/pit_stop": {
+            "questions": 13,
+            "modal_accuracy": 0.7692307692307693,
+            "brier": 0.4785595907160497,
+            "cross_entropy": 1.2971969110502473
+          },
+          "community-diversity-v2/ir/warranty_transfer": {
+            "questions": 15,
+            "modal_accuracy": 0.8666666666666667,
+            "brier": 0.14068735459076726,
+            "cross_entropy": 0.17808536247316048
+          },
+          "community-diversity-v2/rag/independent_sources": {
+            "questions": 21,
+            "modal_accuracy": 0.47619047619047616,
+            "brier": 0.9915299974666058,
+            "cross_entropy": 2.310007596806081
+          },
+          "community-diversity-v2/rubric_judge/balanced_tradeoff": {
+            "questions": 12,
+            "modal_accuracy": 0.75,
+            "brier": 0.4814089394948157,
+            "cross_entropy": 1.102602184903888
+          },
+          "community-diversity-v2/shell_history/incremental_copy": {
+            "questions": 17,
+            "modal_accuracy": 0.8823529411764706,
+            "brier": 0.20441354684881466,
+            "cross_entropy": 0.5009642986702474
+          },
+          "community-diversity-v2/support/outage_compensation": {
+            "questions": 3,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.4884574890213706,
+            "cross_entropy": 1.3886592311282289
+          },
+          "connect_four": {
+            "questions": 128,
+            "modal_accuracy": 0.59375,
+            "brier": 0.2561712647462938,
+            "cross_entropy": 1.802544508268942
+          },
+          "customer-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 1.0,
+            "brier": 0.00019936847019355491,
+            "cross_entropy": 0.015971454914726523
+          },
+          "devign": {
+            "questions": 46,
+            "modal_accuracy": 0.5217391304347826,
+            "brier": 0.5225763998633566,
+            "cross_entropy": 0.722933623300632
+          },
+          "entity_belief_tracking": {
+            "questions": 19,
+            "modal_accuracy": 0.8421052631578947,
+            "brier": 0.259415578060587,
+            "cross_entropy": 0.6433967485445069
+          },
+          "event_state_reconstruction": {
+            "questions": 30,
+            "modal_accuracy": 1.0,
+            "brier": 0.015897166327449284,
+            "cross_entropy": 0.02589146516518676
+          },
+          "event_triggered_execution": {
+            "questions": 18,
+            "modal_accuracy": 0.8888888888888888,
+            "brier": 0.16562186759207273,
+            "cross_entropy": 0.2757516689442136
+          },
+          "evidence_sufficiency": {
+            "questions": 13,
+            "modal_accuracy": 0.6923076923076923,
+            "brier": 0.5880021120988826,
+            "cross_entropy": 1.4889127617030355
+          },
+          "goal_directed_planning": {
+            "questions": 19,
+            "modal_accuracy": 0.8421052631578947,
+            "brier": 0.2509895082794504,
+            "cross_entropy": 0.4540842796343451
+          },
+          "graph_navigation": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.44283093090962034,
+            "cross_entropy": 1.3194420415465578
+          },
+          "gsm8k": {
+            "questions": 106,
+            "modal_accuracy": 0.7075471698113207,
+            "brier": 0.43472841661725453,
+            "cross_entropy": 0.8890682888838121
+          },
+          "instruction_execution": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.16555489236402793,
+            "cross_entropy": 0.49206138807591987
+          },
+          "multi_view_adjudication": {
+            "questions": 20,
+            "modal_accuracy": 1.0,
+            "brier": 5.2326944475445606e-05,
+            "cross_entropy": 0.0019525076494628021
+          },
+          "needle_retrieval": {
+            "questions": 5,
+            "modal_accuracy": 1.0,
+            "brier": 3.199473123353177e-08,
+            "cross_entropy": 8.974482553742138e-05
+          },
+          "partial_observation_calibration": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.02661607292190903,
+            "cross_entropy": 0.5399351074049796
+          },
+          "policy_applicability": {
+            "questions": 17,
+            "modal_accuracy": 1.0,
+            "brier": 6.106492112719508e-05,
+            "cross_entropy": 0.003675418740395231
+          },
+          "policy_under_uncertainty": {
+            "questions": 8,
+            "modal_accuracy": 0.875,
+            "brier": 0.10177113114450002,
+            "cross_entropy": 0.560772453887988
+          },
+          "race": {
+            "questions": 104,
+            "modal_accuracy": 0.9134615384615384,
+            "brier": 0.16990537457587326,
+            "cross_entropy": 0.3822661135973287
+          },
+          "ragtruth": {
+            "questions": 128,
+            "modal_accuracy": 0.8515625,
+            "brier": 0.2317892915096781,
+            "cross_entropy": 0.39549112820971377
+          },
+          "reasoning-control-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9111111111111111,
+            "brier": 0.1285355661042637,
+            "cross_entropy": 0.30974376864438463
+          },
+          "record_aggregation": {
+            "questions": 14,
+            "modal_accuracy": 0.7857142857142857,
+            "brier": 0.35331876706309845,
+            "cross_entropy": 0.696535349171323
+          },
+          "record_eligibility": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 0.00021253785775029812,
+            "cross_entropy": 0.0043067743592292455
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.5625,
+            "brier": 0.6406304488569265,
+            "cross_entropy": 1.8806145121011755
+          },
+          "snake-v1": {
+            "questions": 30,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.04564391684280246,
+            "cross_entropy": 0.2431433189091872
+          },
+          "state_perturbation": {
+            "questions": 11,
+            "modal_accuracy": 1.0,
+            "brier": 0.0002344655717187281,
+            "cross_entropy": 0.005994309073216685
+          },
+          "table_lookup": {
+            "questions": 13,
+            "modal_accuracy": 0.9230769230769231,
+            "brier": 0.1498702753732909,
+            "cross_entropy": 0.3551783229099786
+          },
+          "taxonomy_routing": {
+            "questions": 8,
+            "modal_accuracy": 1.0,
+            "brier": 0.045424091680658744,
+            "cross_entropy": 0.07014486556652395
+          },
+          "tic_tac_toe-v1": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.26386162320957124,
+            "cross_entropy": 1.1837003824426573
+          },
+          "tile_platformer-v1": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.10164899419557688,
+            "cross_entropy": 0.220570822391115
+          },
+          "toolace": {
+            "questions": 128,
+            "modal_accuracy": 0.9921875,
+            "brier": 0.01562046345535589,
+            "cross_entropy": 0.04367168294967396
+          },
+          "trex_runner-v1": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.36032773502188475,
+            "cross_entropy": 0.5817032440936637
+          },
+          "vizdoom-basic-v1": {
+            "questions": 45,
+            "modal_accuracy": 0.9555555555555556,
+            "brier": 0.08693939517676995,
+            "cross_entropy": 0.20638551041670192
+          },
+          "wanli-decisions-v1": {
+            "questions": 128,
+            "modal_accuracy": 0.6875,
+            "brier": 0.5529247926341857,
+            "cross_entropy": 1.0802996209215674
+          },
+          "workflow-controls-v1/agent_trace_observability": {
+            "questions": 15,
+            "modal_accuracy": 0.9333333333333333,
+            "brier": 0.13168080686035147,
+            "cross_entropy": 0.3395590286213042
+          },
+          "workflow-controls-v1/customer_service": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 7.55085699524451e-06,
+            "cross_entropy": 0.0007668466946998275
+          },
+          "workflow-controls-v1/invoice_processing": {
+            "questions": 15,
+            "modal_accuracy": 1.0,
+            "brier": 1.1169422484519878e-06,
+            "cross_entropy": 0.0002747598457579768
+          },
+          "workflow-controls-v1/security_incidents": {
+            "questions": 14,
+            "modal_accuracy": 1.0,
+            "brier": 2.4204779837314167e-09,
+            "cross_entropy": 2.2446460647594408e-05
+          },
+          "yuri_v3": {
+            "questions": 256,
+            "modal_accuracy": 0.83203125,
+            "brier": 0.03526590762889966,
+            "cross_entropy": 0.8245288048118414
+          }
+        }
+      },
+      "transfer": {
+        "questions": 89,
+        "modal_accuracy": 0.651685393258427,
+        "brier": 0.47914605790680204,
+        "cross_entropy": 1.0043952783379058,
+        "by_kind": {
+          "choice": {
+            "questions": 33,
+            "modal_accuracy": 0.6363636363636364,
+            "brier": 0.5172571933930417,
+            "cross_entropy": 1.2048737574594803
+          },
+          "noul": {
+            "questions": 41,
+            "modal_accuracy": 0.7317073170731707,
+            "brier": 0.3731759473027374,
+            "cross_entropy": 0.7052377430670229
+          },
+          "score": {
+            "questions": 15,
+            "modal_accuracy": 0.4666666666666667,
+            "brier": 0.6849531954881848,
+            "cross_entropy": 1.381039887344188
+          }
+        },
+        "by_source": {
+          "basic_cls": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.4038371458185728,
+            "cross_entropy": 0.7355992645221435
+          },
+          "code": {
+            "questions": 6,
+            "modal_accuracy": 0.3333333333333333,
+            "brier": 0.8287228425681598,
+            "cross_entropy": 1.262918501594028
+          },
+          "dialog": {
+            "questions": 12,
+            "modal_accuracy": 0.5833333333333334,
+            "brier": 0.30448197088852874,
+            "cross_entropy": 1.1186203509870534
+          },
+          "judge": {
+            "questions": 8,
+            "modal_accuracy": 0.875,
+            "brier": 0.3292730875017608,
+            "cross_entropy": 0.7769559856794094
+          },
+          "know": {
+            "questions": 10,
+            "modal_accuracy": 0.8,
+            "brier": 0.26539280354851125,
+            "cross_entropy": 0.4839723017070307
+          },
+          "ret": {
+            "questions": 10,
+            "modal_accuracy": 0.5,
+            "brier": 0.814367705138209,
+            "cross_entropy": 1.6245878852220568
+          },
+          "route": {
+            "questions": 9,
+            "modal_accuracy": 0.4444444444444444,
+            "brier": 0.7471958010258476,
+            "cross_entropy": 1.3369414656592875
+          },
+          "safety": {
+            "questions": 12,
+            "modal_accuracy": 0.9166666666666666,
+            "brier": 0.1672822055997213,
+            "cross_entropy": 0.43712494222128856
+          },
+          "step": {
+            "questions": 8,
+            "modal_accuracy": 0.625,
+            "brier": 0.6510657904776266,
+            "cross_entropy": 1.5015904290656892
+          },
+          "struct": {
+            "questions": 2,
+            "modal_accuracy": 0.5,
+            "brier": 0.8996839553398099,
+            "cross_entropy": 1.4855438457791856
+          }
+        }
+      }
+    }
+  },
+  "scope": {
+    "new_validation": "1024 selection questions: 512 new and 512 previous; all four candidate checkpoints scored",
+    "new_test": "512 reserved new questions; first scored after checkpoint selection",
+    "test": "2048 earlier questions; reused to check retention of earlier abilities",
+    "transfer": "89 general questions; small reused development check",
+    "parent": "Previous actions-48k adapter, re-exported to byte-identical NVFP4 and rescored with the same Eider binary",
+    "calibration": "All scores are uncalibrated. No fitted profile for the new weights.",
+    "bf16": "Matching merged BF16 weights are included but were not independently scored in this native evaluation.",
+    "base_model": "The unfine-tuned NVIDIA model was not rescored in this evaluation."
+  }
+}

nvfp4/evaluation/runtime.json

@@ -0,0 +1,36 @@
+{
+  "server_sha256": "935335c9df451b4b9108d65fc46882abbfd7a0b6cb20b3ba76f4e8c42fa7ab0c",
+  "training_recipe_server_sha256": "2119b000902c32d33d57b2be9fc7630ff57b830818edc09a0a768a6e6dc8ebbd",
+  "reason": "Use one frozen copy of the current Eider binary for all candidates after unrelated module development.",
+  "scripts_sha256": {
+    "dory_lora.py": "cdcd877f30e8b2c6c9892a22281770125261423d22eec9890246496cb3d3ad26",
+    "dory-decision-corpus.py": "f81de0dc66b7c433a13b71d705d5650ed813c31fb600ba4f73e459191387c4c0",
+    "dory_nvfp4.py": "baf5ad4237ff4390c41696a32ba97cb3b82ec87fe0bb78ea8878909f0a1b3772",
+    "dory-decision-state.py": "3324e067b8e03913bcceff4cbe8a35675604f4f6a861ccb9f17750f6ef1a689d",
+    "dory-decision-lora.py": "6fdf5ba67a09b7ac75dbec8450a1044c277437b14340b7cd31e65d99015f309c",
+    "dory-decision-verda.py": "5c347955220c157d8fa4fb2bc157569245f6ab70419f98f5c141645d75ef9f1c",
+    "dory-decision-nvfp4.py": "cbd6c833654b1fff319abf1d8f3cb491af5d96b78d541e593a283b684268da04",
+    "sha256.json": "f0e48a77a313f739023f0ee23dd5c287c7197bb31ef61facc8c9eb3f5b4c1b49",
+    "dory-decision-targeted.py": "f78f2633a35fa96967a956ee8060cafb5861e4d65e449c3edb5e7dcbc84ded82",
+    "dory-decision-continuation.py": "b044299772c69184e256c593aa97cb33fc2d79a8f0bb1bab4c6230aedd46ba40",
+    "decision-eval.py": "15cf0b44d2d21ae2484e97dc298b4b3e25e541bedf653436b0c5f7431a982956",
+    "run-dory-decision-overnight.sh": "f12d624177c78a28a255b4b910e24b92d4883662076ddaf2afe3561288dd76f7",
+    "dory-decision-overnight.py": "0ffa9a9353edf2cb77ce8b9254182a1753a88a036517b1b9f199f8d4981a9c1a",
+    "dory_pytorch.py": "51582e13b8a8742f0597bf1cb6c3aaf2b1fa03b3d3f8a2e02a1577b0b58f6d0d",
+    "dory-decision-actions.py": "37e23a2d72b5738b5a977bd1625a385cdee9b014173ca6165d47d7b250c1b735"
+  },
+  "validation_split": "new_validation",
+  "test_splits": [
+    "new_test",
+    "test",
+    "transfer"
+  ],
+  "calibration": "uncalibrated",
+  "candidate_steps": [
+    0,
+    2000,
+    4000,
+    8000
+  ],
+  "parent_comparison_step": 0
+}

nvfp4/generation_config.json

@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": [
+    2,
+    11
+  ],
+  "pad_token_id": 11,
+  "top_p": 0.95,
+  "temperature": 1.0,
+  "transformers_version": "4.57.6"
+}

nvfp4/hf_quant_config.json

@@ -0,0 +1,15 @@
+{
+  "producer": {
+    "name": "eider-mse",
+    "version": "1"
+  },
+  "quantization": {
+    "quant_algo": "W4A16_NVFP4",
+    "kv_cache_quant_algo": null,
+    "group_size": 16,
+    "exclude_modules": [
+      "backbone.embeddings",
+      "lm_head"
+    ]
+  }
+}

nvfp4/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bc834f4dce29a48499b6614c11772d189710138b29c637a384f753b1326b4c0
+size 3202498032

nvfp4/provenance.json

@@ -0,0 +1,861 @@
+{
+  "base_model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16",
+  "base_revision": "b366b87f762ea334ba3eb55912031baba1515810",
+  "base_weights_sha256": "207483664dfef54c4c4438863260d3ba423bf3d6009b43fe84264b281e9fda23",
+  "release": "state-32k",
+  "selected_step": 8000,
+  "selected_stage_examples": 32000,
+  "precision": "nvfp4",
+  "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+  "weights_sha256": {
+    "model.safetensors": "9bc834f4dce29a48499b6614c11772d189710138b29c637a384f753b1326b4c0",
+    "adapter/adapter.safetensors": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d"
+  },
+  "evaluated_nvfp4_sha256": "9bc834f4dce29a48499b6614c11772d189710138b29c637a384f753b1326b4c0",
+  "accuracy_evaluation_precision": "nvfp4",
+  "bf16_merge": null,
+  "previous_release": {
+    "repo": "rdaum/eider-dory-1.0-4b-decisions-nvfp4",
+    "revision": "8236908a6268050e9cba08f4647d73895be79804",
+    "tag": "actions-48k"
+  },
+  "training": {
+    "rank": 8,
+    "alpha": 16.0,
+    "loops": 2,
+    "accumulation": 4,
+    "micro_batch_size": 4,
+    "learning_rate": 1e-05,
+    "seed": 20261024,
+    "initialization": {
+      "adapter_sha256": "4b6a812baccf0e4586ab7c31c1574abab979f064ac81dec8baee7e424b659e96",
+      "run_sha256": "5aafa5d840ea5517c42325ed1ce9c61fcf94519b2701c8e9255d6668833f9876",
+      "report_sha256": "39bf8fb9a20c9a811c8907b31bcf93cad312a3869e59c9b0e5e2f4e8f9adbbaf",
+      "parent_completed_steps": 12000,
+      "parent_examples_seen": 48000,
+      "optimizer": "fresh"
+    },
+    "loss": "soft-target conditional cross-entropy + 0.5 * Brier",
+    "torch": "2.12.0+cu130"
+  },
+  "training_report": {
+    "adapter_sha256": "60723ac470780ac7e8bfe7785c23665be2d5c5d6653cbd2fad653523214aea7d",
+    "training_state_sha256": "3e39348ed11b584af07de43d1138399d551714226c9703a1c1794853ddb7b6bf",
+    "completed_steps": 8000,
+    "examples_seen": 32000,
+    "training_seconds": 2700.4836061550304,
+    "peak_memory_gib": 52.067845821380615
+  },
+  "selection": {
+    "criterion": "NVFP4 validation Brier, cross-entropy, earliest step",
+    "candidates": [
+      {
+        "step": 0,
+        "examples_seen": 0,
+        "checkpoint": "step-000000",
+        "export": "step-000000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.6005859375,
+          "brier": 0.5027092950491365,
+          "cross_entropy": 1.6357855528374599,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.6030075187969924,
+              "brier": 0.4978527031414729,
+              "cross_entropy": 1.564796901472784
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.7361111111111112,
+              "brier": 0.4178836877295325,
+              "cross_entropy": 0.9008511909421006
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.5023255813953489,
+              "brier": 0.5745442767822497,
+              "cross_entropy": 2.347590209909279
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8359375,
+              "brier": 0.19931059120575503,
+              "cross_entropy": 0.7444107758516407
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.7083333333333334,
+              "brier": 0.428241328947602,
+              "cross_entropy": 1.050083253299498
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.39655172413793105,
+              "brier": 0.7305371209519513,
+              "cross_entropy": 1.4973851876562216
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 4.986938069711992e-05,
+              "cross_entropy": 0.003831617325445817
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.6,
+              "brier": 0.7805844658893524,
+              "cross_entropy": 1.7801497473019683
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.7142857142857143,
+              "brier": 0.4790055889595874,
+              "cross_entropy": 1.1121963454176031
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.6101325803838552,
+              "cross_entropy": 2.0033591272403792
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.7,
+              "brier": 0.6003855550573081,
+              "cross_entropy": 1.611779150841159
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.75,
+              "brier": 0.42100449664852624,
+              "cross_entropy": 1.9128633478153523
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5357142857142857,
+              "brier": 0.2741592752209492,
+              "cross_entropy": 1.860460665385586
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 2.0002025677443218e-06,
+              "cross_entropy": 0.0004476462766485558
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 0.5,
+              "brier": 0.41023068518803085,
+              "cross_entropy": 0.6006732852714238
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0013119361198151522,
+              "cross_entropy": 0.02163855006976333
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.4296875,
+              "brier": 0.5887451086780798,
+              "cross_entropy": 2.801737422463417
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.6041666666666666,
+              "brier": 0.5293956848932907,
+              "cross_entropy": 1.6646787560472525
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.8826525379691635,
+              "cross_entropy": 2.2968173663260414
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.31120510035647425,
+              "cross_entropy": 0.4402550888479215
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6953125,
+              "brier": 0.4588827899272394,
+              "cross_entropy": 1.3847298212396886
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.4739583333333333,
+              "brier": 0.8022954836937664,
+              "cross_entropy": 2.0042890400598963
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 2.2949296518888497e-08,
+              "cross_entropy": 9.582785411797238e-05
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7358490566037735,
+              "brier": 0.0564251534824245,
+              "cross_entropy": 0.8602018822761758
+            }
+          }
+        }
+      },
+      {
+        "step": 2000,
+        "examples_seen": 8000,
+        "checkpoint": "step-002000",
+        "export": "step-002000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.7275390625,
+          "brier": 0.31872868889688055,
+          "cross_entropy": 1.116584448395983,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7398496240601504,
+              "brier": 0.3486314543321744,
+              "cross_entropy": 1.1847615210242326
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8472222222222222,
+              "brier": 0.24375927242920462,
+              "cross_entropy": 0.5036143361404846
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6093023255813953,
+              "brier": 0.27645081427769413,
+              "cross_entropy": 1.3162586012657769
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8515625,
+              "brier": 0.20112173562767713,
+              "cross_entropy": 0.7568937670273396
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.75,
+              "brier": 0.3778614792863498,
+              "cross_entropy": 1.0070228429929422
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.5517241379310345,
+              "brier": 0.6998687564446888,
+              "cross_entropy": 1.449144847710849
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 8.502797038864707e-06,
+              "cross_entropy": 0.0017491505823356403
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.6,
+              "brier": 0.64090513735759,
+              "cross_entropy": 1.2842974245060323
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.7142857142857143,
+              "brier": 0.5732229366022509,
+              "cross_entropy": 1.3498112142920367
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.75,
+              "brier": 0.45820962916227576,
+              "cross_entropy": 1.7893452530760972
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.6,
+              "brier": 0.7878584523917267,
+              "cross_entropy": 2.649907510922513
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.8062445012416534,
+              "cross_entropy": 1.8181549807503623
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5714285714285714,
+              "brier": 0.268346840698743,
+              "cross_entropy": 1.8491581077937622
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 4.014360934536871e-05,
+              "cross_entropy": 0.0018805145026462434
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 0.43785613217853164,
+              "cross_entropy": 0.6306845393838758
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0023963047501729595,
+              "cross_entropy": 0.02626187987425209
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.546875,
+              "brier": 0.20973734203719346,
+              "cross_entropy": 1.5167245986871056
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9114583333333334,
+              "brier": 0.1816467064252445,
+              "cross_entropy": 0.8230249970733392
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.7315860423730358,
+              "cross_entropy": 2.0558892146843153
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.5,
+              "brier": 0.6525781536661275,
+              "cross_entropy": 0.9808095278526777
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6953125,
+              "brier": 0.430457617431226,
+              "cross_entropy": 1.3267025517668618
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.6979166666666666,
+              "brier": 0.44132333714721916,
+              "cross_entropy": 0.939129528604241
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 1.4302336637850583e-07,
+              "cross_entropy": 0.00026036870363595557
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7358490566037735,
+              "brier": 0.05366664986435064,
+              "cross_entropy": 0.8647152527990684
+            }
+          }
+        }
+      },
+      {
+        "step": 4000,
+        "examples_seen": 16000,
+        "checkpoint": "step-004000",
+        "export": "step-004000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.740234375,
+          "brier": 0.31259603426070215,
+          "cross_entropy": 1.0641685621781927,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7413533834586467,
+              "brier": 0.35520899641938364,
+              "cross_entropy": 1.150404749554445
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8333333333333334,
+              "brier": 0.2587362169867197,
+              "cross_entropy": 0.527162702406588
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6744186046511628,
+              "brier": 0.2168667033394475,
+              "cross_entropy": 1.157107070094022
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.859375,
+              "brier": 0.1875144540034567,
+              "cross_entropy": 0.6708904667559872
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.7083333333333334,
+              "brier": 0.4071248120462074,
+              "cross_entropy": 1.072867837361791
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.4827586206896552,
+              "brier": 0.7226287558174715,
+              "cross_entropy": 1.4270151869584489
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 6.599285085529234e-06,
+              "cross_entropy": 0.0016185379703032327
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.8,
+              "brier": 0.39998067070790544,
+              "cross_entropy": 1.08609476980073
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.8571428571428571,
+              "brier": 0.28687468365522234,
+              "cross_entropy": 0.7248149597143485
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.75,
+              "brier": 0.477438586205062,
+              "cross_entropy": 1.9109389754190291
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.6,
+              "brier": 0.7693115419877218,
+              "cross_entropy": 2.051549980800976
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.25,
+              "brier": 1.1149087692116901,
+              "cross_entropy": 2.466424832470031
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.6428571428571429,
+              "brier": 0.26805081416207754,
+              "cross_entropy": 1.8479820614570504
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 1.6518035601985627e-05,
+              "cross_entropy": 0.0012598198888345246
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 0.5,
+              "brier": 0.43338634881169164,
+              "cross_entropy": 0.6242116836662074
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0005199031989910008,
+              "cross_entropy": 0.013970904245707016
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.5625,
+              "brier": 0.2123470190045601,
+              "cross_entropy": 1.5161004021555795
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9010416666666666,
+              "brier": 0.18652079389211781,
+              "cross_entropy": 0.7723674627217165
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.6806357215652038,
+              "cross_entropy": 1.9339496668179939
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.5780328053534197,
+              "cross_entropy": 0.9064917649110132
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.6796875,
+              "brier": 0.49342345206712573,
+              "cross_entropy": 1.4302781643246851
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.765625,
+              "brier": 0.3713316872108095,
+              "cross_entropy": 0.7410961018920553
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 3.664008183371336e-07,
+              "cross_entropy": 0.00041208643515311584
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7547169811320755,
+              "brier": 0.056719285382028715,
+              "cross_entropy": 0.8641864337643139
+            }
+          }
+        }
+      },
+      {
+        "step": 8000,
+        "examples_seen": 32000,
+        "checkpoint": "step-008000",
+        "export": "step-008000",
+        "validation": {
+          "questions": 1024,
+          "modal_accuracy": 0.76171875,
+          "brier": 0.27786017825450965,
+          "cross_entropy": 0.9927829125866763,
+          "by_kind": {
+            "choice": {
+              "questions": 665,
+              "modal_accuracy": 0.7654135338345864,
+              "brier": 0.3189465668614991,
+              "cross_entropy": 1.0534065611001757
+            },
+            "noul": {
+              "questions": 144,
+              "modal_accuracy": 0.8958333333333334,
+              "brier": 0.16471210037099535,
+              "cross_entropy": 0.39482479972079665
+            },
+            "score": {
+              "questions": 215,
+              "modal_accuracy": 0.6604651162790698,
+              "brier": 0.22656192147115203,
+              "cross_entropy": 1.2057654334760237
+            }
+          },
+          "by_source": {
+            "alfworld": {
+              "questions": 128,
+              "modal_accuracy": 0.8671875,
+              "brier": 0.1790825849971539,
+              "cross_entropy": 0.6080971490404414
+            },
+            "arithmetic": {
+              "questions": 24,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.5263100827359132,
+              "cross_entropy": 1.403012675529207
+            },
+            "babyai": {
+              "questions": 58,
+              "modal_accuracy": 0.5517241379310345,
+              "brier": 0.684833327087956,
+              "cross_entropy": 1.5818612127101548
+            },
+            "community-diversity-v2/browser_tools/idempotent_retry": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 8.422208826818114e-06,
+              "cross_entropy": 0.0016334122276107624
+            },
+            "community-diversity-v2/contracts/governing_law_venue": {
+              "questions": 5,
+              "modal_accuracy": 0.8,
+              "brier": 0.39072800981047207,
+              "cross_entropy": 0.8926558622999824
+            },
+            "community-diversity-v2/games/racing_overtake": {
+              "questions": 7,
+              "modal_accuracy": 0.8571428571428571,
+              "brier": 0.28439123582360026,
+              "cross_entropy": 0.7928804085918381
+            },
+            "community-diversity-v2/ir/access_schedule": {
+              "questions": 12,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.6286826358235,
+              "cross_entropy": 2.0004531092707
+            },
+            "community-diversity-v2/rag/policy_exception": {
+              "questions": 10,
+              "modal_accuracy": 0.7,
+              "brier": 0.44538602593083176,
+              "cross_entropy": 1.117689014223367
+            },
+            "community-diversity-v2/rubric_judge/trace_consistency": {
+              "questions": 4,
+              "modal_accuracy": 0.25,
+              "brier": 1.4342464951649092,
+              "cross_entropy": 3.381933827755728
+            },
+            "connect_four": {
+              "questions": 56,
+              "modal_accuracy": 0.5535714285714286,
+              "brier": 0.2622696208064482,
+              "cross_entropy": 1.8219529101323455
+            },
+            "customer-control-v1": {
+              "questions": 7,
+              "modal_accuracy": 1.0,
+              "brier": 2.417104630113183e-06,
+              "cross_entropy": 0.0005900334430327367
+            },
+            "devign": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 0.43780762834608883,
+              "cross_entropy": 0.6306076256145883
+            },
+            "gsm8k": {
+              "questions": 4,
+              "modal_accuracy": 1.0,
+              "brier": 0.0003744313065983422,
+              "cross_entropy": 0.011988960485765076
+            },
+            "helpsteer2": {
+              "questions": 128,
+              "modal_accuracy": 0.5859375,
+              "brier": 0.20697473621325052,
+              "cross_entropy": 1.5034032587243362
+            },
+            "planning": {
+              "questions": 192,
+              "modal_accuracy": 0.9635416666666666,
+              "brier": 0.11802009440200961,
+              "cross_entropy": 0.6579051066101222
+            },
+            "race": {
+              "questions": 4,
+              "modal_accuracy": 0.5,
+              "brier": 0.7315329429054658,
+              "cross_entropy": 2.0647249708287245
+            },
+            "ragtruth": {
+              "questions": 6,
+              "modal_accuracy": 0.6666666666666666,
+              "brier": 0.609860991064251,
+              "cross_entropy": 0.9741778978250341
+            },
+            "scienceworld": {
+              "questions": 128,
+              "modal_accuracy": 0.703125,
+              "brier": 0.44470820194957994,
+              "cross_entropy": 1.2102900099661185
+            },
+            "scone": {
+              "questions": 192,
+              "modal_accuracy": 0.796875,
+              "brier": 0.29338767561802165,
+              "cross_entropy": 0.6140028830315581
+            },
+            "toolace": {
+              "questions": 2,
+              "modal_accuracy": 1.0,
+              "brier": 2.1804417353162663e-07,
+              "cross_entropy": 0.0003234946827488933
+            },
+            "yuri_v3": {
+              "questions": 53,
+              "modal_accuracy": 0.7169811320754716,
+              "brier": 0.056977519481840835,
+              "cross_entropy": 0.860752644211149
+            }
+          }
+        }
+      }
+    ],
+    "winner": {
+      "step": 8000,
+      "examples_seen": 32000,
+      "checkpoint": "step-008000",
+      "export": "step-008000",
+      "validation": {
+        "questions": 1024,
+        "modal_accuracy": 0.76171875,
+        "brier": 0.27786017825450965,
+        "cross_entropy": 0.9927829125866763,
+        "by_kind": {
+          "choice": {
+            "questions": 665,
+            "modal_accuracy": 0.7654135338345864,
+            "brier": 0.3189465668614991,
+            "cross_entropy": 1.0534065611001757
+          },
+          "noul": {
+            "questions": 144,
+            "modal_accuracy": 0.8958333333333334,
+            "brier": 0.16471210037099535,
+            "cross_entropy": 0.39482479972079665
+          },
+          "score": {
+            "questions": 215,
+            "modal_accuracy": 0.6604651162790698,
+            "brier": 0.22656192147115203,
+            "cross_entropy": 1.2057654334760237
+          }
+        },
+        "by_source": {
+          "alfworld": {
+            "questions": 128,
+            "modal_accuracy": 0.8671875,
+            "brier": 0.1790825849971539,
+            "cross_entropy": 0.6080971490404414
+          },
+          "arithmetic": {
+            "questions": 24,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.5263100827359132,
+            "cross_entropy": 1.403012675529207
+          },
+          "babyai": {
+            "questions": 58,
+            "modal_accuracy": 0.5517241379310345,
+            "brier": 0.684833327087956,
+            "cross_entropy": 1.5818612127101548
+          },
+          "community-diversity-v2/browser_tools/idempotent_retry": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 8.422208826818114e-06,
+            "cross_entropy": 0.0016334122276107624
+          },
+          "community-diversity-v2/contracts/governing_law_venue": {
+            "questions": 5,
+            "modal_accuracy": 0.8,
+            "brier": 0.39072800981047207,
+            "cross_entropy": 0.8926558622999824
+          },
+          "community-diversity-v2/games/racing_overtake": {
+            "questions": 7,
+            "modal_accuracy": 0.8571428571428571,
+            "brier": 0.28439123582360026,
+            "cross_entropy": 0.7928804085918381
+          },
+          "community-diversity-v2/ir/access_schedule": {
+            "questions": 12,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.6286826358235,
+            "cross_entropy": 2.0004531092707
+          },
+          "community-diversity-v2/rag/policy_exception": {
+            "questions": 10,
+            "modal_accuracy": 0.7,
+            "brier": 0.44538602593083176,
+            "cross_entropy": 1.117689014223367
+          },
+          "community-diversity-v2/rubric_judge/trace_consistency": {
+            "questions": 4,
+            "modal_accuracy": 0.25,
+            "brier": 1.4342464951649092,
+            "cross_entropy": 3.381933827755728
+          },
+          "connect_four": {
+            "questions": 56,
+            "modal_accuracy": 0.5535714285714286,
+            "brier": 0.2622696208064482,
+            "cross_entropy": 1.8219529101323455
+          },
+          "customer-control-v1": {
+            "questions": 7,
+            "modal_accuracy": 1.0,
+            "brier": 2.417104630113183e-06,
+            "cross_entropy": 0.0005900334430327367
+          },
+          "devign": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 0.43780762834608883,
+            "cross_entropy": 0.6306076256145883
+          },
+          "gsm8k": {
+            "questions": 4,
+            "modal_accuracy": 1.0,
+            "brier": 0.0003744313065983422,
+            "cross_entropy": 0.011988960485765076
+          },
+          "helpsteer2": {
+            "questions": 128,
+            "modal_accuracy": 0.5859375,
+            "brier": 0.20697473621325052,
+            "cross_entropy": 1.5034032587243362
+          },
+          "planning": {
+            "questions": 192,
+            "modal_accuracy": 0.9635416666666666,
+            "brier": 0.11802009440200961,
+            "cross_entropy": 0.6579051066101222
+          },
+          "race": {
+            "questions": 4,
+            "modal_accuracy": 0.5,
+            "brier": 0.7315329429054658,
+            "cross_entropy": 2.0647249708287245
+          },
+          "ragtruth": {
+            "questions": 6,
+            "modal_accuracy": 0.6666666666666666,
+            "brier": 0.609860991064251,
+            "cross_entropy": 0.9741778978250341
+          },
+          "scienceworld": {
+            "questions": 128,
+            "modal_accuracy": 0.703125,
+            "brier": 0.44470820194957994,
+            "cross_entropy": 1.2102900099661185
+          },
+          "scone": {
+            "questions": 192,
+            "modal_accuracy": 0.796875,
+            "brier": 0.29338767561802165,
+            "cross_entropy": 0.6140028830315581
+          },
+          "toolace": {
+            "questions": 2,
+            "modal_accuracy": 1.0,
+            "brier": 2.1804417353162663e-07,
+            "cross_entropy": 0.0003234946827488933
+          },
+          "yuri_v3": {
+            "questions": 53,
+            "modal_accuracy": 0.7169811320754716,
+            "brier": 0.056977519481840835,
+            "cross_entropy": 0.860752644211149
+          }
+        }
+      }
+    },
+    "validation_split": "new_validation"
+  },
+  "calibration": {
+    "status": "not fitted for state-32k",
+    "included": false,
+    "enabled_by_default": false,
+    "previous_profile": "Removed from main because it applies to actions-48k weights; retained in actions-48k history."
+  }
+}

nvfp4/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

nvfp4/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:623c34567aebb18582765289fbe23d901c62704d6518d71866e0e58db892b5b7
+size 17077484