| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-2B-Base |
| language: [en] |
| pipeline_tag: text-classification |
| tags: [decision-model, calibrated, structured-output, multi-task, system-one, one-pass] |
| --- |
| |
| # decider-2b: typed decisions with calibrated probabilities in one forward pass |
|
|
| A language model that does not generate text. It reads a state and one or more typed questions, each with an explicit option |
| list, and returns a probability distribution over the options for every question from one forward pass. There is no decoding, |
| no parsing and no output outside the options you defined. It is called from software, not chatted with. It is an open |
| reproduction of the "System One" model class (TypeSafe AI's Jev). |
|
|
| Base model: [Qwen/Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base) (1.9B parameters). The supervised stages |
| (v1 to v8) fine-tune it with cross-entropy, a proper scoring rule, on a mixture of about 95 public decision datasets, agent |
| trajectories, web element choice, game states and teacher-written custom questions, in two prompt layouts and with isolated |
| Score levels. **This repository holds v10**: the v8 weights continued for 384 steps of calibration-aware reinforcement learning |
| whose only rewards are outcomes (live browser task checkers and the exact probability laws of games), with a hard KL limit to |
| the v8 weights on replayed training rows. Code, data registry, training scripts and the recipe are at |
| https://github.com/Mapika/decider; `decider/` in this repository is the inference subset of that package. The other sizes and the vision variant are listed under The decider family. |
|
|
| **Contents:** [The decider family](#the-decider-family) · [Usage](#usage) · [How it works](#how-it-works) · [Field types](#field-types) · [Training](#training) · [Evaluation](#evaluation) · [Speed](#speed) · [Limitations](#limitations) · [Changelog](#changelog) · [Reproduction](#reproduction) |
|
|
| ## The decider family |
|
|
| All six repositories share one interface (`decider.infer.Decider`, `POST /v1/systemone` in TypeSafe's format) and one |
| readout: the letter logits at an answer slot, softmaxed over the options. Pick by size and input. |
|
|
| | model | base | weights | use it for | numbers | |
| |---|---|---|---|---| |
| | [decider-2b](https://huggingface.co/Mapika/decider-2b) v10 | Qwen3.5-2B-Base | 3.5 GB bf16 | the default: routing, classification, judgments, browser agents; 4 ms per request with CUDA graphs on one GPU | regression set 0.805 in-task / 0.755 held-out; live browser 93%; Bespoke suite 0.704 | |
| | [decider-4b](https://huggingface.co/Mapika/decider-4b) v1 | Qwen3.5-4B-Base | 8.4 GB bf16 | the middle point: knowledge and reasoning questions above the 2B in a dense 8.4 GB model; no RL stage | 0.834 / 0.788, above the 2B on 87 of 95 tasks; JevBench hard 0.541; Bespoke 0.757 | |
| | [decider-35b-a3b](https://huggingface.co/Mapika/decider-35b-a3b) v1 | Qwen3.5-35B-A3B-Base (3B active) | 65 GB bf16 | when accuracy is worth 3 to 4 times the cost per decision: knowledge and multi-step questions, long policies | 0.855 / 0.810, above the 2B on 93 of 95 tasks; JevBench hard 0.676; Bespoke 0.774; no RL stage | |
| | [decider-35b-a3b-nvfp4](https://huggingface.co/Mapika/decider-35b-a3b-nvfp4) | the 35B in NVFP4 | 19.6 GB | the 35B on Blackwell through vLLM or TensorRT-LLM | 1.0 to 1.5 points under bf16 on the measured fixtures | |
| | [decider-0.8b](https://huggingface.co/Mapika/decider-0.8b) | Qwen3.5-0.8B-Base | 1.4 GB bf16 | the smallest: routing, yes/no and short-state lookups within 1 to 4 points of the 2B, 1.5x faster | 0.776 / 0.707 on the single-run protocol (2B: 0.809 / 0.739) | |
| | [decider-2b-vision](https://huggingface.co/Mapika/decider-2b-vision) | Qwen3.5-2B vision-language, v5 text weights | 4.1 GB bf16 | decisions from an image plus a question; game frames | Visual7W 0.89; Breakout 41 from pixels | |
|
|
| Code, data registry, training scripts, the changelog and the per-version history: https://github.com/Mapika/decider. |
|
|
| ## Usage |
|
|
| ```python |
| from decider.infer import Decider # decider/ is included in this repo |
| d = Decider("Mapika/decider-2b") |
| d.decide("My card was charged twice for the same purchase.", |
| [{"question": "Which department should handle this?", "options": ["billing", "technical support", "sales"]}, |
| {"question": "Does this need a refund action?", "options": ["no", "yes"]}]) |
| # [{'choice': 'billing', 'confidence': 0.99, 'probs': {...}}, {'choice': 'yes', 'confidence': 0.99, 'probs': {...}}] |
| ``` |
|
|
| `decide_batch` scores many states, each with many questions, in one call. `abstain_below=t` returns `None` for decisions with |
| confidence under `t`. A question can have 2 to 255 options (more than 10 options use one label token per option, see |
| `decider/prompt.py`). |
|
|
| The same request shape as TypeSafe's Jev (`POST /v1/systemone`), in process or over HTTP: |
|
|
| ```python |
| d.system_one({"ticket": {"messages": [{"from": "customer", "text": "I was charged twice for order A-104. Please refund the duplicate."}]}, |
| "refund_policy": "Duplicate charges are eligible for a refund."}, |
| {"department": {"type": "choice", "instructions": "Which team should handle this?", |
| "criteria": {"returns": "Exchanges, refunds, wrong or damaged items", |
| "billing": {"what": "Charges, invoices", "not_for": "delivery"}, "other": None}}, |
| "refund_requested": {"type": "noul", "instructions": "Does `ticket.messages[0].text` request a refund?"}, |
| "frustration": {"type": "score", "instructions": "How frustrated is the customer?", "criteria": ["calm", "frustrated", "very frustrated"]}}) |
| # {"model": "decider-v10", "answers": {"department": {"type": "choice", "choice": "billing", "confidence": ..., "certainty": ..., "probabilities": {...}}, |
| # "refund_requested": {"type": "noul", "noul": ...}, "frustration": {"type": "score", "score": ..., "legend": {...}, ...}}, "usage": {...}} |
| ``` |
|
|
| The state may be a string, object or array (up to 32k tokens with the questions). `instructions` and every option |
| description may be a string or any JSON value. Question ids are never shown to the model. Each question is scored in its own |
| row, so an answer does not depend on which other questions are asked (`independent=False` packs them into one row, about half |
| the latency for short states). Each Score level is likewise judged in its own row, without its number or its neighbours, and the |
| per-level fits are normalised (`"isolated": false` restores listwise scoring). The answer also reports `level_fit` and their |
| sum `fit_mass`, which is near 1 when exactly one level fits. |
|
|
| For a fixed set of questions, `s = d.schema(questions)` computes the question prefix once and `s(state)` / `s.batch(states)` |
| then run only the state (1.2 to 2.4x faster per request, up to 19x per batch). It uses a questions-first prompt layout that |
| costs accuracy: about 1.5 points on fixed label sets, 5 on per-example options, more on 50 or more options and on states of |
| several thousand tokens. `decider.serve` exposes the same thing as `POST /v1/systemone`; the official `typesafe-sdk` works |
| against it unchanged with `TYPESAFE_BASE_URL` pointing at the server. |
|
|
| Requirements: `torch`, `transformers>=5`, and `flash-linear-attention` (Triton kernels for the Qwen3.5 linear-attention |
| layers; the model runs without it but several times slower). Python 3.11 or newer lets those kernels use `torch.compile`. |
|
|
| Without the helper package, the same computation in plain `transformers`: |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| tok = AutoTokenizer.from_pretrained(REPO); m = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16).cuda().eval() |
| prompt = ("Context:\nMy card was charged twice for the same purchase.\n\n" |
| "Question: Which department should handle this?\nOptions:\n(A) billing\n(B) technical support\n(C) sales\nAnswer: (") |
| ids = tok(prompt, return_tensors="pt").to("cuda") |
| with torch.no_grad(): |
| logits = m(**ids).logits[0, -1] |
| letters = [tok.encode(L, add_special_tokens=False)[0] for L in "ABC"] |
| probs = torch.softmax(logits[letters].float() / 1.30, -1) # -> P(billing), P(technical support), P(sales); 1.30 is the stored temperature |
| ``` |
|
|
| For several questions in one pass, append further `Question k: ... Answer k: (` blocks and read the logits at each `(` |
| position (see `decider/prompt.py`). |
|
|
| ## How it works |
|
|
| The prompt is `Context: ...` followed by, for each question, the question text, the lettered options `(A) ... (B) ...` and an |
| answer slot `Answer k: (`. The hidden state at each slot is projected with the option-letter rows of the LM head and softmaxed |
| over the valid letters, divided by the temperature in `decider_config.json`. Letters are never generated, so all slots are read |
| from one pass. Large label sets were sub-sampled to at most 10 options per training example (gold always kept, order shuffled), |
| so the model conditions on the supplied candidates rather than on a fixed head. |
|
|
| ## Field types |
|
|
| * **noul**: probability of "yes". |
| * **choice** with `criteria` {name: description | JSON | null}: the argmax option, its probability (`x_p_max`, the calibrated |
| number), `confidence` (TypeSafe's definition, `(n·x_p_max − 1)/(n − 1)` for n options), `certainty` (1 minus the normalised |
| entropy) and the full distribution. |
| * **score** with `criteria` [level descriptions]: the expected level, the probability of the most likely level (`x_p_max`), |
| `confidence` (TypeSafe's definition: 1 minus the expected distance from the most likely level, divided by the mean distance of |
| the levels from the middle of the scale, floored at 0), the distribution, and the per-level fits. |
|
|
| Before decider-ai 1.3.0, `confidence` in `system_one` and `POST /v1/systemone` answers was the probability that is now `x_p_max`. |
| The plain form (`decide`, `POST /decide`) still reports the top probability as `confidence`. |
|
|
| ## Training |
|
|
| **Supervised stages (v1 to v8).** One epoch on a mixture of public decision datasets (intent detection, ticket routing, topic |
| classification, sentiment, emotion, moderation, NLI, paraphrase, fact verification, passage relevance, reading comprehension, |
| multiple-choice QA, ordinal rating scales, pairwise response preference, tool selection), then continuation epochs that added |
| next-action choice from agent trajectories (AgentGym), web element choice (Mind2Web), teacher-written situations and game states, |
| the input shapes of the Jev API (described options, up to 255 options, JSON states with path references, long inputs), teacher- |
| written custom questions with a generic option next to a catch-all, a second cacheable prompt layout, and isolated Score levels. |
| In 10% of questions with three or more options an abstain option is added; in a quarter of those the option list is replaced by |
| labels from an unrelated task so that the abstain option is correct. The full list of components with sizes is in |
| `decider/data/mixture.py` of the GitHub repository; `scripts/train.sh full` reproduces the supervised stages in one run. |
|
|
| **Reinforcement learning stage (v8 to v10).** 384 optimizer steps at a peak learning rate of 1e-6 (cosine, 16 warm-up steps), |
| selected among the checkpoints of a 576-step run. Each of the 48 iterations plays 4 live MiniWoB++ click tasks, 4 minesweeper |
| boards and 4 game boards (a 5x5 grid with a slippery move, draws from bags of known composition), 4 repeats each, through the |
| same one-pass readout that serves requests. Three loss terms use those rollouts: a PPO clipped surrogate (clip 0.2) on the |
| terminal outcome with a leave-one-replicate-out baseline; a proper log score of the model's stated belief about the immediate |
| outcome of its action against the exact law (games, minesweeper) or the realised outcome (browser); and a rendering-consistency |
| term that pulls the model's answer in the other prompt layout and the reversed option order toward its served answer. A fourth |
| term keeps the model where it was: on 8 replayed supervised rows per step, KL(v8 ‖ student) on the served distribution must |
| stay under 0.01 nats on average and 0.05 on any row, otherwise the step drops the reward terms and follows only the KL |
| gradient. Six browser tasks were held out from reward and used for validation only. No gold labels were used. The recipe and |
| every measurement are in `docs/RL.md` of the GitHub repository. |
|
|
| ## Evaluation |
|
|
| **94 public tasks, original protocol.** Large label sets sub-sampled to 10 options; one temperature fitted on in-task data |
| and stored in `decider_config.json`. "In-task" means the test splits of the training datasets; "held-out" means datasets never |
| seen in training (TREC, BBC news, PAWS, SciQ, Social IQa, StrategyQA, PubMedQA, TruthfulQA, tweet irony, financial sentiment, |
| ADE, MASSIVE scenario, student question categories, Dolly categories, CR reviews, Financial PhraseBank, CommitmentBank, |
| QuALITY, XStoryCloze, RewardBench, Arena preferences, Hermes tool selection, and an abstention probe). ECE is the expected |
| calibration error with 15 bins. |
|
|
| | model | in-task (69 tasks) acc / NLL / ECE | held-out (24 tasks) acc / NLL / ECE | |
| |---|---|---| |
| | Qwen3.5-2B-Base, zero-shot | 0.620 / 0.908 / 0.121 | 0.642 / 0.853 / 0.105 | |
| | decider-2b v8, T=1.30 | 0.811 / 0.460 / 0.037 | 0.741 / 0.655 / 0.088 | |
| | decider-2b v9, T=1.36 | 0.812 / 0.464 / 0.041 | 0.741 / 0.655 / 0.087 | |
| | decider-2b v8, rebuilt set (67 / 28 tasks, see note), T=1.30 | 0.806 / 0.473 / 0.038 | 0.757 / 0.622 / 0.083 | |
| | **decider-2b v10 (this repository), rebuilt set, T=1.30** | 0.805 / 0.474 / 0.037 | 0.755 / 0.622 / 0.084 | |
| | v8, questions-first layout (schema cache), T=1.18 | 0.790 / 0.500 / 0.038 | 0.707 / 0.757 / 0.104 | |
|
|
| The two "rebuilt set" rows were measured after the data pipeline was rebuilt on another machine: two datasets no longer download |
| (TREC-fine, the game states) and the current mixture adds held-out probes, so that set has 67 in-task and 28 held-out tasks. Its |
| numbers are comparable to each other, not to the rows above. v10 matches v8 on it. |
|
|
| <details> |
| <summary><b>Per-task accuracy / ECE on the 28 held-out datasets, v8 against v10</b></summary> |
|
|
| Per-task accuracy / ECE on the held-out datasets of the rebuilt set, v8 against v10: |
|
|
| | task | v8 acc / ECE | v10 acc / ECE | |
| |---|---|---| |
| | abstain_probe | 0.633 / 0.112 | 0.606 / 0.134 | |
| | ade | 0.811 / 0.044 | 0.817 / 0.038 | |
| | arena_pref | 0.487 / 0.173 | 0.483 / 0.189 | |
| | bbc_news | 0.924 / 0.014 | 0.927 / 0.013 | |
| | cb | 0.911 / 0.090 | 0.857 / 0.093 | |
| | cr_reviews | 0.900 / 0.027 | 0.903 / 0.031 | |
| | dbpedia_l2 | 0.948 / 0.017 | 0.950 / 0.018 | |
| | dbpedia_l3 | 0.989 / 0.007 | 0.987 / 0.005 | |
| | dolly_category | 0.291 / 0.209 | 0.299 / 0.203 | |
| | fin_phrasebank | 0.684 / 0.043 | 0.694 / 0.042 | |
| | fin_sentiment | 0.794 / 0.069 | 0.793 / 0.058 | |
| | hermes_tools | 0.718 / 0.209 | 0.723 / 0.208 | |
| | hwu64 | 0.964 / 0.031 | 0.961 / 0.030 | |
| | massive_scenario | 0.766 / 0.040 | 0.756 / 0.041 | |
| | offtopic_probe | 0.841 / 0.033 | 0.841 / 0.027 | |
| | paws | 0.707 / 0.169 | 0.724 / 0.145 | |
| | pubmedqa | 0.752 / 0.083 | 0.756 / 0.085 | |
| | quality | 0.495 / 0.236 | 0.494 / 0.233 | |
| | quality_full | 0.505 / 0.205 | 0.508 / 0.198 | |
| | reward_bench | 0.825 / 0.042 | 0.819 / 0.045 | |
| | sciq | 0.982 / 0.022 | 0.982 / 0.024 | |
| | social_iqa | 0.698 / 0.072 | 0.708 / 0.077 | |
| | strategyqa | 0.559 / 0.123 | 0.552 / 0.138 | |
| | student_questions | 0.927 / 0.036 | 0.925 / 0.045 | |
| | trec | 0.792 / 0.057 | 0.784 / 0.066 | |
| | truthfulqa | 0.529 / 0.102 | 0.537 / 0.090 | |
| | tweet_irony | 0.801 / 0.048 | 0.795 / 0.052 | |
| | xstory_cloze | 0.962 / 0.017 | 0.962 / 0.017 | |
|
|
| </details> |
|
|
| **v10 against v8 on the same rows.** Every row below is scored by both models on identical inputs and seeds. Intervals are |
| 95% bootstrap or paired intervals. |
|
|
| | | v8 | v10 | difference | |
| |---|---|---|---| |
| | live MiniWoB++ click tasks, 22 tasks x 8 seeds, sampled play | 83.0% | 93.2% | +10.2 (+5.1 to +15.9) | |
| | the 6 tasks never used for reward | 72.9% | 91.7% | +18.8 (+6.2 to +31.2) | |
| | same tasks, greedy play | 90.3% | 90.9% | +0.6 | |
| | Mind2Web element and action choice, 1,770 rows | 81.1% | 82.7% | +1.5 (+0.7 to +2.4) | |
| | bag-draw games, win rate, 64 boards x 4 | 35.2% | 41.4% | +6.2 (+0.8 to +11.7) | |
| | slippery-grid games, win rate, 64 boards x 4 | 14.1% | 18.8% | +4.7 (−2.0 to +11.3) | |
| | stated belief, nats above the exact law (lower is better) | 0.473 | 0.219 | | |
| | click-outcome prediction, log score (higher is better) | −0.349 | −0.034 | | |
| | TypeSafe workflow decisions, 102 rows, accuracy / NLL | 78.4% / 0.594 | 80.4% / 0.585 | +2.0 (−2.0 to +5.9) | |
| | 847 in-task validation rows, accuracy / NLL | 83.6% / 0.443 | 83.2% / 0.444 | −0.4 (−1.3 to +0.6) | |
| | Bespoke's public suite, 13 subsets, macro accuracy | 0.706 | 0.704 | | |
| | JevBench public items, easy / standard / hard accuracy | 1.000 / 0.875 / 0.441 | 1.000 / 0.889 / 0.459 | +1 / +2 items | |
| | OpenJev, 5,252 rows, accuracy / NLL | 64.1% / 0.906 | 63.3% / 0.916 | −0.8 (−1.3 to −0.3) | |
|
|
| The browser gain is in the served distribution rather than in the argmax: sampled play improves by ten points, greedy play by |
| under one. Tic-tac-toe and minesweeper play did not change; a 2B model without search loses most of those games either |
| way. The one measured regression is OpenJev, under one point. The JevBench row was read again for both versions on 2026-09-23, in process, bf16, decider-ai |
| 1.2.1. The values first published here (standard / hard: v8 0.861 / 0.459, v10 0.847 / 0.459) came from the FP8 server of |
| 2026-09-19 and do not reproduce item for item. |
|
|
| **Bespoke's public suite** (13 human-labelled subsets, 3,880 records in Jev's wire format, answered through `system_one` as |
| shipped). decider-2b v10 macro 0.704 / micro 0.711; v9 0.701 / 0.711; Nimble-9B 0.748 / 0.759; Jev 1.13.0 0.760 / 0.773 |
| (the last two copied from Bespoke's report). Per-subset numbers, the JevBench public-item comparison (decider-2b v10 is at 1.000 / 0.889 / 0.459 on the easy / standard / |
| hard public items, against Jev 1.13.0 at 1.000 / 0.986 / 0.730) and recordings of both versions on the same browser pages and |
| game boards are in the GitHub README. |
|
|
| ## Speed |
|
|
| One NVIDIA B300, decider-ai 1.2.1, measured 2026-09-23. Support-ticket states of about 230 tokens with 3 typed questions each |
| (the first 64 `support_tickets` examples). `decider.infer.Decider` uses shape-bucketed CUDA graphs; the batching server is |
| `decider/serve.py`, whose default since 1.1 is bf16. |
|
|
| | setting | p50 latency | throughput | |
| |---|---|---| |
| | single request, eager PyTorch | 18.9 ms | | |
| | single request, CUDA graphs + torch.compile (helper default) | 3.2 ms | | |
| | batch of 32, in-process, bf16 | 35.5 ms | about 2,700 decisions/s | |
| | batch of 32, in-process, FP8 linears | 32.3 ms | about 2,980 decisions/s | |
| | HTTP server `/decide` (bf16, default), 1 client | 6.1 ms | 158 req/s | |
| | HTTP server `/decide` (bf16, default), 64 clients | 134 ms | 436 req/s, 2,181 decisions/s | |
| | HTTP server `/decide` with `DECIDER_FP8=1`, 64 clients | 212 ms | 286 req/s, 1,429 decisions/s | |
|
|
| On the B300, FP8 is faster in process but slower through the server, so the server default is bf16. The schema-cache figures |
| were measured earlier on one GH200 and not repeated: with `Decider.schema`, 10 described questions on short chat messages ran at |
| 11,180 decisions/s in a batch, and one question with 151 options at 19x the full-forward rate. FP8 (e4m3 weights, per-token |
| activation scales) changes accuracy and calibration by less than the evaluation noise. |
|
|
| ## Limitations |
|
|
| * A 2B model without reasoning. Knowledge-heavy multiple choice (MMLU, MedQA, ARC) improves little over the base model, and a |
| judgment that needs several steps should be split into several questions. |
| * English only. Calibration is measured on public datasets and teacher-labelled probes, not on your traffic. Check it on your |
| own labels before using confidence for routing. |
| * v10 continues the v8 weights. The v9 data for terse bucket names (`support`, `help`, `account` next to `other`) is not in it: |
| on held-out terse-bucket messages v8 chose the generic bucket correctly 59% of the time where v9 reached 86%. Name or |
| describe the generic option as a bucket (`general_support`, or a description). |
| * Rules written into the question ("fill if empty, otherwise skip") are not followed at this size. State the decision as a |
| plain question with described options. |
| * Picking one record out of a long JSON array by position is the least accurate input shape (0.51 with 64 records against |
| 0.70 with one). Address records by key, or let the helper write the index into the array (0.62). |
| * Full label sets cost accuracy against 10 sampled options: CLINC 151-way 0.88 against 0.98; DBpedia level 2 with 70 labels |
| is the least calibrated case (ECE 0.14). |
| * Questions packed into one row (`independent=False`) see the earlier question texts, and reversing their order changes up to |
| 12% of answers. The default path scores each question alone. |
| * The v10 browser results are on 22 click-only MiniWoB++ tasks: small synthetic pages with the elements listed as text. Typing, |
| scrolling and real websites were not tested. OpenJev accuracy is 0.8 points lower than v8. |
| * Abstention: a catch-all option ("none of the above", "other", "unsure") is chosen when nothing on offer fits, not when the |
| exact fine-grained label is merely absent. Wordings far from the training data remain the main risk. |
| * One in-task dataset, `tweet_hate` (SemEval-2019 HatEval), stays near chance on its test split, whose collection and label |
| definition differ from the training split. The number is reported as measured. |
|
|
| ## Changelog |
|
|
| | version | what changed | |
| |---|---| |
| | **v10** (2026-09-19, these weights) | v8 plus 384 steps of calibration-aware RL on live browser tasks and exact games. Measured on the same rows: live browser click tasks 83% to 93% sampled success (held-out tasks 73% to 92%), stated beliefs about action outcomes 0.47 to 0.22 nats above the exact law, Mind2Web +1.5 points, general accuracy and Bespoke's public suite unchanged, OpenJev −0.8 points. | |
| | v9 | terse-bucket routing messages and labelled shell commands in the data; described in the GitHub README, but the Hub weights stayed v8, so v10 does not contain it | |
| | v8 (Hub tag `v8`) | isolated Score levels, teacher-written custom questions with a generic option next to a catch-all, the cacheable schema-first layout | |
| | v6 to v7 | the input shapes Jev accepts: described options, up to 255 options, JSON states with path references, long inputs | |
| | v4 to v5 | next-action choice from agent trajectories and game states; the proper abstention fix | |
| | v1 to v3 | the one-pass readout on the public decision mixture, one fitted temperature | |
|
|
| The full entries, with the browser and game recordings and the same-rows comparison against v8, are in |
| [docs/CHANGELOG.md](https://github.com/Mapika/decider/blob/main/docs/CHANGELOG.md) of the GitHub repository; |
| [docs/HISTORY.md](https://github.com/Mapika/decider/blob/main/docs/HISTORY.md) has how each stage was trained and measured. |
|
|
| ## Reproduction |
|
|
| Code, data registry, training and evaluation scripts, the RL recipe and the per-version history: |
| https://github.com/Mapika/decider. Each release is staged with `scripts/stage_release.py` and uploaded with |
| `scripts/upload_hf.py`; the previous weights are kept under the tag `v8` in this repository. |
|
|
