Expand Pivot model card, benchmarks, CPU tools and charts

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+evaluation/2026-09-24/performance_overview.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -12,28 +12,85 @@ tags:
 
 # Pivot
 
-Pivot is a set-scored decision encoder trained with the reviewed DSBT method.
+### Fast closed-set decisions from context and candidate actions
 
-Input is one context plus 2-16 semantic candidates. Output is a closed-set probability vector.
-The training recipe uses two CE warmup epochs followed by live multiclass Brier + PCGrad.
-Probabilities are computed as `softmax(logits.float())`.
+Pivot is a **357.6M-parameter** bidirectional decision encoder. Give it a context and two or more candidate answers; it scores the set in one forward pass and returns the chosen answer, its position, and a probability for every candidate. It does not generate free-form text. The full FP32 checkpoint, tokenizer and Transformers custom runtime are stored in this model repository.
 
-The package vendors the exact bidirectional LFM2 runtime used for Pivot and is verified in a
-fresh offline Python process before upload.
+| Measured result | Pivot |
+|---|---:|
+| JevBench v1.4.1 public accuracy | **46.32%** (107 / 231) |
+| NVIDIA H200 warm single-decision p50 / p95 | **15.8 / 19.9 ms** |
+| NVIDIA H200 throughput, batch 32 | **545.3 decisions/s** |
+| 4-thread Xeon CPU warm single-decision p50 / p95 | **797.6 / 1,089.0 ms** |
+| 4-thread Xeon CPU throughput, batch 4 | **3.77 decisions/s** |
 
-## Load
+These are measurements on the pinned checkpoint, in FP32, including tokenization and scoring. GPU and CPU throughput used **different batch sizes**. The 46.32% figure is public-task accuracy; the **official JevBench v1.4 composite score has not been measured** because the sealed/judge portion and cost input are unavailable. [Protocol and limitations](docs/BENCHMARK.md) · [Full measurement details](docs/PERFORMANCE.md)
+
+![Public accuracy and warm local speed](evaluation/2026-09-24/performance_overview.png)
+
+## Start with one decision
+
+Install a suitable PyTorch build and the runtime packages:
+
+```bash
+python -m pip install "transformers==5.17.0" "safetensors==0.8.0"
+```
 
 ```python
+import torch
 from transformers import AutoModel, AutoTokenizer
 
-tok = AutoTokenizer.from_pretrained("Q1z/Pivot", trust_remote_code=True)
-model = AutoModel.from_pretrained("Q1z/Pivot", trust_remote_code=True)
-result = model.choose(tok, "Customer says the invoice is wrong.", [
+tokenizer = AutoTokenizer.from_pretrained("Q1z/Pivot", trust_remote_code=True)
+model = AutoModel.from_pretrained(
+    "Q1z/Pivot", trust_remote_code=True, dtype=torch.float32
+).eval()
+
+context = "CONTEXT:\nA customer disputes an invoice and asks for a correction."
+options = [
     "route to billing support",
     "route to technical support",
     "route to sales",
-])
-print(result)
+]
+decision = model.choose(tokenizer, context, options)
+print(decision)  # {"choice": ..., "index": ..., "probs": [...]}
 ```
 
-Training/evaluation provenance is under `training/`.
+The serving configuration now defaults to **512 context tokens and 128 option tokens**, matching the published public evaluation. It does not change the checkpoint weights. Run on CUDA with `model.to("cuda")` if your PyTorch build supports it. Review the repository's custom model code before enabling `trust_remote_code=True`.
+
+## Three ways to use Pivot
+
+| Method | Input | Output |
+|---|---|---|
+| `model.choose(tokenizer, context, options)` | Context and ordered answer strings | Chosen answer, index, probability vector |
+| `model.decide_native(tokenizer, context, candidates)` | Candidate IDs, semantic text, optional abstain action | Selected ID, relative confidence, per-candidate probabilities |
+| `model.decide(tokenizer, state, questions)` | Several typed choice / yes-no / score questions | Typed decision response |
+
+For repeated decisions using the same options, `encode_candidates`, `encode_context`, and `choose_cached` reuse candidate representations. This can avoid repeated candidate encoding; the measured throughput above uses the **uncached** path. Examples: [basic](examples/quickstart.py), [structured decisions](examples/native_decision.py), [cached candidates](examples/cached_candidates.py), and [full inference guide](docs/INFERENCE.md).
+
+Pivot's probabilities are **relative to the supplied candidate set**. Give each candidate a clear, distinct meaning. An abstain route must be an explicit candidate; a high relative probability alone does not establish real-world correctness or safety.
+
+## Evaluated performance
+
+The official JevBench v1.4.1 **public tasks** were scored with the official per-task scorer at commit `24b9b5c1609a7a9e8fa14f49e5985a836c9dc842`. The exact evaluated model commit was `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578`.
+
+| Public tier | Correct / tasks | Accuracy | ECE, 10 bins |
+|---|---:|---:|---:|
+| Original | 27 / 72 | 37.50% | 0.525 |
+| Easy | 39 / 48 | 81.25% | 0.114 |
+| Hard | 41 / 111 | 36.94% | 0.379 |
+| **All public tasks** | **107 / 231** | **46.32%** | — |
+
+The frozen evaluation format is a `CONTEXT:` prefix, task rubric descriptions where provided (otherwise humanized labels) in official label order, right truncation, 512 context tokens and 128 option tokens. Public accuracy is not an official leaderboard score. [Reproduce public results](benchmarks/README.md) or [run the CPU speed entry point](cpu-speed/README.md). The [CPU notebook](notebooks/Pivot_CPU_Speed.ipynb) provides an interactive alternative.
+
+## Repository guide
+
+- [Inference methods and examples](docs/INFERENCE.md)
+- [Serving interfaces and request formats](serving/README.md)
+- [JevBench protocol, revision, and score scope](docs/BENCHMARK.md)
+- [Results, hardware, charts, and caveats](docs/PERFORMANCE.md)
+- [Reproducible public benchmark script](benchmarks/jevbench_public.py)
+- [Standalone CPU speed script](cpu-speed/benchmark.py)
+- [Pinned result files and chart](evaluation/2026-09-24/README.md)
+- [What changed in this update](docs/UPDATE_NOTES.md)
+
+The files `model.safetensors`, `modeling_pivot.py`, `modeling_lfm2_bidirectional.py`, `pivot_model.py` and `pivot_infer.py` contain the checkpoint and model runtime. [Package manifest](manifest.json) records file hashes. The original checkpoint and evaluated metrics remain anchored to the exact revision above.

benchmarks/README.md

@@ -0,0 +1,14 @@
+# Reproduce Pivot's public results
+
+Run the standalone `jevbench_public.py` script from this repository. It checks out the official JevBench v1.4.1 source at commit `24b9b5c1609a7a9e8fa14f49e5985a836c9dc842`, verifies a clean checkout, and uses its public tasks and official per-task scorer.
+
+Install Python, Git, and a PyTorch build for your hardware. The requirements file records the versions used for the reported run. On Windows PowerShell:
+
+```powershell
+py -m pip install -r benchmarks/requirements.txt
+py benchmarks/jevbench_public.py --device cpu --threads 4 --batch-size 4 --output pivot_cpu_results.json
+```
+
+On a CUDA machine use `--device cuda --batch-size 32`. If you only need timings, add `--mode speed`. For public task accuracy only, add `--mode accuracy`. The model is pinned to commit `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578` unless you deliberately provide another full SHA via `--revision`. Allow time and disk space for the approximately 1.43 GB checkpoint.
+
+The script uses the same candidate wording, input rendering, token limits, FP32 scoring, warmup count, and timing protocol as the published run. Timings still vary with hardware, software and background load. Only public JevBench tasks are available here; `official_jevbench_score` is always null. The full official score requires organizer-held sealed and judge tasks and cost measurement. See [the benchmark protocol](../docs/BENCHMARK.md).

benchmarks/jevbench_public.py

@@ -0,0 +1,296 @@
+"""Standalone Pivot evaluation on the pinned official JevBench public tasks.
+
+The sealed benchmark and cost component are unavailable here. This script only
+reports public accuracy and local warm FP32 speed; it never fabricates a score.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import os
+import platform
+import statistics
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+MODEL_ID = "Q1z/Pivot"
+MODEL_REVISION = "14bf8c26bf344ebdf88e22a4b6152dc5f75f3578"
+JEVBENCH_REVISION = "24b9b5c1609a7a9e8fa14f49e5985a836c9dc842"  # v1.4.1
+CONTRACT = {
+    "context_arm": "context_header",
+    "option_arm": "original",
+    "context_tokens": 512,
+    "option_tokens": 128,
+    "truncation": "right",
+}
+
+
+def _run(*args: str) -> str:
+    return subprocess.check_output(args, text=True).strip()
+
+
+def checkout_jevbench(cache: Path) -> Path:
+    repo = cache / "jevbench"
+    created = not repo.exists()
+    if created:
+        repo.parent.mkdir(parents=True, exist_ok=True)
+        subprocess.run(
+            ["git", "clone", "--filter=blob:none", "--no-checkout",
+             "https://github.com/fstandhartinger/jevbench.git", str(repo)],
+            check=True,
+        )
+    if not (repo / ".git").exists():
+        raise RuntimeError(f"Expected a clean git checkout at {repo}")
+    if not created and _run("git", "-C", str(repo), "status", "--porcelain", "--untracked-files=all"):
+        raise RuntimeError("JevBench checkout has local changes; choose a clean cache")
+    if _run("git", "-C", str(repo), "rev-parse", "HEAD") != JEVBENCH_REVISION:
+        subprocess.run(["git", "-C", str(repo), "fetch", "--depth", "1", "origin", JEVBENCH_REVISION], check=True)
+        subprocess.run(["git", "-C", str(repo), "checkout", "--detach", JEVBENCH_REVISION], check=True)
+    required = [repo / "jevbench" / "tasks.py"] + [
+        repo / "datasets" / "public" / f"{tier}.jsonl"
+        for tier in ("original", "easy", "hard")
+    ]
+    if (_run("git", "-C", str(repo), "rev-parse", "HEAD") != JEVBENCH_REVISION
+            or _run("git", "-C", str(repo), "status", "--porcelain", "--untracked-files=all")
+            or not all(path.is_file() for path in required)):
+        raise RuntimeError("Official JevBench revision or public files differ")
+    for name in ("jevbench", "jevbench.tasks", "jevbench.scoring", "jevbench.metrics"):
+        module = sys.modules.get(name)
+        if module is not None and Path(getattr(module, "__file__", "") or "").resolve().is_relative_to(repo.resolve()) is False:
+            raise RuntimeError(f"{name} was imported from a different checkout")
+    sys.path.insert(0, str(repo))
+    return repo
+
+
+def _clean(value: object) -> str:
+    return " ".join(str(value).split())
+
+
+def _humanize(label: object) -> str:
+    return _clean(str(label).replace("_", " ").replace("-", " "))
+
+
+def _render_state(value: object) -> str:
+    if isinstance(value, str):
+        return value.strip()
+    if value is None:
+        return "null"
+    if isinstance(value, (bool, int, float)):
+        return str(value)
+    if isinstance(value, list):
+        return "\n".join(f"- {_render_state(item)}" for item in value)
+    if isinstance(value, dict):
+        parts = []
+        for key, child in value.items():
+            rendered = _render_state(child)
+            if rendered:
+                parts.append(f"{_humanize(key).title()}: {rendered}")
+        return "\n\n".join(parts)
+    return json.dumps(value, ensure_ascii=False, sort_keys=True)
+
+
+def prepare_rows(tasks: list) -> tuple[list[dict], list]:
+    """Same rendering and exclusions as the evaluated Pivot_test public run."""
+    rows, kept = [], []
+    for task in tasks:
+        if task.expected is None or task.provenance.get("exclude_reason"):
+            continue
+        labels = [str(item) for item in task.labels]
+        gold = str(task.expected)
+        if gold not in labels:
+            continue
+        criteria = task.question.get("criteria")
+        kind = str(task.question["type"])
+        options = []
+        for label in labels:
+            description = None
+            if isinstance(criteria, dict):
+                key = {"no": "false", "yes": "true"}.get(label.strip().lower(), label)
+                description = criteria.get(key)
+                if description is None:
+                    description = criteria.get(label)
+            elif isinstance(criteria, list) and kind == "score":
+                try:
+                    description = criteria[int(label)]
+                except (ValueError, IndexError, TypeError):
+                    pass
+            text = _clean(description) if description is not None else _humanize(label)
+            if not text:
+                raise ValueError(f"Empty option for {label!r}")
+            options.append(text)
+        focus = str(task.question.get("instructions") or "").strip()
+        evidence = _render_state(task.state)
+        body = f"{focus}\n\n{evidence}".strip() if focus else evidence
+        rows.append({"context": "CONTEXT:\n" + body, "options": options})
+        kept.append(task)
+    return rows, kept
+
+
+def score_rows(model, tokenizer, rows: list[dict], device: str, batch_size: int) -> list[list[float]]:
+    """Encode each context and option in FP32, then score the original set."""
+    import torch
+
+    output: list[list[float]] = []
+    previous = tokenizer.truncation_side
+    tokenizer.truncation_side = "right"
+    try:
+        with torch.inference_mode():
+            for start in range(0, len(rows), batch_size):
+                batch = rows[start:start + batch_size]
+                ctx = tokenizer([row["context"] for row in batch], padding=True,
+                                truncation=True, max_length=512, return_tensors="pt")
+                opt = tokenizer([text for row in batch for text in row["options"]],
+                                padding=True, truncation=True, max_length=128,
+                                return_tensors="pt")
+                dev = torch.device(device)
+                h_ctx = model.network.encode(ctx["input_ids"].to(dev), ctx["attention_mask"].to(dev))
+                h_opt = model.network.encode(opt["input_ids"].to(dev), opt["attention_mask"].to(dev))
+                width = max(len(row["options"]) for row in batch)
+                pieces, masks, offset = [], [], 0
+                for row in batch:
+                    count = len(row["options"])
+                    part = h_opt[offset:offset + count]
+                    offset += count
+                    if count < width:
+                        part = torch.cat([part, part.new_zeros(width - count, part.shape[-1])])
+                    pieces.append(part)
+                    mask = torch.zeros(width, dtype=torch.bool, device=dev)
+                    mask[:count] = True
+                    masks.append(mask)
+                result = model.network.score_preencoded(h_ctx, torch.stack(pieces), torch.stack(masks))
+                for idx, row in enumerate(batch):
+                    probabilities = result.probs[idx, :len(row["options"])].float().cpu().tolist()
+                    if (not all(math.isfinite(x) for x in probabilities)
+                            or abs(sum(probabilities) - 1.0) > 1e-5):
+                        raise ValueError(f"Invalid probabilities at row {start + idx}")
+                    output.append(probabilities)
+    finally:
+        tokenizer.truncation_side = previous
+    return output
+
+
+def speed(model, tokenizer, rows: list[dict], device: str, batch_size: int) -> dict:
+    """Mirror the original run: 5+2 warmups; 32 singles; 3x64 bulk."""
+    import torch
+
+    def timed(subset: list[dict], size: int) -> float:
+        if device == "cuda":
+            torch.cuda.synchronize()
+        start = time.perf_counter()
+        score_rows(model, tokenizer, subset, device, size)
+        if device == "cuda":
+            torch.cuda.synchronize()
+        return time.perf_counter() - start
+
+    samples = rows[:32]
+    for row in samples[:5]:
+        timed([row], 1)
+    latencies = [timed([row], 1) for row in samples]
+    bulk = rows[:64]
+    for _ in range(2):
+        timed(bulk, batch_size)
+    repeats = [timed(bulk, batch_size) for _ in range(3)]
+    return {
+        "measurement": "warm local FP32; includes tokenization, inference and scoring",
+        "hardware": torch.cuda.get_device_name(0) if device == "cuda" else platform.processor(),
+        "threads": torch.get_num_threads() if device == "cpu" else None,
+        "single_decision": {
+            "n": len(latencies), "batch_size": 1,
+            "p50_ms": 1000 * statistics.median(latencies),
+            "p95_ms": 1000 * sorted(latencies)[math.ceil(.95 * len(latencies)) - 1],
+            "mean_ms": 1000 * statistics.mean(latencies),
+        },
+        "throughput": {
+            "batch_size": batch_size, "decisions_per_repeat": len(bulk),
+            "repeats": 3, "repeat_seconds": repeats,
+            "decisions_per_second": len(bulk) / statistics.median(repeats),
+        },
+    }
+
+
+def main(argv: list[str] | None = None) -> dict:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--mode", choices=("speed", "accuracy", "all"), default="all")
+    parser.add_argument("--device", choices=("auto", "cpu", "cuda"), default="auto")
+    parser.add_argument("--batch-size", type=int, default=None)
+    parser.add_argument("--threads", type=int, default=0)
+    parser.add_argument("--revision", default=MODEL_REVISION)
+    parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "pivot_jevbench")
+    parser.add_argument("--output", type=Path, default=Path("pivot_public_results.json"))
+    args = parser.parse_args(argv)
+    if args.batch_size is not None and args.batch_size < 1 or args.threads < 0:
+        parser.error("batch-size must be positive and threads nonnegative")
+    if len(args.revision) != 40 or any(ch not in "0123456789abcdef" for ch in args.revision):
+        parser.error("revision must be a full lowercase commit SHA")
+
+    import torch
+    from transformers import AutoModel, AutoTokenizer
+
+    if args.threads:
+        torch.set_num_threads(args.threads)
+    device = ("cuda" if torch.cuda.is_available() else "cpu") if args.device == "auto" else args.device
+    if device == "cuda" and not torch.cuda.is_available():
+        parser.error("CUDA is unavailable")
+    batch_size = args.batch_size or (32 if device == "cuda" else 4)
+    checkout = checkout_jevbench(args.cache)
+    from jevbench.tasks import load_jsonl
+    from jevbench.scoring import score_task
+    from jevbench.metrics import ece_top_label
+
+    torch.backends.cuda.matmul.allow_tf32 = False
+    torch.backends.cudnn.allow_tf32 = False
+    torch.manual_seed(42)
+    print("Loading checkpoint and tokenizer...", flush=True)
+    model = AutoModel.from_pretrained(MODEL_ID, revision=args.revision,
+                                      trust_remote_code=True, dtype=torch.float32)
+    model.to(device=torch.device(device), dtype=torch.float32).eval()
+    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, revision=args.revision)
+    root = checkout / "datasets" / "public"
+    results = {
+        "model": MODEL_ID, "model_revision": args.revision,
+        "benchmark_release": "v1.4.1", "benchmark_commit": JEVBENCH_REVISION,
+        "scope": "public tasks only", "official_jevbench_score": None,
+        "contract": CONTRACT, "device": device,
+    }
+    if args.mode in ("accuracy", "all"):
+        cohorts = {}
+        for name in ("original", "easy", "hard"):
+            tasks = load_jsonl(str(root / f"{name}.jsonl"))
+            rows, kept = prepare_rows(tasks)
+            scored = score_rows(model, tokenizer, rows, device, batch_size)
+            official = [score_task(dict(zip(map(str, task.labels), probs)), task)
+                        for task, probs in zip(kept, scored)]
+            valid = [item for item in official if item["valid"]]
+            correct = sum(item["correct"] is True for item in official)
+            cohorts[name] = {
+                "n": len(official), "correct": correct,
+                "invalid": len(official) - len(valid),
+                "accuracy": correct / len(official) if official else None,
+                "ece_10": ece_top_label([(max(item["probs"].values()), item["correct"])
+                                         for item in valid])["ece"] if valid else None,
+            }
+            print(f"{name}: {correct}/{len(official)}", flush=True)
+        total = sum(item["n"] for item in cohorts.values())
+        results.update(cohorts=cohorts, public_n=total,
+                       public_correct=sum(item["correct"] for item in cohorts.values()),
+                       public_accuracy=sum(item["correct"] for item in cohorts.values()) / total)
+    if args.mode in ("speed", "all"):
+        tasks = load_jsonl(str(root / "original.jsonl"))
+        rows, _ = prepare_rows(tasks)
+        results["speed"] = speed(model, tokenizer, rows, device, batch_size)
+        print(json.dumps({"p50_ms": results["speed"]["single_decision"]["p50_ms"],
+                          "p95_ms": results["speed"]["single_decision"]["p95_ms"],
+                          "decisions_per_second": results["speed"]["throughput"]["decisions_per_second"]},
+                         indent=2), flush=True)
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(results, indent=2, allow_nan=False) + "\n", encoding="utf-8")
+    print(f"Saved {args.output.resolve()}", flush=True)
+    return results
+
+
+if __name__ == "__main__":
+    main()

benchmarks/requirements.txt

@@ -0,0 +1,5 @@
+torch==2.8.0
+transformers==5.17.0
+safetensors==0.8.0
+numpy==1.26.4
+huggingface_hub>=0.26.0

config.json

@@ -73,7 +73,7 @@
     },
     "serving": {
       "max_context_tokens": 512,
-      "max_option_tokens": 64
+      "max_option_tokens": 128
     }
   },
   "encoder_config": {
@@ -162,4 +162,4 @@
     "method": "CE warmup -> live multiclass Brier + PCGrad",
     "probability_contract": "softmax(logits.float())"
   }
-}
+}

cpu-speed/README.md

@@ -0,0 +1,14 @@
+# Pivot CPU speed
+
+This folder is a CPU-only entry point to the standalone public JevBench runner. It downloads the exact measured Pivot checkpoint and the pinned official public JevBench source; no private account or token is needed for the public files.
+
+From the model repository root on Windows PowerShell:
+
+```powershell
+py -m pip install -r benchmarks/requirements.txt
+py cpu-speed/benchmark.py --threads 4 --batch-size 4 --output pivot_cpu_results.json
+```
+
+The run measures warm FP32 latency for 32 individual decisions and throughput over three sets of 64 decisions. It includes tokenization, inference, and scoring. It reports p50, p95, mean, decisions per second, thread count, hardware, pinned revisions, and the contract. To evaluate public accuracy too, run `py benchmarks/jevbench_public.py --device cpu --threads 4 --mode all`.
+
+The published 4-thread Xeon result is p50 **797.6 ms**, p95 **1,089.0 ms**, and **3.77 decisions/s** at batch 4. Your CPU result will differ. GPU throughput used batch 32, so these throughput values are not a matched batch comparison. See [archived measurements](../evaluation/2026-09-24/).

cpu-speed/benchmark.py

@@ -0,0 +1,9 @@
+"""CPU-only wrapper for the pinned public JevBench speed protocol."""
+from pathlib import Path
+import sys
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "benchmarks"))
+from jevbench_public import main
+
+if __name__ == "__main__":
+    main([*sys.argv[1:], "--device", "cpu", "--mode", "speed"])

docs/BENCHMARK.md

@@ -0,0 +1,26 @@
+# JevBench public evaluation
+
+## Exact identity
+
+| Item | Pinned value |
+|---|---|
+| Model | `Q1z/Pivot` |
+| Evaluated model commit | `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578` |
+| Official JevBench release | v1.4.1 |
+| JevBench commit | `24b9b5c1609a7a9e8fa14f49e5985a836c9dc842` |
+| Tasks | Public original, easy, hard; 72 + 48 + 111 |
+| Precision | FP32 |
+| Rendering | `CONTEXT:` prefix, rubric text where present, official label order |
+| Limits | 512 context and 128 option tokens, right truncation |
+
+Pivot encodes every context and candidate, scores the candidate set, then passes probabilities indexed by each official task's labels to JevBench's `score_task`. The candidate text is the task rubric description when available, otherwise a humanized label. The published public accuracy is 107 / 231 = **46.32%**. Use [the standalone script](../benchmarks/jevbench_public.py) to repeat the calculation. It checks out the pinned official source, refuses a modified benchmark checkout, and writes results including both revisions and the frozen input contract.
+
+## Meaning of the score
+
+The **official JevBench v1.4 composite score is unavailable**. Public tasks do not include the organizer-held sealed and judge components or an official cost input. The 46.32% figure is a measured public accuracy, not a held-out score, official composite, rank, or cost-adjusted value. No such missing numbers are estimated in the result files.
+
+The scripts also measure **local** warm single-decision latency and batched throughput on the public original tasks. These timings include tokenization, model inference and scoring, and depend on CPU/GPU, batch size, software and load. They are not the benchmark's adjusted speed/cost metric.
+
+The runtime serving option limit in this update is now 128, matching the frozen benchmark. This is a default inference setting change, while checkpoint weights and the original evaluated commit remain intact. The original measurements cannot automatically predict the speed of every serving input.
+
+See [results and charts](../evaluation/2026-09-24/README.md) and [reproduction instructions](../benchmarks/README.md).

docs/INFERENCE.md

@@ -0,0 +1,72 @@
+# Using Pivot
+
+Pivot evaluates a **supplied set of options** for each context. It is a bidirectional encoder with a set scorer, so its answer is an index into your options, not generated text. Use 2–16 distinct candidate meanings for the structured native interface. A single plain `choose()` call accepts two or more options; large candidate lists cost more memory and time.
+
+## Load once
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+repo = "Q1z/Pivot"
+tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
+model = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.float32).eval()
+```
+
+This repository ships custom Transformers code. Inspect it before loading with `trust_remote_code=True` in a production environment. A CPU-only PyTorch build works. For CUDA, call `model.to("cuda")`. Keep the tokenizer and model loaded across requests.
+
+## Plain choice
+
+```python
+context = "CONTEXT:\nA user requests a refund for an incorrect charge."
+options = ["route to billing", "route to technical support", "route to sales"]
+result = model.choose(tokenizer, context, options)
+assert result["choice"] == options[result["index"]]
+assert len(result["probs"]) == len(options)
+```
+
+The probability vector follows the original option order and sums to approximately one. Changing the option wording or adding an option changes the set being scored. The public benchmark uses the `CONTEXT:` prefix, a 512-token context limit and a 128-token option limit.
+
+## Native structured choice
+
+`decide_native()` accepts a nonempty context and 2–16 objects with unique `id` and distinct `text` values. Optional `kind: "abstain"` is allowed on at most one candidate. Machine IDs stay outside the option encoder.
+
+```python
+result = model.decide_native(
+    tokenizer,
+    "A customer disputes an invoice.",
+    [
+        {"id": "billing", "text": "route to billing"},
+        {"id": "technical", "text": "route to technical support"},
+        {"id": "unknown", "text": "none of the routes fits", "kind": "abstain"},
+    ],
+)
+print(result["selected"]["id"], result["prob_vector"])
+```
+
+The abstain option competes with the other supplied options; it is not an independently calibrated refusal threshold.
+
+## Several typed decisions
+
+```python
+result = model.decide(tokenizer, "Customer invoice disputed", [
+    {"id": "route", "primitive": "choice", "options": ["billing", "tech", "sales"]},
+    {"id": "approve", "primitive": "noul", "options": ["true", "false"]},
+    {"id": "severity", "primitive": "score", "options": ["0", "1", "2", "3"]},
+])
+```
+
+The typed method scores each question in turn. The examples under [`serving/`](../serving/) show request and response shapes.
+
+## Cache repeated candidates
+
+For repeated contexts against an unchanged option set:
+
+```python
+options = ["billing support", "technical support", "sales"]
+candidate_vectors = model.encode_candidates(tokenizer, options)
+context_vector = model.encode_context(tokenizer, "Invoice is incorrect")
+result = model.choose_cached(context_vector, candidate_vectors, options)
+```
+
+Cache embeddings only with the same model, tokenizer, precision and option text. This path can avoid repeated option encoding. Published speed measurements instead use the uncached end-to-end path. [Runnable examples](../examples/cached_candidates.py).

docs/PERFORMANCE.md

@@ -0,0 +1,32 @@
+# Pivot performance
+
+All figures here refer to checkpoint `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578`, public JevBench v1.4.1 commit `24b9b5c1609a7a9e8fa14f49e5985a836c9dc842`, FP32 and the same frozen 512/128-token input contract.
+
+## Accuracy on public tasks
+
+| Tier | Correct | Tasks | Accuracy | Top-label ECE, 10 bins |
+|---|---:|---:|---:|---:|
+| Original | 27 | 72 | 37.50% | 0.5247 |
+| Easy | 39 | 48 | 81.25% | 0.1139 |
+| Hard | 41 | 111 | 36.94% | 0.3792 |
+| **Total** | **107** | **231** | **46.32%** | — |
+
+![Public-tier accuracy](../evaluation/2026-09-24/accuracy.png)
+
+The official JevBench composite score is **unavailable** because the sealed and judge tasks and official cost input were not measured.
+
+## Local speed
+
+| Warm local FP32 measure | H200 GPU | Xeon CPU, 4 threads |
+|---|---:|---:|
+| Single decision p50, 32 measured | 15.7668 ms | 797.5558 ms |
+| Single decision p95, 32 measured | 19.8708 ms | 1089.0266 ms |
+| Single decision mean | 16.1614 ms | 770.7159 ms |
+| Batch size for throughput | 32 | 4 |
+| Throughput, median of 3 × 64 decisions | 545.2833 decisions/s | 3.7708 decisions/s |
+
+![Warm local latency and throughput](../evaluation/2026-09-24/latency_throughput.png)
+
+The CPU p50 single-decision time is **50.6×** the H200 p50 for these two machines. CPU and GPU batch throughput used different batch sizes and should not be read as a same-batch comparison. The timings cover tokenizer + inference + scoring with 5 warmup singles and 2 warmup bulk passes. Reproduce them on your own hardware using the [CPU script](../cpu-speed/README.md) or [full public runner](../benchmarks/README.md).
+
+Source: [full structured summary](../evaluation/2026-09-24/performance.json), [original public benchmark result](../evaluation/2026-09-24/jevbench_public.json), and [original CPU measurement](../evaluation/2026-09-24/cpu_speed.json).

docs/UPDATE_NOTES.md

@@ -0,0 +1,9 @@
+# Model repository update — 2026-09-24
+
+This upload replaces the short model card with a full model overview, measured public accuracy, CPU/GPU timing, usage methods and reproducibility links. It adds inference examples, a serving guide, a standalone public JevBench v1.4.1 runner, a CPU-specific entry point, a CPU notebook, two focused charts, a combined chart, and the exact results in JSON.
+
+The `config.json` serving option limit changes from **64 to 128** tokens to align the default API with the measured evaluation input limit. The checkpoint and runtime source files are not modified. `manifest.json` is regenerated for every changed or new repository file; existing weight and runtime hashes are preserved.
+
+The reported public accuracy is **107 / 231 = 46.32%** at the pinned model revision `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578`. The official JevBench composite score is unavailable because the sealed/judge portion and cost input are not included. Local CPU and GPU speeds are measured at distinct batch sizes and are identified as such.
+
+Upload the contents of this package to the **root** of `Q1z/Pivot`, replacing the names it contains. No delete operation is required. Model weight files, tokenizer files, and pre-existing serving request/response examples stay in the Hub repository.

evaluation/2026-09-24/README.md

@@ -0,0 +1,14 @@
+# Evaluated checkpoint and measured results
+
+Checkpoint: `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578`. Official JevBench public tasks: v1.4.1, commit `24b9b5c1609a7a9e8fa14f49e5985a836c9dc842`.
+
+- [`performance.json`](performance.json): compact accuracy, speed, hardware, revisions, and limitations.
+- [`jevbench_public.json`](jevbench_public.json): original GPU public JevBench result with official per-task scoring.
+- [`cpu_speed.json`](cpu_speed.json): original four-thread CPU timing result.
+- [`performance_overview.png`](performance_overview.png): combined chart.
+- [`accuracy.png`](accuracy.png): public task accuracy.
+- [`latency_throughput.png`](latency_throughput.png): GPU and CPU speed.
+
+Public accuracy: **107 / 231 = 46.32%**. Warm single-decision p50: **15.8 ms** on H200, **797.6 ms** on Xeon CPU. GPU throughput uses batch 32, CPU throughput batch 4. The official sealed JevBench composite score and cost metric were not measured.
+
+This update changes the default serving option limit from 64 to 128, while the checkpoint and archived measurements remain anchored to the prior exact commit. See [the full protocol](../../docs/BENCHMARK.md).

evaluation/2026-09-24/cpu_speed.json

@@ -0,0 +1,39 @@
+{
+  "scope": "public original tasks; warm local inference including tokenization and scoring",
+  "device": "cpu",
+  "hardware": "x86_64",
+  "dtype": "float32",
+  "single_decision": {
+    "batch_size": 1,
+    "n": 32,
+    "p50_ms": 797.5558204999516,
+    "p95_ms": 1089.0265890000137,
+    "mean_ms": 770.7159108749977
+  },
+  "throughput": {
+    "batch_size": 4,
+    "decisions_per_repeat": 64,
+    "repeats": 3,
+    "decisions_per_second": 3.7708397043400965,
+    "repeat_seconds": [
+      16.58172093600001,
+      17.219981521999898,
+      16.97234701499997
+    ]
+  },
+  "note": "Local warm FP32 timing; not official adjusted JevBench Speed. No cost estimate without a specified hardware price.",
+  "cpu_model": "Intel(R) Xeon(R) CPU @ 2.60GHz",
+  "cpu_threads": 4,
+  "logical_cpu_count": 4,
+  "model_revision": "14bf8c26bf344ebdf88e22a4b6152dc5f75f3578",
+  "benchmark_commit": "24b9b5c1609a7a9e8fa14f49e5985a836c9dc842",
+  "benchmark_release": "v1.4.1",
+  "contract": {
+    "context_arm": "context_header",
+    "option_arm": "original",
+    "context_tokens": 512,
+    "option_tokens": 128,
+    "truncation": "right"
+  },
+  "test_scope": "speed only; no accuracy or official composite score computed"
+}

evaluation/2026-09-24/jevbench_public.json

@@ -0,0 +1,585 @@
+{
+  "benchmark_commit": "24b9b5c1609a7a9e8fa14f49e5985a836c9dc842",
+  "benchmark_release": "v1.4.1",
+  "scope": "public_subset_only",
+  "official_held_out": null,
+  "official_jevbench_score": null,
+  "public_accuracy": 0.46320346320346323,
+  "public_n": 231,
+  "speed": {
+    "scope": "public original tasks; warm local inference including tokenization and scoring",
+    "device": "cuda",
+    "hardware": "NVIDIA H200",
+    "dtype": "float32",
+    "single_decision": {
+      "batch_size": 1,
+      "n": 32,
+      "p50_ms": 15.766771499897914,
+      "p95_ms": 19.87079100013034,
+      "mean_ms": 16.16144687499599
+    },
+    "throughput": {
+      "batch_size": 32,
+      "decisions_per_repeat": 64,
+      "repeats": 3,
+      "decisions_per_second": 545.2832753857399,
+      "repeat_seconds": [
+        0.12318335300005856,
+        0.1173701870000059,
+        0.11606121599993457
+      ]
+    },
+    "note": "Local warm FP32 timing; not official adjusted JevBench Speed. No cost estimate without a specified hardware price."
+  },
+  "contract": {
+    "context_arm": "context_header",
+    "option_arm": "original",
+    "context_tokens": 512,
+    "option_tokens": 128,
+    "truncation": "right",
+    "selection_arm": "option_len:128",
+    "dev_sample_macro_accuracy": 0.7741666666666666,
+    "dev_sample_macro_nll": 0.9652628596545907,
+    "verified_as_scored": true,
+    "run_id": "f9f472314d223516818433b6f731ed3a05c999bd0be412134ad671d5ddc7808c"
+  },
+  "cohorts": {
+    "original": {
+      "n_tasks_file": 72,
+      "n_scored": 72,
+      "official_public": {
+        "n": 72,
+        "correct": 27,
+        "invalid": 0,
+        "accuracy": 0.375,
+        "ece_10": 0.5247104664643605
+      },
+      "metrics": {
+        "macro_accuracy": 0.375,
+        "macro_nll": 5.007958399361118,
+        "micro": {
+          "n": 72,
+          "accuracy": 0.375,
+          "brier_mean": 0.30594551421877425,
+          "brier_sum": 1.1185596241362858,
+          "ece_15": 0.5325825545522902,
+          "nll": 5.007958399361118,
+          "entropy": 0.23031783027727182,
+          "margin": 4.621087094148,
+          "predicted_positions": {
+            "0": 23,
+            "1": 19,
+            "3": 14,
+            "4": 2,
+            "2": 14
+          },
+          "gold_positions": {
+            "0": 24,
+            "1": 20,
+            "2": 12,
+            "3": 10,
+            "4": 4,
+            "5": 2
+          }
+        },
+        "by_family": {
+          "adequacy": {
+            "n": 12,
+            "accuracy": 0.3333333333333333,
+            "brier_mean": 0.6032912907957645,
+            "brier_sum": 1.206582581591529,
+            "ece_15": 0.674643059571584,
+            "nll": 2.811368828710537,
+            "entropy": 0.35683153857926314,
+            "margin": 2.7182071606318154,
+            "predicted_positions": {
+              "1": 8,
+              "0": 4
+            },
+            "gold_positions": {
+              "1": 6,
+              "0": 6
+            }
+          },
+          "extraction": {
+            "n": 12,
+            "accuracy": 0.16666666666666666,
+            "brier_mean": 0.33462631759767253,
+            "brier_sum": 1.3385052703906901,
+            "ece_15": 0.7197072158257167,
+            "nll": 5.611249398437437,
+            "entropy": 0.25773084664473744,
+            "margin": 3.4950687090555825,
+            "predicted_positions": {
+              "0": 9,
+              "1": 3
+            },
+            "gold_positions": {
+              "1": 2,
+              "3": 4,
+              "0": 4,
+              "2": 2
+            }
+          },
+          "intent": {
+            "n": 12,
+            "accuracy": 0.5,
+            "brier_mean": 0.193049684303942,
+            "brier_sum": 0.9652484215197101,
+            "ece_15": 0.49123724798361457,
+            "nll": 6.693062391390858,
+            "entropy": 0.16276790362346089,
+            "margin": 5.134465376536052,
+            "predicted_positions": {
+              "1": 4,
+              "3": 4,
+              "4": 2,
+              "0": 2
+            },
+            "gold_positions": {
+              "1": 2,
+              "2": 4,
+              "3": 2,
+              "0": 2,
+              "4": 2
+            }
+          },
+          "ordinal": {
+            "n": 12,
+            "accuracy": 0.25,
+            "brier_mean": 0.3469416665898746,
+            "brier_sum": 1.3877666663594983,
+            "ece_15": 0.7055613746245701,
+            "nll": 5.438651597684344,
+            "entropy": 0.1746427791074148,
+            "margin": 5.965866764386495,
+            "predicted_positions": {
+              "3": 10,
+              "2": 2
+            },
+            "gold_positions": {
+              "0": 4,
+              "1": 2,
+              "2": 4,
+              "3": 2
+            }
+          },
+          "policy": {
+            "n": 12,
+            "accuracy": 0.8333333333333334,
+            "brier_mean": 0.08333248779901603,
+            "brier_sum": 0.16666497559803206,
+            "ece_15": 0.16530502339204153,
+            "nll": 0.2814556436290131,
+            "entropy": 0.40680371489783257,
+            "margin": 1.9209110339482625,
+            "predicted_positions": {
+              "0": 8,
+              "1": 4
+            },
+            "gold_positions": {
+              "0": 6,
+              "1": 6
+            }
+          },
+          "routing": {
+            "n": 12,
+            "accuracy": 0.16666666666666666,
+            "brier_mean": 0.27443163822637595,
+            "brier_sum": 1.6465898293582557,
+            "ece_15": 0.8280483782291412,
+            "nll": 9.211962536314518,
+            "entropy": 0.023130198810922015,
+            "margin": 8.492003520329794,
+            "predicted_positions": {
+              "2": 12
+            },
+            "gold_positions": {
+              "0": 2,
+              "1": 2,
+              "2": 2,
+              "3": 2,
+              "4": 2,
+              "5": 2
+            }
+          }
+        }
+      }
+    },
+    "easy": {
+      "n_tasks_file": 48,
+      "n_scored": 48,
+      "official_public": {
+        "n": 48,
+        "correct": 39,
+        "invalid": 0,
+        "accuracy": 0.8125,
+        "ece_10": 0.1139311231672764
+      },
+      "metrics": {
+        "macro_accuracy": 0.8125,
+        "macro_nll": 1.0037392714069566,
+        "micro": {
+          "n": 48,
+          "accuracy": 0.8125,
+          "brier_mean": 0.08969436292922732,
+          "brier_sum": 0.3277746810853164,
+          "ece_15": 0.17484351620078087,
+          "nll": 1.0037392714069566,
+          "entropy": 0.27674493139652984,
+          "margin": 5.322023505965869,
+          "predicted_positions": {
+            "0": 13,
+            "1": 16,
+            "2": 9,
+            "3": 7,
+            "4": 3
+          },
+          "gold_positions": {
+            "0": 15,
+            "1": 12,
+            "2": 8,
+            "3": 8,
+            "4": 5
+          }
+        },
+        "by_family": {
+          "extraction": {
+            "n": 12,
+            "accuracy": 0.5,
+            "brier_mean": 0.20218808597619678,
+            "brier_sum": 0.8518348596262159,
+            "ece_15": 0.4715841561555862,
+            "nll": 3.1491165426466026,
+            "entropy": 0.32298662421478147,
+            "margin": 3.7243812878926597,
+            "predicted_positions": {
+              "1": 3,
+              "2": 3,
+              "3": 3,
+              "0": 3
+            },
+            "gold_positions": {
+              "1": 2,
+              "3": 4,
+              "2": 2,
+              "0": 3,
+              "4": 1
+            }
+          },
+          "fact": {
+            "n": 12,
+            "accuracy": 0.8333333333333334,
+            "brier_mean": 0.10789432132950423,
+            "brier_sum": 0.21578864265900846,
+            "ece_15": 0.21092924475669866,
+            "nll": 0.3680396559776326,
+            "entropy": 0.5416488102436233,
+            "margin": 1.0842649737993877,
+            "predicted_positions": {
+              "1": 8,
+              "0": 4
+            },
+            "gold_positions": {
+              "1": 6,
+              "0": 6
+            }
+          },
+          "intent": {
+            "n": 12,
+            "accuracy": 0.9166666666666666,
+            "brier_mean": 0.04868382763629129,
+            "brier_sum": 0.24341913818145647,
+            "ece_15": 0.18598496417204538,
+            "nll": 0.49580653492545496,
+            "entropy": 0.23149866629527152,
+            "margin": 5.446817874908447,
+            "predicted_positions": {
+              "0": 4,
+              "1": 3,
+              "2": 2,
+              "3": 2,
+              "4": 1
+            },
+            "gold_positions": {
+              "0": 4,
+              "1": 2,
+              "2": 2,
+              "3": 2,
+              "4": 2
+            }
+          },
+          "tool_selection": {
+            "n": 12,
+            "accuracy": 1.0,
+            "brier_mean": 1.1216774916947182e-05,
+            "brier_sum": 5.6083874584735916e-05,
+            "ece_15": 0.0019801706075668335,
+            "nll": 0.0019943520781363655,
+            "entropy": 0.010845624832442837,
+            "margin": 11.03262988726298,
+            "predicted_positions": {
+              "0": 2,
+              "1": 2,
+              "2": 4,
+              "3": 2,
+              "4": 2
+            },
+            "gold_positions": {
+              "0": 2,
+              "1": 2,
+              "2": 4,
+              "3": 2,
+              "4": 2
+            }
+          }
+        }
+      }
+    },
+    "hard": {
+      "n_tasks_file": 111,
+      "n_scored": 111,
+      "official_public": {
+        "n": 111,
+        "correct": 41,
+        "invalid": 0,
+        "accuracy": 0.36936936936936937,
+        "ece_10": 0.3791593909263611
+      },
+      "metrics": {
+        "macro_accuracy": 0.38833493046341344,
+        "macro_nll": 2.1149551345876114,
+        "micro": {
+          "n": 111,
+          "accuracy": 0.36936936936936937,
+          "brier_mean": 0.3142712859450615,
+          "brier_sum": 0.951039241082317,
+          "ece_15": 0.37663869814829787,
+          "nll": 2.2420954526317662,
+          "entropy": 0.6359462567449854,
+          "margin": 1.9191036170667357,
+          "predicted_positions": {
+            "4": 5,
+            "3": 11,
+            "1": 50,
+            "0": 34,
+            "2": 11
+          },
+          "gold_positions": {
+            "3": 12,
+            "1": 46,
+            "2": 20,
+            "0": 31,
+            "4": 2
+          }
+        },
+        "by_family": {
+          "adversarial": {
+            "n": 6,
+            "accuracy": 0.3333333333333333,
+            "brier_mean": 0.329005561541213,
+            "brier_sum": 1.0360720430251906,
+            "ece_15": 0.6766607860724131,
+            "nll": 2.9314609134083995,
+            "entropy": 0.524170048996028,
+            "margin": 1.9538231293360393,
+            "predicted_positions": {
+              "0": 3,
+              "1": 2,
+              "2": 1
+            },
+            "gold_positions": {
+              "1": 2,
+              "0": 3,
+              "2": 1
+            }
+          },
+          "ambiguous": {
+            "n": 7,
+            "accuracy": 0.2857142857142857,
+            "brier_mean": 0.35496685184985993,
+            "brier_sum": 1.1069028860902297,
+            "ece_15": 0.523924435888018,
+            "nll": 2.373063987899727,
+            "entropy": 0.601728218515645,
+            "margin": 1.9064587184361048,
+            "predicted_positions": {
+              "2": 2,
+              "1": 3,
+              "0": 2
+            },
+            "gold_positions": {
+              "1": 2,
+              "0": 4,
+              "2": 1
+            }
+          },
+          "judge_hard": {
+            "n": 17,
+            "accuracy": 0.47058823529411764,
+            "brier_mean": 0.4486379756320042,
+            "brier_sum": 0.8972759512640084,
+            "ece_15": 0.48566271978266107,
+            "nll": 2.037071307444988,
+            "entropy": 0.3662706503467713,
+            "margin": 2.5419286559609806,
+            "predicted_positions": {
+              "1": 12,
+              "0": 5
+            },
+            "gold_positions": {
+              "1": 7,
+              "0": 10
+            }
+          },
+          "long_policy": {
+            "n": 19,
+            "accuracy": 0.2631578947368421,
+            "brier_mean": 0.27236642425885704,
+            "brier_sum": 0.9291074031175274,
+            "ece_15": 0.5016953207944569,
+            "nll": 2.2953665892409245,
+            "entropy": 0.8144000351229689,
+            "margin": 1.7862926752943742,
+            "predicted_positions": {
+              "4": 2,
+              "3": 4,
+              "1": 8,
+              "0": 3,
+              "2": 2
+            },
+            "gold_positions": {
+              "3": 4,
+              "1": 9,
+              "2": 3,
+              "0": 2,
+              "4": 1
+            }
+          },
+          "multi_hop": {
+            "n": 18,
+            "accuracy": 0.3888888888888889,
+            "brier_mean": 0.2834510893430854,
+            "brier_sum": 1.0717193129324514,
+            "ece_15": 0.5134819232755237,
+            "nll": 2.763476913613433,
+            "entropy": 0.5936464927522234,
+            "margin": 2.359899971220228,
+            "predicted_positions": {
+              "3": 7,
+              "0": 7,
+              "2": 2,
+              "1": 2
+            },
+            "gold_positions": {
+              "2": 7,
+              "0": 2,
+              "1": 4,
+              "3": 5
+            }
+          },
+          "probability": {
+            "n": 10,
+            "accuracy": 0.1,
+            "brier_mean": 0.4951685533837183,
+            "brier_sum": 1.3519257679233312,
+            "ece_15": 0.7558035969734191,
+            "nll": 3.180345622394031,
+            "entropy": 0.4637652637685445,
+            "margin": 2.383845841884613,
+            "predicted_positions": {
+              "1": 4,
+              "0": 4,
+              "2": 2
+            },
+            "gold_positions": {
+              "1": 6,
+              "0": 1,
+              "2": 3
+            }
+          },
+          "routing_hard": {
+            "n": 5,
+            "accuracy": 0.6,
+            "brier_mean": 0.09874917209342585,
+            "brier_sum": 0.4937458604671292,
+            "ece_15": 0.3892843127250671,
+            "nll": 0.7523306657503555,
+            "entropy": 0.7205350702007266,
+            "margin": 1.654204750061035,
+            "predicted_positions": {
+              "1": 3,
+              "0": 2
+            },
+            "gold_positions": {
+              "1": 5
+            }
+          },
+          "temporal_numeric": {
+            "n": 15,
+            "accuracy": 0.4,
+            "brier_mean": 0.22509232850928135,
+            "brier_sum": 0.8210278367875746,
+            "ece_15": 0.266293736298879,
+            "nll": 1.984593750841293,
+            "entropy": 0.95446657543888,
+            "margin": 1.2332109769185384,
+            "predicted_positions": {
+              "1": 8,
+              "0": 3,
+              "2": 2,
+              "4": 2
+            },
+            "gold_positions": {
+              "1": 5,
+              "0": 4,
+              "2": 3,
+              "3": 3
+            }
+          },
+          "tradeoff": {
+            "n": 6,
+            "accuracy": 0.6666666666666666,
+            "brier_mean": 0.11440804834587807,
+            "brier_sum": 0.395298481023521,
+            "ece_15": 0.3396676282087962,
+            "nll": 0.6505642991908996,
+            "entropy": 0.5314685338007524,
+            "margin": 1.6512227058410645,
+            "predicted_positions": {
+              "0": 2,
+              "1": 3,
+              "4": 1
+            },
+            "gold_positions": {
+              "2": 1,
+              "1": 4,
+              "4": 1
+            }
+          },
+          "trap": {
+            "n": 8,
+            "accuracy": 0.375,
+            "brier_mean": 0.37663994260610417,
+            "brier_sum": 1.0909660520406792,
+            "ece_15": 0.685746856033802,
+            "nll": 2.1812772960920648,
+            "entropy": 0.6376175324074207,
+            "margin": 0.9758526459336281,
+            "predicted_positions": {
+              "1": 5,
+              "0": 3
+            },
+            "gold_positions": {
+              "0": 5,
+              "1": 2,
+              "2": 1
+            }
+          }
+        }
+      }
+    }
+  },
+  "note": "The public-only result is not an official JevBench v1.4 score. Sealed and judge tasks are unavailable; cost is not measured.",
+  "run_id": "f9f472314d223516818433b6f731ed3a05c999bd0be412134ad671d5ddc7808c"
+}

evaluation/2026-09-24/performance.json

@@ -0,0 +1,96 @@
+{
+  "model": "Q1z/Pivot",
+  "model_revision": "14bf8c26bf344ebdf88e22a4b6152dc5f75f3578",
+  "benchmark": {
+    "name": "JevBench",
+    "release": "v1.4.1",
+    "commit": "24b9b5c1609a7a9e8fa14f49e5985a836c9dc842",
+    "scope": "public tasks only",
+    "official_v1_4_score": null,
+    "public_accuracy": 0.46320346320346323,
+    "public_correct": 107,
+    "public_n": 231,
+    "cohorts": {
+      "original": {
+        "n": 72,
+        "accuracy": 0.375,
+        "ece_10": 0.5247104664643605,
+        "invalid": 0
+      },
+      "easy": {
+        "n": 48,
+        "accuracy": 0.8125,
+        "ece_10": 0.1139311231672764,
+        "invalid": 0
+      },
+      "hard": {
+        "n": 111,
+        "accuracy": 0.36936936936936937,
+        "ece_10": 0.3791593909263611,
+        "invalid": 0
+      }
+    },
+    "input_contract": {
+      "context_arm": "context_header",
+      "option_arm": "original",
+      "context_tokens": 512,
+      "option_tokens": 128,
+      "truncation": "right"
+    }
+  },
+  "speed": {
+    "precision": "float32",
+    "measurement": "warm local end-to-end; includes tokenization, inference, and scoring",
+    "h200": {
+      "hardware": "NVIDIA H200",
+      "single_decision": {
+        "batch_size": 1,
+        "n": 32,
+        "p50_ms": 15.766771499897914,
+        "p95_ms": 19.87079100013034,
+        "mean_ms": 16.16144687499599
+      },
+      "throughput": {
+        "batch_size": 32,
+        "decisions_per_repeat": 64,
+        "repeats": 3,
+        "decisions_per_second": 545.2832753857399,
+        "repeat_seconds": [
+          0.12318335300005856,
+          0.1173701870000059,
+          0.11606121599993457
+        ]
+      }
+    },
+    "cpu": {
+      "hardware": "Intel(R) Xeon(R) CPU @ 2.60GHz",
+      "threads": 4,
+      "single_decision": {
+        "batch_size": 1,
+        "n": 32,
+        "p50_ms": 797.5558204999516,
+        "p95_ms": 1089.0265890000137,
+        "mean_ms": 770.7159108749977
+      },
+      "throughput": {
+        "batch_size": 4,
+        "decisions_per_repeat": 64,
+        "repeats": 3,
+        "decisions_per_second": 3.7708397043400965,
+        "repeat_seconds": [
+          16.58172093600001,
+          17.219981521999898,
+          16.97234701499997
+        ]
+      }
+    }
+  },
+  "comparison": {
+    "cpu_to_h200_single_decision_p50_ratio": 50.584599421962544,
+    "throughput_batches_differ": true
+  },
+  "limits": {
+    "official_score": "Requires organizer-held sealed evaluation and cost; not available from public tasks.",
+    "latency": "These are local timings on specific machines. Batch 32 GPU and batch 4 CPU throughput should not be compared as equal configurations."
+  }
+}

evaluation/render_charts.py

@@ -0,0 +1,94 @@
+"""Render the published charts from evaluation/2026-09-24/performance.json.
+
+Usage: python evaluation/render_charts.py
+"""
+import json
+from pathlib import Path
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import numpy as np
+
+RESULTS = Path(__file__).resolve().parent / "2026-09-24"
+data = json.loads((RESULTS / "performance.json").read_text(encoding="utf-8"))
+bench, speed = data["benchmark"], data["speed"]
+cohorts = bench["cohorts"]
+names = list(cohorts)
+accuracies = [100 * cohorts[name]["accuracy"] for name in names]
+ece = [cohorts[name]["ece_10"] for name in names]
+palette = ["#2a58d7", "#129184", "#dc8240"]
+
+plt.rcParams.update({
+    "font.family": "DejaVu Sans", "font.size": 11, "axes.spines.top": False,
+    "axes.spines.right": False, "axes.edgecolor": "#c8d3e4",
+    "axes.facecolor": "#f5f7fb", "text.color": "#182640",
+})
+
+def public_chart(ax):
+    bars = ax.barh(names[::-1], accuracies[::-1], color=palette[::-1], height=.58)
+    ax.set_xlim(0, 105)
+    ax.set_xlabel("Accuracy (%)")
+    ax.set_title("JevBench v1.4.1 public tasks", loc="left", fontweight="bold")
+    for bar, name, value in zip(bars, names[::-1], accuracies[::-1]):
+        ax.text(value + 2, bar.get_y() + bar.get_height() / 2,
+                f"{value:.2f}%  (n={cohorts[name]['n']})", va="center", fontsize=10)
+    ax.text(.02, -.18, f"Total  {bench['public_correct']} / {bench['public_n']} = {100*bench['public_accuracy']:.2f}%",
+            transform=ax.transAxes, fontweight="bold")
+
+def latency_chart(ax):
+    x = np.arange(2)
+    p50 = [speed[key]["single_decision"]["p50_ms"] for key in ("h200", "cpu")]
+    p95 = [speed[key]["single_decision"]["p95_ms"] for key in ("h200", "cpu")]
+    ax.bar(x - .18, p50, .35, color=palette[0], label="p50")
+    ax.bar(x + .18, p95, .35, color=palette[2], label="p95")
+    ax.set_yscale("log")
+    ax.set_ylim(5, 2100)
+    ax.set_xticks(x, ["H200 GPU", "Xeon CPU"])
+    ax.set_ylabel("Milliseconds (log scale)")
+    ax.set_title("Warm single-decision latency", loc="left", fontweight="bold")
+    ax.legend(frameon=False)
+    for pos, values, offset in ((0, p50, -.18), (1, p95, .18)):
+        for index, value in enumerate(values):
+            ax.text(index + offset, value * 1.15, f"{value:.1f} ms", ha="center", fontsize=9)
+
+def throughput_chart(ax):
+    values = [speed[key]["throughput"]["decisions_per_second"] for key in ("h200", "cpu")]
+    batches = [speed[key]["throughput"]["batch_size"] for key in ("h200", "cpu")]
+    ax.bar([f"H200\nbatch {batches[0]}", f"Xeon CPU\nbatch {batches[1]}"],
+           values, color=palette[:2], width=.58)
+    ax.set_yscale("log")
+    ax.set_ylim(1, 2000)
+    ax.set_ylabel("Decisions/s (log scale)")
+    ax.set_title("Warm batched throughput", loc="left", fontweight="bold")
+    for index, value in enumerate(values):
+        ax.text(index, value * 1.15, f"{value:.2f}/s", ha="center", fontsize=10)
+
+fig, ax = plt.subplots(figsize=(8.2, 4.8), layout="constrained")
+public_chart(ax)
+fig.savefig(RESULTS / "accuracy.png", dpi=180)
+plt.close(fig)
+
+fig, axes = plt.subplots(1, 2, figsize=(11, 4.7), layout="constrained")
+latency_chart(axes[0])
+throughput_chart(axes[1])
+fig.suptitle("Pivot | local FP32 inference", fontsize=16, fontweight="bold")
+fig.savefig(RESULTS / "latency_throughput.png", dpi=180)
+plt.close(fig)
+
+fig, axes = plt.subplots(2, 2, figsize=(12.6, 8.8), layout="constrained")
+public_chart(axes[0, 0])
+axes[0, 1].barh(names[::-1], ece[::-1], color=palette[::-1], height=.58)
+axes[0, 1].set_xlim(0, .68)
+axes[0, 1].set_xlabel("ECE, 10 bins (lower is better)")
+axes[0, 1].set_title("Public calibration", loc="left", fontweight="bold")
+for index, value in enumerate(ece[::-1]):
+    axes[0, 1].text(value + .01, index, f"{value:.3f}", va="center")
+latency_chart(axes[1, 0])
+throughput_chart(axes[1, 1])
+fig.suptitle("Pivot | measured public performance", fontsize=19, fontweight="bold")
+fig.text(.02, -.015, "Pinned checkpoint | warm local tokenization + inference + scoring | throughput batch sizes differ",
+         fontsize=9, color="#53637b")
+fig.savefig(RESULTS / "performance_overview.png", dpi=180, bbox_inches="tight")
+plt.close(fig)
+print(f"Saved charts in {RESULTS}")

examples/cached_candidates.py

@@ -0,0 +1,21 @@
+"""Reuse candidate embeddings across multiple calls with the same options."""
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+MODEL = "Q1z/Pivot"
+tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)
+model = AutoModel.from_pretrained(MODEL, trust_remote_code=True, dtype=torch.float32).eval()
+
+options = [
+    "route to billing support",
+    "route to technical support",
+    "route to sales",
+]
+candidate_vectors = model.encode_candidates(tokenizer, options)
+
+for context in (
+    "A customer disputes an invoice amount.",
+    "A customer reports a failed software update.",
+):
+    context_vector = model.encode_context(tokenizer, context)
+    print(model.choose_cached(context_vector, candidate_vectors, options))

examples/native_decision.py

@@ -0,0 +1,20 @@
+"""Structured decisions keep candidate IDs out of the option encoder."""
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+MODEL = "Q1z/Pivot"
+tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)
+model = AutoModel.from_pretrained(MODEL, trust_remote_code=True, dtype=torch.float32).eval()
+
+decision = model.decide_native(
+    tokenizer,
+    "A customer disputes an invoice and asks for a billing correction.",
+    [
+        {"id": "billing", "text": "route to billing support"},
+        {"id": "technical", "text": "route to technical support"},
+        {"id": "sales", "text": "route to sales"},
+        {"id": "abstain", "text": "none of these routes is appropriate", "kind": "abstain"},
+    ],
+)
+print(decision["selected"])
+print(decision["prob_vector"])

examples/quickstart.py

@@ -0,0 +1,16 @@
+"""One context, multiple candidate answers; no text generation."""
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+MODEL = "Q1z/Pivot"
+tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)
+model = AutoModel.from_pretrained(MODEL, trust_remote_code=True, dtype=torch.float32).eval()
+
+context = "CONTEXT:\nA customer disputes an invoice and asks for a billing correction."
+options = [
+    "route to billing support",
+    "route to technical support",
+    "route to sales",
+]
+decision = model.choose(tokenizer, context, options)
+print(decision)  # choice, index, probabilities in supplied candidate order

manifest.json

@@ -7,12 +7,12 @@
   "encoder_runtime_sha256": "f171f518be2a07da48b17fdea5655cad0a2452ab548e90e8ae903143686647e2",
   "files": {
     "README.md": {
-      "sha256": "fa91584d2acb11a7a313d6467002e39f0b4ae02be3b61d2f885e84a326135b49",
-      "bytes": 1097
+      "sha256": "f9661656045863e2293a38b98d1b79c736ef4864c01bf419c47ede101a61c8f5",
+      "bytes": 5574
     },
     "config.json": {
-      "sha256": "7fc9a4faf690fb75956e8fe87d4a858dcbba3977840ce9c33d69e714cb82f47e",
-      "bytes": 3881
+      "sha256": "225cf873bf254492dc6a791aeeb702149485b9df8902c43c056e5ae911b04b77",
+      "bytes": 3883
     },
     "configuration_pivot.py": {
       "sha256": "a65e6ef32f3483dd0015dbbd6faaf75f456c3641a4692764cee7c0b7e39c6628",
@@ -89,6 +89,103 @@
     "training/train_report.json": {
       "sha256": "d597894ba21c47a7c959fca8e4b8a65f901da7647517e3ae66661cedec0d1412",
       "bytes": 4100
+    },
+    "benchmarks/README.md": {
+      "sha256": "b8afec4eb9ef30b85f8fd1efe970ad7be707f1f749005eee38cec1c145283f29",
+      "bytes": 1427
+    },
+    "benchmarks/jevbench_public.py": {
+      "sha256": "de8bd8a3d867dcd1f9e443bc0820bcdda0ca7f0460832c076247c3e5745878ca",
+      "bytes": 13590
+    },
+    "benchmarks/requirements.txt": {
+      "sha256": "7397364a88101a1e34d246f1c4a828cf0a6a2a17e95da61afe9817ace943d7b1",
+      "bytes": 91
+    },
+    "cpu-speed/README.md": {
+      "sha256": "84157cf465febe80a4964c8dac0023e3f6b705bfc71470e5bc9c9ec0ddcbf68f",
+      "bytes": 1136
+    },
+    "cpu-speed/benchmark.py": {
+      "sha256": "22321de328e88fd646840f9266a9d092e72dddb290f6289926da24a72932aad0",
+      "bytes": 308
+    },
+    "docs/BENCHMARK.md": {
+      "sha256": "3bd6f3c535717c2f08378ec286a34cd83ded8b3920ad5ffda290f3f9d41394ba",
+      "bytes": 2151
+    },
+    "docs/INFERENCE.md": {
+      "sha256": "bc00ea8e2998072767208610a5062205ada4d1fe3bb0007d3318666933a2ea9f",
+      "bytes": 3349
+    },
+    "docs/PERFORMANCE.md": {
+      "sha256": "b2bdf733b584d9039b89d3301b1458f5cf5401c205405f8fc05ed6e9ea7be1e3",
+      "bytes": 1866
+    },
+    "docs/UPDATE_NOTES.md": {
+      "sha256": "bb3e44ddd2fc2f615034b9aa4b78cf90e0ec2cc51b57d36743100fe41355fa59",
+      "bytes": 1308
+    },
+    "evaluation/2026-09-24/README.md": {
+      "sha256": "9a1d611d964dd798b88d85e67a677943e25b240fb1a052a77193357421403959",
+      "bytes": 1156
+    },
+    "evaluation/2026-09-24/accuracy.png": {
+      "sha256": "0405c91c2408d049aac19940c30ff2db07a43ab605f9440645fd20d04643ecd1",
+      "bytes": 53007
+    },
+    "evaluation/2026-09-24/cpu_speed.json": {
+      "sha256": "b02c3c1f6b5ba01f0857ecbd6e3a6d50e9df2f4d0cfbbb34eb92b44f86b23697",
+      "bytes": 1203
+    },
+    "evaluation/2026-09-24/jevbench_public.json": {
+      "sha256": "b4c878e64a5c6f5203a5fe97e2b4b9c0e262ddd78feb7b634fe58ee88d1b9af7",
+      "bytes": 16603
+    },
+    "evaluation/2026-09-24/latency_throughput.png": {
+      "sha256": "fb50dfbb7c1e7a37f6da9636cdad56c14caff3b11685ea55c41bf9ce94400805",
+      "bytes": 89972
+    },
+    "evaluation/2026-09-24/performance.json": {
+      "sha256": "81642b8b7b85095d9758e3cc8fed1e595f23fe5744a3986c6c40ca2d397f61b2",
+      "bytes": 2644
+    },
+    "evaluation/2026-09-24/performance_overview.png": {
+      "sha256": "e91059540c6d0e03bcc855a2a009edb8672e61cf99fcd7081ef8dfd354342a04",
+      "bytes": 185988
+    },
+    "evaluation/render_charts.py": {
+      "sha256": "3b8bafdc9ddc7b86655238cf8d8378e57381308875cb13b2f68ac4054a3f3672",
+      "bytes": 4191
+    },
+    "examples/cached_candidates.py": {
+      "sha256": "892e6858506599da59a487b569559a94c23f253eb1ead1dbdfaf954f15c1559a",
+      "bytes": 751
+    },
+    "examples/native_decision.py": {
+      "sha256": "b9edf1839fb90426585908d0b702e7371d6b1b6d973e3fb3eb169fef9eea6a0c",
+      "bytes": 792
+    },
+    "examples/quickstart.py": {
+      "sha256": "355a0d2b5f12dc2016344bee7c8a24e27509911c7f9d1aa8d4501af71bcc48e1",
+      "bytes": 637
+    },
+    "notebooks/Pivot_CPU_Speed.ipynb": {
+      "sha256": "ee50b481a1fd841e06b91656e6febcc9422e7fcf9182de4192f19cbb761636d7",
+      "bytes": 3589
+    },
+    "serving/README.md": {
+      "sha256": "a23d3d61c64958bce4381a34c6308756edad918efc032741f59093d05cab68e5",
+      "bytes": 1081
+    },
+    "serving/typed_decisions.py": {
+      "sha256": "427cb1eac5a1e9bde56f592e6d5fefdb7d537d1aff6d374fd11d9cf1a7ae929d",
+      "bytes": 733
     }
-  }
-}
+  },
+  "documentation_release": "2026-09-24",
+  "evaluated_model_revision": "14bf8c26bf344ebdf88e22a4b6152dc5f75f3578",
+  "benchmark_release": "JevBench v1.4.1 public tasks",
+  "benchmark_commit": "24b9b5c1609a7a9e8fa14f49e5985a836c9dc842",
+  "serving_option_tokens_default": 128
+}

notebooks/Pivot_CPU_Speed.ipynb

@@ -0,0 +1,101 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 5,
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Pivot — CPU speed\n",
+    "\n",
+    "Run this notebook in a Python CPU environment with Git installed. It asks for a Hugging Face key only if you choose to use one; public model files do not need a key. It pins the measured Pivot checkpoint and the official JevBench v1.4.1 public source, then writes a JSON result, chart and ZIP. The result is local speed, not an official composite score.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "import sys, subprocess\n",
+    "subprocess.check_call([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", \"torch==2.8.0\", \"transformers==5.17.0\", \"safetensors==0.8.0\", \"numpy==1.26.4\", \"huggingface_hub>=0.26.0\", \"matplotlib>=3.8,<4\"])\n"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "from getpass import getpass\n",
+    "from huggingface_hub import login\n",
+    "if input(\"Use a private Hugging Face key? (y/N): \").strip().lower() == \"y\":\n",
+    "    login(token=getpass(\"HF token (hidden): \"), add_to_git_credential=False)\n"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "from pathlib import Path\n",
+    "from huggingface_hub import hf_hub_download\n",
+    "script = Path.cwd() / \"benchmarks\" / \"jevbench_public.py\"\n",
+    "if not script.is_file():\n",
+    "    script = Path(hf_hub_download(\"Q1z/Pivot\", \"benchmarks/jevbench_public.py\"))\n",
+    "print(\"Benchmark runner:\", script)\n"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "import json\n",
+    "outdir = Path.cwd() / \"pivot_cpu_run\"\n",
+    "outdir.mkdir(exist_ok=True)\n",
+    "result = outdir / \"cpu_speed.json\"\n",
+    "subprocess.check_call([sys.executable, str(script), \"--mode\", \"speed\", \"--device\", \"cpu\", \"--threads\", \"4\", \"--batch-size\", \"4\", \"--output\", str(result)])\n",
+    "report = json.loads(result.read_text(encoding=\"utf-8\"))\n",
+    "report[\"speed\"]\n"
+   ],
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "from zipfile import ZipFile, ZIP_DEFLATED\n",
+    "from IPython.display import display, FileLink\n",
+    "s = report[\"speed\"]\n",
+    "fig, axes = plt.subplots(1, 2, figsize=(9, 3.5), layout=\"constrained\")\n",
+    "axes[0].bar([\"p50\", \"p95\"], [s[\"single_decision\"][\"p50_ms\"], s[\"single_decision\"][\"p95_ms\"]], color=[\"#2959db\", \"#139684\"])\n",
+    "axes[0].set(title=\"CPU single-decision latency\", ylabel=\"ms\")\n",
+    "axes[1].bar([f\"batch {s['throughput']['batch_size']}\"], [s[\"throughput\"][\"decisions_per_second\"]], color=\"#2959db\")\n",
+    "axes[1].set(title=\"CPU throughput\", ylabel=\"decisions/s\")\n",
+    "chart = outdir / \"cpu_speed_chart.png\"\n",
+    "fig.savefig(chart, dpi=170)\n",
+    "plt.show()\n",
+    "archive = outdir / \"pivot_cpu_speed.zip\"\n",
+    "with ZipFile(archive, \"w\", ZIP_DEFLATED) as z:\n",
+    "    z.write(result, result.name)\n",
+    "    z.write(chart, chart.name)\n",
+    "display(FileLink(str(archive)))\n"
+   ],
+   "execution_count": null,
+   "outputs": []
+  }
+ ]
+}

serving/README.md

@@ -0,0 +1,11 @@
+# Pivot serving interfaces
+
+The model exposes three synchronous methods after loading `AutoModel` and `AutoTokenizer` with `trust_remote_code=True`:
+
+1. `choose(tokenizer, context, options)` returns a simple choice, index and probability vector.
+2. `decide_native(tokenizer, context, candidates)` accepts stable machine IDs, semantic candidate text and at most one explicit abstain candidate.
+3. `decide(tokenizer, state, questions)` returns a typed collection of choice, yes/no and score decisions.
+
+The model operates on supplied options; it does not create new options or generate explanatory text. The default serving limits in this update are 512 context tokens and 128 tokens per option. Keep a stable option set if comparing scores between requests.
+
+The pre-existing [typed request](example_request.json) and [response](example_response.json) and [native request](native_example_request.json) and [response](native_example_response.json) illustrate the wire shapes. Run [a typed Python example](typed_decisions.py) or read the [full inference guide](../docs/INFERENCE.md).

serving/typed_decisions.py

@@ -0,0 +1,20 @@
+"""One context, three typed questions using Pivot's bundled runtime."""
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+repo = "Q1z/Pivot"
+tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
+model = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.float32).eval()
+response = model.decide(
+    tokenizer,
+    "Customer dispute: invoice 120 vs PO 100, age=3d, region=US",
+    [
+        {"id": "route", "primitive": "choice",
+         "options": ["billing", "tech", "sales"]},
+        {"id": "approve", "primitive": "noul",
+         "options": ["true", "false"]},
+        {"id": "severity", "primitive": "score",
+         "options": ["0", "1", "2", "3"]},
+    ],
+)
+print(response)