docs: add operational examples and link llama.cpp SH runner

README.md

@@ -15,17 +15,22 @@ tags:
   - decode-free
 ---
 
+
 # Standard One 8B SH
 
+
 > **Updated weights (v4, 2026-10-13).** v4 is trained further on decision data, mostly our own synthetic items. Decision Index 0.3 public index 57.40 (self-scored with the official kit; v3 48.34).
 > Earlier releases stay available under the tags `v3`, `v2` (merged weights) and `v1` (LoRA adapter).
 
+
 **Version:** v4
 
+
 **Standard One 8B SH** is Standard One 8B with a **joint schema head** (SH). It reads the scenario once and
 scores every option of every question in a single forward pass, then returns probabilities through the same
 `POST /v1/systemone` contract as Standard One. It does not generate text.
 
+
 | | |
 |---|---|
 | Backbone | Standard One 8B v2.2 (`mistral3`, Ministral 3 8B text + Pixtral vision tower), fine-tuned for the head and merged; weights in this repository |
@@ -37,19 +42,24 @@ scores every option of every question in a single forward pass, then returns pro
 | Smaller model | [StandardOne-3B-SH](https://huggingface.co/StandardThinking/StandardOne-3B-SH) |
 | License | Apache-2.0 |
 
+
 ## Why a schema head
 
+
 Standard One reads one answer letter per option, so a question is limited by the label alphabet and every
 question in a request needs its own pass over the scenario. The schema head instead attends over the hidden states
 of the whole request (the final layer and one intermediate layer), places every question and option in one joint
 graph, and returns a calibrated distribution for each question at once. In practice:
 
+
 - **Many options.** Intent and routing questions with 100–255 options keep nearly the accuracy of small option sets.
 - **Many questions per scenario.** All questions of a request share one pass over the scenario.
 - **Long scenarios.** Decisions buried in long documents are found far more reliably than with Standard One v2.2.
 
+
 ## Results
 
+
 | Benchmark | SH v3 | **SH v4 (this release)** |
 |---|---:|---:|
 | Decision Index 0.3, public index | 48.34 | **57.40** |
@@ -59,43 +69,59 @@ graph, and returns a calibrated distribution for each question at once. In pract
 | Decision hidden in a long document (our set, 150 items) | 87.33 | 88.00 |
 | JevBench public: easy / original / hard (48 / 72 / 111 items) | 100.00 / 95.83 / 74.77 | 100.00 / 94.44 / 75.68 |
 
+
 Decision Index rows (DI) are chance-corrected skill scores (0 = random guessing, 100 = perfect) from our own runs of the public
 0.3 suite with the official kit, all 140,178 scoreable requests; every benchmark of both runs is in [standardone-8b-sh-v3](https://huggingface.co/datasets/StandardThinking/StandardOne-SH-decision-index/tree/main/runs/standardone-8b-sh-v3)
 and [standardone-8b-sh-v4](https://huggingface.co/datasets/StandardThinking/StandardOne-SH-decision-index/tree/main/runs/standardone-8b-sh-v4). For some DI benchmarks the public train split or the benchmark's public
 generator was part of training (HoVer among them); evaluated items never were. The other rows are accuracy in percent,
 measured with the Standard One engine on identical items.
 
+
 ## Benchmark charts
 
+
 Decision Index 0.3 public index, board data of 2026-10-12; Standard One self-scored with the official kit (see the footnote of each chart).
 The same charts with notes: [Standard One SH on the Decision Index](https://standardthinking.ai/resources/guides/standard-one-sh-decision-index/).
 
+
 ![Public index against model size](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/01-index-vs-size.png)
 
+
 ![Retrieval & Classification](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/02-retrieval-classification.png)
 
+
 ![Tools & Automation](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/03-tools-automation.png)
 
+
 ![Product search relevance (Amazon ESCI)](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/04-product-search.png)
 
+
 ![Multi-hop claim verification (HoVer)](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/05-claim-verification.png)
 
+
 ![Smart-home request handling (Home appliance simulator)](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/06-home-appliance.png)
 
+
 ![Contract understanding (ContractNLI)](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/07-contract-nli.png)
 
+
 ![Reading chords from a score (POP909-CL)](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/08-chords.png)
 
+
 ![Same decisions in 21 languages](https://huggingface.co/StandardThinking/StandardOne-8B-SH/resolve/main/assets/charts/09-multilingual.png)
 
+
 ## Changes in v4
 
+
 - **Further training on synthetic decision data.** v4 continues from v3 on decision items built for this round, mostly our own
   synthetic items (code-checked, LLM-written in varied styles), with replay of earlier data. See "Results" for the
   Decision Index 0.3 and the long-context checks.
 
+
 ## Changes in v3
 
+
 - **Hard decision tasks.** v3 is fine-tuned on about 2,200 new decision items written in the families where v2 was
   weakest: long policies with exceptions and overrides, multi-criteria trade-offs, judging a response against a
   rubric, and date and number reasoning. The items were authored by large language models, every gold answer was
@@ -111,22 +137,29 @@ The same charts with notes: [Standard One SH on the Decision Index](https://stan
   points (41.17 vs 46.00) and the JevBench original tier two items (95.83 vs 98.61). Intent labels, WinoGrande and
   policy judgement are within half a point of v2. Keep `v2` if those held-out suites matter most to you.
 
+
 ## Prompt format
 
+
 The included `chat_template.jinja` has an empty default system message, and the reference server renders requests
 without one (`SH_SYSTEM_PROMPT=none`, the default in `code/render.py`). Do not add the Ministral default system
 prompt: the head was trained without it, and adding it changes the hidden states it reads.
 
+
 ## Quick start
 
+
 ```bash
-pip install -r code/requirements.txt   # torch, transformers, safetensors
+python -m pip install --upgrade huggingface_hub
 hf download StandardThinking/StandardOne-8B-SH --local-dir StandardOne-8B-SH
+python -m pip install -r StandardOne-8B-SH/code/requirements.txt
+
 
 python StandardOne-8B-SH/code/serve_head.py \
   --snapshot StandardOne-8B-SH --head StandardOne-8B-SH/head --port 30171
 ```
 
+
 ```bash
 curl -s localhost:30171/v1/systemone -H 'content-type: application/json' -d '{
   "state": {"message": "Please close my card, I lost it yesterday."},
@@ -139,19 +172,23 @@ curl -s localhost:30171/v1/systemone -H 'content-type: application/json' -d '{
 }'
 ```
 
+
 The server batches concurrent requests (`--max-batch-requests`, `--max-batch-tokens`) and binds `127.0.0.1` by
 default. It runs on one GPU with the backbone in bfloat16 and the head in float32. On Blackwell GPUs (B200/B300)
 install a CUDA 13 build of PyTorch first (`pip install torch --index-url https://download.pytorch.org/whl/cu130`); the
 CUDA 12.8 build lacks kernels for those GPUs.
 
+
 ## Fast serving with the Standard One engine
 
+
 The reference server above is plain PyTorch. For production throughput and latency, serve this repository with the
 **Standard One engine**, our SGLang fork with a prefill-only decision mode and the schema-head readout
 ([standard-one-sglang](https://github.com/standardthinkingai/standard-one-sglang),
 [standard-one-adapter](https://github.com/standardthinkingai/standard-one-adapter), Apache-2.0). It is the engine we run
 in production. Build the image once (it replaces only the Python package of a stock SGLang image):
 
+
 ```bash
 mkdir ctx && cd ctx
 git clone https://github.com/standardthinkingai/standard-one-sglang sglang-jev
@@ -161,8 +198,10 @@ docker build -f sglang-jev/examples/runtime/jev/rocm/Dockerfile.decision \
   --build-arg BASE_IMAGE=lmsysorg/sglang:nightly-dev-cu13-20260929-79cafec0 -t standard-one-engine:cu13 .
 ```
 
+
 Then serve the downloaded repository as it is (weights at the top level, head in `head/`):
 
+
 ```bash
 docker run -d --gpus all --network host --ipc host --shm-size 32g \
   -v $PWD/StandardOne-8B-SH:/models/m:ro standard-one-engine:cu13 python3 -m sglang.launch_server \
@@ -173,9 +212,11 @@ docker run -d --gpus all --network host --ipc host --shm-size 32g \
     --disable-radix-cache --disable-decode-cuda-graph --disable-prefill-cuda-graph --decision-deadline-seconds 110
 ```
 
+
 It answers `POST /v1/systemone` on port 30000. Unlike the reference server, the engine requires the `model` field
 (the served model name), as in the System One API:
 
+
 ```bash
 curl -s localhost:30000/v1/systemone -H 'content-type: application/json' -d '{
   "model": "standardthinking/standard-one-8b-sh",
@@ -184,13 +225,114 @@ curl -s localhost:30000/v1/systemone -H 'content-type: application/json' -d '{
 }'
 ```
 
+
 Checked on one NVIDIA B300 against the reference server on 1,431 evaluation questions: 99.58 % the same answer, all
 231 JevBench public answers identical. On AMD MI350X/MI355X build with
 `BASE_IMAGE=lmsysorg/sglang-rocm:v0.5.20-rocm720-mi35x-20260929` and run with
 `--device /dev/kfd --device /dev/dri --group-add video -e SGLANG_USE_AITER=0` instead of `--gpus all`.
 
+
+
+## Run locally with llama.cpp
+
+For CPU or Apple-silicon use, follow the download, build and server instructions in [StandardOne-8B-SH-GGUF](https://huggingface.co/StandardThinking/StandardOne-8B-SH-GGUF#quick-start-with-llamacpp). The SH head reads hidden states from the backbone, so it requires the supplied state extractor and Python runner; stock `llama-server -hf` alone does not execute the schema head. Once the runner is listening on port 30171, use the requests below unchanged.
+
+## Practical examples
+
+Use the running server at `http://127.0.0.1:30171/v1/systemone`. SH scores all questions jointly without generating text. These examples are illustrative workloads, not measured accuracy results; inspect the actual response on your chosen model and quantization.
+
+### Customer support: route, handoff and priority
+
+Run this in a second terminal after starting the server:
+
+```bash
+curl --fail-with-body http://127.0.0.1:30171/v1/systemone \
+  -H "Content-Type: application/json" --data-binary @- <<'JSON'
+{
+  "state": "My account was charged twice. I have contacted support three times and want a person to help with the refund today.",
+  "questions": {
+    "team": {
+      "type": "choice",
+      "instructions": "Which team should handle this message?",
+      "criteria": {
+        "billing": "Charges, refunds and invoices",
+        "technical": "Application errors and login problems",
+        "other": "Requests outside billing and technical support"
+      }
+    },
+    "handoff": {"type": "noul", "instructions": "Does the customer explicitly request a human agent?"},
+    "priority": {
+      "type": "score",
+      "instructions": "How urgently should support review this request?",
+      "criteria": ["Can wait", "Review today", "Service is blocked right now"]
+    }
+  }
+}
+JSON
+```
+
+The intended routing label here is `billing`, with a human handoff. This describes the task, not a promised model output. `choice` selects from supplied categories; `noul` scores a yes/no question; `score` uses an ordered scale and returns an expected zero-based level that can be fractional.
+
+### Ticket routing, incident severity, agent completion and document routing
+
+This Python client uses only the standard library and prints the complete server response. The requests work with the BF16 reference server and the SH GGUF runner on port 30171.
+
+```python
+import json
+from urllib.request import Request, urlopen
+
+def decide(state, questions):
+    request = Request(
+        "http://127.0.0.1:30171/v1/systemone",
+        data=json.dumps({"state": state, "questions": questions}).encode("utf-8"),
+        headers={"Content-Type": "application/json"},
+        method="POST",
+    )
+    with urlopen(request, timeout=300) as response:
+        return json.load(response)
+
+# Route a ticket to one queue from your own taxonomy.
+print(json.dumps(decide(
+    "I cannot sign in after resetting my password. The reset link has expired.",
+    {"queue": {"type": "choice", "instructions": "Which queue owns the ticket?",
+               "criteria": {"account_access": "Login and password problems",
+                            "billing": "Payments and invoices",
+                            "bug_report": "Other application defects",
+                            "other": "None of these queues"}}}
+), indent=2))
+
+# Score an ordered incident severity scale.
+print(json.dumps(decide(
+    "Production checkout is unavailable for all customers. The incident is ongoing.",
+    {"severity": {"type": "score", "instructions": "Assess the incident severity.",
+                  "criteria": ["Informational; no service impact",
+                               "Minor impact with a workaround",
+                               "Major degradation affecting some customers",
+                               "Critical outage affecting all customers"]}}
+), indent=2))
+
+# Check an observed result against the original goal.
+print(json.dumps(decide(
+    "Goal: send the weekly report to the team. Observed state: draft saved; the message has not been sent.",
+    {"finished": {"type": "noul", "instructions": "Has the agent completed the goal?"}}
+), indent=2))
+
+# Classify a document before selecting an extraction workflow.
+print(json.dumps(decide(
+    "INVOICE 2048. Amount due: USD 240. Payment due within 30 days.",
+    {"document_type": {"type": "choice", "instructions": "Classify the document.",
+                       "criteria": {"invoice": "A request for payment",
+                                    "receipt": "Confirmation of completed payment",
+                                    "meeting_notes": "Notes from a meeting",
+                                    "other": "None of these document types"}}}
+), indent=2))
+```
+
+Each `choice` question selects one category. For independent tags, submit one `noul` question per tag in the same request. Choose probability thresholds on representative validation data before automating routing or handoff. These examples use text input, the validated path for this release.
+
 ## Files
 
+
 | Path | Contents |
 |---|---|
 | `model-0000{1..4}-of-00004.safetensors`, `config.json`, tokenizer files | merged backbone, bfloat16 (`Mistral3ForConditionalGeneration`) |
@@ -199,8 +341,10 @@ Checked on one NVIDIA B300 against the reference server on 1,431 evaluation ques
 | `code/` | reference `/v1/systemone` server: request rendering, head, scoring |
 | `SHA256SUMS`, `release-manifest.json` | checksums; source and merge details |
 
+
 ## Limitations
 
+
 - Requests longer than 32,768 tokens are refused, not truncated. This is a serving limit: the backbone handles longer
   inputs, but the head was not trained on them.
 - On our hard held-out decision suite (600 items; long policies, multi-step lookups, date and number reasoning)
@@ -209,7 +353,10 @@ Checked on one NVIDIA B300 against the reference server on 1,431 evaluation ques
 - The reference server is a PyTorch server; for throughput use the Standard One engine (see above).
 - Probabilities are calibrated on our held-out data; refit thresholds on your own distribution before relying on them.
 
+
 ## License
 
+
 Apache-2.0. Built on [Standard One 8B](https://huggingface.co/StandardThinking/StandardOne-8B) (Apache-2.0), which is
 built on [Ministral 3 8B](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-BF16) (Apache-2.0).
+