mlboydaisuke/decider-0.8b-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine.

This model has no row on DeviceMark, the on-device LLM leaderboard.

decider-0.8b — Core AI (System One decision model, int8, pipelined GPU engine)

Apple Core AI (.aimodel) conversion of Mapika/decider-0.8b. The zoo card and the gate scripts live in coreai-model-zoo; this page is the same text with repository links.

🤗 mlboydaisuke/decider-0.8b-CoreAI · Apache-2.0 · source Mapika/decider-0.8b (revision 1ea5412) · base Qwen/Qwen3.5-0.8B-Base

A System One decision model: it reads a state (text or JSON) and a set of typed questions — Choice (2–255 options), Score (2–10 described levels), Noul (probability of yes) — and returns a probability for every option from the letter logits at an answer slot. It never generates text. Mapika fine-tuned Qwen3.5-0.8B-Base for this readout (one epoch over 1.47M examples, per the author's card) and ships it behind the same POST /v1/systemone shape as the larger decider-2b. The author's own numbers, quoted from the source card and not re-measured here: in-task accuracy 0.776, held-out 0.707, calibrated with temperature 1.03.

This is the zoo's first decision model: the value is the calibrated probability, so the gate below is a probability-parity gate against the author's fp32 inference code, not a token match. The bundle is the Qwen3.5-0.8B ship recipe with the HF id swapped (int8hu --head-sym: linear int8 per block of 32 with an absmax-symmetric int8 head, decode-only loop-free S=1 graph on the pipelined GPU engine), 1.34 GB, context 4,096.

Readout contract

Every question is one independent row in the author's state_first layout:

Context:\n<state>\n\nQuestion: <question>\nOptions:\n(A) <option>\n(B) <option>...\nAnswer: (

• The answer slot is the last token of the row; the next-token logits at that position are the readout. No chat template, no BOS, no generated token.
• Labels come from the bundle's tokenizer: A..Z, then the first 229 two-letter strings that encode as one token (255 labels; AA = 5840, AB = 1803, …). Only the first nopts labels are read; p = softmax(logits[labels[:nopts]] / 1.03).
• A Score question with the checkpoint's isolated_levels = true becomes one yes/no row per level (<question>\nProposed answer: <level>\nDoes the proposed answer fit?); the level probabilities are the normalized yes-mass. neutralize_none = false: option strings are used unchanged.
• Rows must fit the bundle's 4,096-token context. A 255-option row is ≥ 1,275 tokens by construction; the fixture's is 1,965.

conversion/decider/oracle_decider.py builds the fixture rows through the author's unchanged decider/ package (downloaded from the checkpoint) and records the fp32 probabilities: fixtures-decider-0.8b.json — 13 requests, 44 rows (23 Choice rows with 3–10 options, 9 Noul, 10 isolated Score rows from 2 Score questions, one 11-option row, one 255-option row), every row's ids, slot, label ids, fp32 slot logits and probabilities, and the author's system_one API output for each request (assembled answers equal the row-level probabilities, 13/13). Minimum oracle top-2 margin 0.51 — no near-ties, so the argmax gate has no exemptions.

Measured (Apple M4 Max, macOS 27.0 26A428, 2026-09-21)

	fp16 build (reference)	int8hu --head-sym (ship)
letter argmax = fp32 oracle	44/44	44/44
full-vocabulary argmax is one of the row's labels	44/44	44/44
max |Δp| over all option probabilities	0.0050	0.0084
mean of per-row mean |Δp|	0.00018	0.00067
Swift pipelined engine, first greedy token = oracle label	44/44	44/44
state reset proof (row 1 re-run, logits bit-identical)	yes	yes

Ship bar: argmax 44/44 with no exemption, max |Δp| ≤ 0.02 and mean of row means ≤ 0.002 — four times the fp16 build's floor. The fp16 floor is the graph's own fp16 logits (the pipelined engine requires a float16 logits output), not conversion error.

Two paths produce those rows, because the pipelined engine samples on the GPU and exposes no logits:

• Probabilities: the bundle is AOT-compiled (coreai-build compile … --platform macOS --preferred-compute gpu --architecture h16c --expect-frequent-reshapes) and the .aimodelc is driven S=1 through the Core AI Python runtime with fresh zero states per row — conversion/decider/readout_gate_decider.py, transcript gate-decider-0.8b-readout.json. AOT is required, not an optimization: on 26A428 the Python runtime's JIT of this graph logged MTL4CommandQueueErrorDomain error 1 on every forward and returned all-zero logits.
• Engine argmax: Release llm-runner --raw-tokens <row ids> --max-tokens 1 --temperature 0.0 --inference-engine-variant coreai-pipelined --warmup off (COREAI_CHUNK_THRESHOLD=1) must emit the oracle's label string — conversion/decider/engine_argmax_decider.py, transcript gate-decider-0.8b-engine.json.

The zoo's language-model gate also passes on the ship bundle — coreai_gate.py, prompt "The alphabet begins A, B, C, D, E, F,", 16/16 token-exact vs the fp32 overlay oracle (gate-decider-0.8b.json): the fine-tune still speaks, which the System One API never asks of it.

Throughput (ship bundle, Release llm-benchmark, p=128 g=256, coreai-pipelined, COREAI_CHUNK_THRESHOLD=1, 2 launches × 3 trials): decode 193.6 tok/s median (186.5–197.0), prefill 226.4 (201.6–236.6), load 1.5 s cold / 0.2 s warm. No other Core AI work was on the GPU; a CPU-bound job from another lane ran on the same machine during the measurement. Because prefill is S=1 on this graph, a System One request costs about rows × (state + question tokens) / decode rate — ten independent questions over a 300-token state are ~3,500 steps.

Swift side

coreai-kit has no systemOne op yet. The design — CoreAI.systemOne(state:questions:options:) mirroring the author's wire shape, the prompt builder port line by line, a tokenizer-parity contract on the 44 fixture rows, and the readout primitive (recommended: a completion-synchronized read-last-logits call on the pipelined engine, whose decodeLogitsBuffers already hold the fp16 logits; fallback: the zoo's N-state low-level runner) — is in knowledge/decider-systemone-op-design.md. Until it exists the Swift engine gives the argmax only (the first greedy token), and the probabilities come from the Python runtime.

iPhone: not measured. The frozen fork's pipelined engine caps the iOS growing KV cache at 1,024 tokens, so the 255-option row does not run on the phone as is; the 43 other rows fit.

⬇️ Bundle

mlboydaisuke/decider-0.8b-CoreAI gpu-pipelined/decider_0_8b_decode_int8hu_block32_sym/ — .aimodel (main.mlirb 1,309,263,719 B, sha256 2ab6d715…aaf4), metadata.json, tokenizer/. Runs on the pipelined engine with the zoo's apps/coreai-pipelined-extra-states.patch (the hybrid's conv/rec states) and COREAI_CHUNK_THRESHOLD=1, like every Qwen3.5 bundle here.

Reproduce

# export (recipe.toml): the Qwen3.5 exporter with the HF id swapped; the decider checkpoint
# stores a flat qwen3_5_text config, so the loader falls back from text_config to the root.
python3 conversion/zoo_convert.py run decider-0.8b

# fixtures + fp32 oracle through the author's own decider/ package (uv-managed env, CPU, ~5 min)
uv run conversion/decider/oracle_decider.py --out models/decider-0.8b/fixtures-decider-0.8b.json

# probability gate: AOT h16c + Python runtime (overlay interpreter, DEVELOPER_DIR = Xcode 27 RC)
python3 conversion/decider/readout_gate_decider.py exports/decider_0_8b_decode_int8hu_block32_sym \
    models/decider-0.8b/fixtures-decider-0.8b.json --transcript models/decider-0.8b/gate-decider-0.8b-readout.json

# engine argmax gate: Release llm-runner from the patched fork
python3 conversion/decider/engine_argmax_decider.py exports/decider_0_8b_decode_int8hu_block32_sym \
    models/decider-0.8b/fixtures-decider-0.8b.json --runner <fork>/.build/release/llm-runner \
    --transcript models/decider-0.8b/gate-decider-0.8b-engine.json

Port notes: knowledge/decider-0.8b-port.md.

License

Source Apache-2.0 (Mapika/decider-0.8b); the bundle inherits it. The author's decider/ inference code is used by the oracle script at gate time and is not part of the bundle.