Avicennasis/GEV-26B-Decide-mlx-8bit-system2

GEV-26B-Decide MLX System 2 add-on (8-bit)

This repository (Avicennasis/GEV-26B-Decide-mlx-8bit-system2) adds System 2 (adaptive thinking) to the published System 1 MLX port Avicennasis/GEV-26B-Decide-mlx-8bit. It holds only the base-model tensors that the System 1 adapter changes (615 tensors in overlay.safetensors), the manifest.json that pairs them with System 1 by content, and the loader gev_mlx_s2.py. Every other weight is shared, in memory, with System 1.

Pin both revisions

Pin this repository's revision and the System 1 revision it pairs with (Avicennasis/GEV-26B-Decide-mlx-8bit @ 060159c9aac49dd86415cdbb7949ad519481e253; any later System 1 commit with the same file contents is also accepted, because pairing is checked by sha256, not by commit id):

import gev_mlx_s2

m = gev_mlx_s2.load(
    system1="Avicennasis/GEV-26B-Decide-mlx-8bit",
    system2="Avicennasis/GEV-26B-Decide-mlx-8bit-system2",
    system1_revision="060159c9aac49dd86415cdbb7949ad519481e253",
    system2_revision="<this repository's commit>",
)
m.decide("noul", "The invoice is marked PAID.", "Has this invoice been paid?", thinking="auto")

python gev_mlx_s2.py predict --system1 Avicennasis/GEV-26B-Decide-mlx-8bit --system2 Avicennasis/GEV-26B-Decide-mlx-8bit-system2 \
    --kind noul --state "The invoice is marked PAID." --question "Has it been paid?" --thinking auto
python gev_mlx_s2.py serve --system1 Avicennasis/GEV-26B-Decide-mlx-8bit --system2 Avicennasis/GEV-26B-Decide-mlx-8bit-system2 --port 8000

The System 1 repository already ships the pinned gev_mlx.py (sha256 09459399b9b3637778358ec33cbf382163a00af8964dadcc1727a52267fccb72); this add-on binds that file from the System 1 snapshot and needs no copy of it.

What it reproduces

The System 2 prompt, policy and readout of upstream serve_decide.py (autotrust/GEV-26B-Decide @ 7c89590ead085bf77630b4bf68264ea30b6ddc78): the thinking frame, sampling (temperature 1.0, top-k 64, top-p 0.95, seeded per question), the stop on <channel|>, the Answer: ( readout over the answer labels, escalation below a System 1 confidence of 0.8 and equal-weight fusion. thinking="off" is exactly System 1.

Provenance

• Overlay: overlay.safetensors, sha256 cf0592b659a5521a7c0a400555e7cc47a468ed44741f71c3859a41a6490ac3cf, converted with mlx_lm.convert (affine, group size 64, 8-bit; routers 8-bit) on mlx 0.32.3 / mlx-lm 0.32.0.
• Loader: gev_mlx_s2.py, sha256 d0634ec33c7d13307c3afd521e59880afea68ddfe6691d984dba1893a77f807a (recorded in manifest.json).
• Manifest: manifest.json, sha256 266f00990cb887cd16b9dbebfda626be8cd50f6c1caae6451f022d80cee6e729 (the released manifest).
• Every shared tensor was checked by sha256 against the System 1 build before release.

Measured on Apple silicon

Readout parity against a bf16 transformers reference on the unmodified base: median per-item max |Δp2| 0.0000 (bar 0.012), maximum 0.1561, 0 top-answer disagreements, all listed in the project report.

Behavior on 200 public items (six public datasets) with thinking="auto" and an 8192-token budget: accuracy 0.7350 with thinking off and 0.8300 with auto; escalation rate 0.4700; thinking time p50 86.0290 s, p95 198.1103 s; end-to-end latency p50 0.4258 s, p95 195.5267 s. Results were measured only on the manifest's mlx and mlx-lm versions.

Limits

• Text only; noul and choice only (score is System 1 only, as upstream).
• Quality was measured on a quantized base, not the bf16 reference.
• The server runs one model call at a time: an escalation holds the model for its whole generation (minutes), and queued System 1 requests wait or get 503. Use a client timeout longer than your worst-case thinking budget. A client that disconnects does not cancel the generation.
• Nothing is enabled automatically; thinking defaults to off.

License

Apache License 2.0 (LICENSE); see NOTICE for the Gemma and loader attribution.

Disclaimer

This is an independent project and is not affiliated with, endorsed by, or connected to autotrust, Google or the upstream GEV-26B-Decide authors.