model card: standard structure, measured facts, verified license declarations

README.md

@@ -1,7 +1,8 @@
 ---
-license: gemma
-license_link: https://ai.google.dev/gemma/terms
+license: apache-2.0
+license_link: https://ai.google.dev/gemma/docs/gemma_4_license
 base_model: google/gemma-4-E2B-it-qat-q4_0-unquantized
+base_model_relation: quantized
 library_name: coreai
 pipeline_tag: text-generation
 tags:
@@ -9,6 +10,7 @@ tags:
   - aimodel
   - apple-silicon
   - on-device
+  - coreai-kit
   - quantized
   - int4
   - qat
@@ -17,70 +19,46 @@ tags:
 
 # Gemma 4 E2B — Core AI (.aimodel)
 
-`google/gemma-4-E2B-it-qat-q4_0-unquantized` converted to Core AI `.aimodel` bundles for
-Apple silicon by [visible-cx](https://huggingface.co/visible-cx). These are derivative
-artifacts: Google's QAT-trained weights rounded onto the int4 grid they were trained for
-and re-expressed as a Core AI graph. They load through Core AI on macOS and are not usable
-by PyTorch, GGUF or MLX.
+`google/gemma-4-E2B-it-qat-q4_0-unquantized` converted to Core AI `.aimodel` bundles for Apple
+silicon by [visible-cx](https://huggingface.co/visible-cx). These are derivative artifacts:
+Google's QAT-trained weights rounded onto the int4 grid they were trained for and re-expressed
+as a Core AI graph. They load through Core AI on macOS and are not usable by PyTorch, GGUF or
+MLX.
 
-Gemma 4 E2B uses **Per-Layer Embeddings**, so these bundles take a large embedding gather
-table as a *static graph input* rather than carrying it in the graph. That table ships in
+Gemma 4 E2B uses **Per-Layer Embeddings**, so these bundles take a large embedding gather table
+as a *static graph input* rather than carrying it in the graph. That table ships in
 `ios-frontend/` and **the bundles do not load without it**; a missing table produces a bare
-input-arity error naming `ple_table`/`ple_scale`.
-
-> ⚠️ **Known issue — memory growth on long generations.** The Core AI runtime caches a
-> graph specialization per input-shape signature, and this export makes the sequence
-> length load-bearing: every generated token adds one token to `position_ids` and thus
-> mints a new signature, retaining roughly **80 MB of GPU allocations per generated
-> token** (both engines, guided and unguided alike, at equal context). Workloads whose
-> prompt/output lengths repeat — e.g. short constrained extraction rows — stay bounded,
-> because previously seen lengths replay from cache at no cost. Long free-form
-> generations grow without bound until the process exits; no in-process mitigation
-> exists (token caps only postpone it, and shape-length reuse is the only stable state).
-> A re-export onto a shape-stable contract that removes this growth is available in this
-> repository under `stable/` — see its section below. It requires updated host support
-> (in progress); on current hosts, use the bundles above for guided generation only.
-
-## Provenance
-
-| | |
-|---|---|
-| Base checkpoint | `google/gemma-4-E2B-it-qat-q4_0-unquantized` (ungated) |
-| Zoo recipe | `gemma-4-e2b`, `status = "verified"` — `int4lin --tbl` |
-| Recipe (pf64) | `export_gemma4_pf_pipelined.py --pf 64` with `--tbl` and `--raw-dir` pointed at the gather table below |
-| Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
-| Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
-| Producer fingerprint | `coreai-core 1.0.0b2` on every inner asset `metadata.json` |
-| Weight format | **int4, per-block-32** (`int4lin`, symmetric-with-clipping) — the ggml q4_0 grid the QAT checkpoint was trained on |
-| Vocab | 262,144 |
-| Export functions | `main` (S=1 decode) and, in `_pf64` bundles, `prefill` (S=64 chunked prefill) |
-
-"QAT-unquantized" means QAT-*trained*, stored at full width; the int4 rounding happens at
-export, onto the grid training already targeted.
-
-`_tbl` = the PLE gather table is bound as a static graph input. `_pf64` = a second
-entrypoint, `prefill`, with a static query width of 64
-(`function_map: {"main": ["main", "prefill"]}`).
+input-arity error naming `ple_table`/`ple_scale`. It is 2.81 GB and it is part of the model's
+memory cost, not a sidecar you can ignore.
+
+> ⚠️ **Known issue — memory growth per generated token.** The Core AI runtime caches one
+> compiled specialization per input-shape signature, and this export makes sequence length
+> load-bearing: every generated token adds one token to `position_ids` and mints a new
+> signature, retaining roughly **81 MB of GPU allocations per generated token** until the
+> process exits.
+>
+> **Guided decoding does not protect you — shape reuse does.** At 64 generated tokens the same
+> prompt costs **9.84 GB free-form and 9.88 GB guided**; the grammar loop is not a defence. What
+> *is* a defence is repeating lengths you have already run, which replay from cache at no cost:
+> a fixed-schema, fixed-cap extraction workload stays bounded, and that is the shape measured
+> at 10/10 below. Long or variable-length generation grows without bound; no in-process
+> mitigation exists, and it is not fixable below a re-export. `stable/` and `ring/` in this repo
+> are those re-exports — see [Shape-stable re-exports](#shape-stable-re-exports).
 
 ## Contents
 
-### Bundles
+### Production bundles
 
-| Path | Bytes | Context | Functions |
-|---|---:|---|---|
-| `gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64` | 2,122,089,973 | 4096 | main + prefill |
-| `w4a8/gemma4_e2b_qat_decode_int4lin_a8_tbl_pf64` | 2,122,679,604 | 16384 | main + prefill |
+| Path | Bytes | Context | Functions | Status |
+|---|---:|---|---|---|
+| `gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64` | 2,122,089,973 | 4096 | main + prefill | **QUALIFIED (guided / bounded shapes)** |
+| `w4a8/gemma4_e2b_qat_decode_int4lin_a8_tbl_pf64` | 2,122,679,604 | 16384 | main + prefill | EXPERIMENTAL |
 
 Each bundle folder holds `<name>.aimodel/` (`main.mlirb` ≈ 2.09 GB, `main.hash`, asset
 `metadata.json`), a bundle-level `metadata.json`, and `tokenizer/` (`tokenizer.json`
 32,169,626 B, `tokenizer_config.json`, `generation_config.json`, `chat_template.jinja`
 18,569 B). The `w4a8` folder additionally ships its `calibration_corpus.jsonl` (35,045 B).
 
-**Stop token:** every bundle declares `eos_token = "<turn|>"` (id 106), which is the
-turn terminator Gemma 4 emits. `generation_config.json` independently lists
-`eos_token_id: [1, 106, 50]`. A host that stops on the raw upstream `<eos>` instead will
-overrun every reply.
-
 ### The PLE gather-table sidecar — required, not optional
 
 | Path | Files | Bytes |
@@ -99,58 +77,138 @@ overrun every reply.
 
 `meta.json` records the shape and the dequant convention:
 `V 262144, D 1536, PLD 8960, L 35, ld 256, embed_scale_pl 16.0` (= √256, which is what the
-exporter assumes).
+exporter assumes). Every `_tbl` bundle binds `ple_table` (from `embed_per_layer.i8`) and
+`ple_scale` (from `embed_per_layer.scale.f32`) as **static** graph inputs. **A QAT bundle must
+be paired with the QAT tables.**
+
+### Diagnostic and proving assets
+
+| Path | Bundle | Bytes | Context |
+|---|---|---:|---|
+| `stable/gpu-pipelined/` | `…_tbl_pf64_stable_c16384` | 2,122,071,656 | 16384 |
+| `ring/gpu-pipelined/` | `…_tbl_pf64_ring_c16384` | 2,122,101,099 | 16384 |
+| `ring2/gpu-pipelined/` | `…_tbl_pf64_ring_c2048` | 2,122,043,748 | 2048 |
+| `ring2/gpu-pipelined/` | `…_tbl_pf64_ring_c8192` | 2,122,068,315 | 8192 |
+| `ring3/gpu-pipelined/` | `…_tbl_pf64_ring_c2048_gqafull` | 2,122,042,254 | 2048 |
+| `ring3/gpu-pipelined/` | `…_tbl_pf64_ring_c4096_gqafull` | 2,122,050,478 | 4096 |
+| `stable-smoke/` `ring-smoke/` `ring2-smoke/` `ring3-smoke/` | 5-layer truncations | ~1.17 GB each | — |
+
+The `*-smoke` folders are **not models**: 5-layer truncations of the decoder, published only so
+host-side work can be developed against a small download. They carry real weights for the
+layers they keep and produce low-quality text by design.
+
+**Stop token:** every bundle declares `eos_token = "<turn|>"` (id 106), the turn terminator
+Gemma 4 emits. `generation_config.json` independently lists `eos_token_id: [1, 106, 50]`. A
+host that stops on the raw upstream `<eos>` will overrun every reply.
 
-Every `_tbl` bundle binds `ple_table` (from `embed_per_layer.i8`, 2.35 GB) and `ple_scale`
-(from `embed_per_layer.scale.f32`) as **static** graph inputs.
+## Provenance
+
+| | |
+|---|---|
+| Base checkpoint | `google/gemma-4-E2B-it-qat-q4_0-unquantized` (ungated) |
+| Zoo recipe | `gemma-4-e2b`, `status = "verified"` — `int4lin --tbl` |
+| Recipe (pf64) | `export_gemma4_pf_pipelined.py --pf 64` with `--tbl` and `--raw-dir` pointed at the gather table above |
+| Toolchain base | `apple/coreai-models` @ `b1cb71b8522d99408059fa0b98b8742171bcb0b8` + the [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) python overlay |
+| Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` |
+| Producer fingerprint | `coreai-core 1.0.0b2` on every inner asset `metadata.json` |
+| Weight format | **int4, per-block-32** (`int4lin`, symmetric-with-clipping) — the ggml q4_0 grid the QAT checkpoint was trained on |
+| Vocab | 262,144 |
+| Export functions | `main` (S=1 decode) and, in `_pf64` bundles, `prefill` (S=64 chunked prefill) |
+
+"QAT-unquantized" means QAT-*trained*, stored at full width; the int4 rounding happens at
+export, onto the grid training already targeted.
+
+`_tbl` = the PLE gather table is bound as a static graph input. `_pf64` = a second entrypoint,
+`prefill`, with a static query width of 64 (`function_map: {"main": ["main", "prefill"]}`).
+
+The `gpu-pipelined/` bundle mirrors the zoo's verified `gemma-4-e2b` recipe with the `pf64`
+multifunction addition; the gather table is the zoo's own
+[mlboydaisuke](https://huggingface.co/mlboydaisuke) artifact layout, and the zoo's
+`gemma-4-E2B-CoreAI` repo is the upstream this one is a sibling of.
 
 ## Requirements
 
 - **Apple silicon Mac**, Core AI runtime.
 - **Engine contract: 4 inputs** — `input_ids`, `position_ids`, plus static `ple_table` and
-  `ple_scale`. Two engines accept that:
-  - **Pipelined engine** — binds the statics zero-copy over the caller's buffer, but does
-    not support logits, so no grammar-constrained decoding.
+  `ple_scale`. Two engines accept that, and the difference matters:
+  - **Pipelined engine** — binds the statics zero-copy over the caller's buffer, but does not
+    expose logits, so no grammar-constrained decoding. This is the default path in CoreAIKit's
+    `GemmaRuntime` and the path the throughput numbers below were measured on.
   - **Sequential engine** — the only logits-capable engine, and therefore the only path for
     guided decoding. It accepts `>= 2` inputs and binds everything beyond
-    `input_ids`/`position_ids` from `EngineOptions.staticInputBuffers`; an extra input with
-    no buffer is rejected by name. It pays a **one-time copy of every constant input at
-    load — 2.19 GB for E2B**, because its submission path otherwise materialises a foreign
-    buffer-backed view on every forward pass. Binding the table as a raw view instead costs
-    ~0.30 s per pass (3.4 tok/s); the load-time copy takes the same bundle to 8.6 s/row.
-- **States:** `keyCache` / `valueCache` `Float16, 15 × 1 × 1 × ? × 512`. Dynamic sequence
-  dim → `GrowingKVCache` (initial 256, doubling), not a static allocation at the manifest
-  maximum.
-- **KV cost: 30,720 bytes per token** (fp16) — 126 MB at 4096, 252 MB at 8192, 503 MB at
-  16384.
-- **Memory envelope, 16 GB Mac** (Metal `recommendedMaxWorkingSetSize` ≈ 10.7 GB):
-
-  | | |
-  |---|---|
-  | Weights (`main.mlirb`) | 2.09 GB |
-  | PLE table bound as static input | 2.35 GB |
-  | Base | **4.44 GB** |
-  | Affordable context, fp16 KV | ~204k tokens → **131,072, the model cap, binds first** |
-
-  KV is not the ceiling at this tier; the PLE table is. Measured peak footprint on a guided
-  run: **8.26 GB**, max RSS 5.40 GB. **Minimum practical machine memory: 16 GB**, for
-  guided decoding only. Free-form generation does not fit this envelope on any machine —
-  see the known issue above.
-- Sliding-window note: E2B interleaves 12 sliding and 3 full attention layers across 35
-  layers collapsed to 15 KV slots. The export models the window in the *mask*, not in
-  memory — sliding layers occupy full-length slots and zero-pad head_dim 256→512, so 80% of
-  KV bytes sit in windowed slots and 40% is dead padding.
-- The bundle manifest declares `runtime_env COREAI_CHUNK_THRESHOLD=1`. Both engines read
-  the static query width off the graph.
-
-## Performance
-
-Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**: 10-sample harness,
-guided JSON-constrained decoding against a fixed schema, greedy, sequential engine with the
-PLE tables bound as static inputs, `reset()` between samples, 128-token cap. Load excluded
-from `s/row`; sample 1 excluded as a cache-warm outlier.
-
-| | `_tbl_pf64` |
+    `input_ids`/`position_ids` from `EngineOptions.staticInputBuffers`. **A caller that does not
+    supply those buffers gets a load failure by name**, not a fallback:
+    `invalidInputType("Inputs beyond input_ids/position_ids must be bound as constant buffers
+    on this engine, but ["ple_table", "ple_scale"] have none.")`. It pays a **one-time copy of
+    every constant input at load — 2.19 GB for E2B** — because its submission path otherwise
+    materialises a foreign buffer-backed view on every forward pass; binding the table as a raw
+    view instead costs ~0.30 s per pass (3.4 tok/s).
+- **States:** `keyCache` / `valueCache` `Float16, 15 × 1 × 1 × ? × 512`. Dynamic sequence dim →
+  `GrowingKVCache` (initial 256, doubling), not a static allocation at the manifest maximum.
+- **KV cost: 30,720 bytes per token** (fp16) — 126 MB at 4096, 503 MB at 16384. **KV is not the
+  ceiling at this tier; the PLE table is.**
+- **Minimum practical machine memory: 16 GB**, for bounded-shape guided work only.
+- Sliding-window note: E2B interleaves sliding and full attention layers across 35 layers
+  collapsed to 15 KV slots. The export models the window in the *mask*, not in memory —
+  sliding layers occupy full-length slots and zero-pad head_dim 256→512, so 80% of KV bytes sit
+  in windowed slots and 40% is dead padding.
+- The bundle manifest declares `runtime_env COREAI_CHUNK_THRESHOLD=1`.
+
+## Measurements
+
+Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**.
+
+### Throughput and memory — n=2, watchdogged, both runs completed
+
+The published `gpu-pipelined/` bundle at its shipped c4096, with the external PLE tables, on the
+pipelined engine (both mandatory), 73-token prompt, 8 calls of 83 generated tokens, watchdog
+sampling at 0.5 s. **Both runs completed 8/8 and neither tripped.**
+
+| | run 1 | run 2 |
+|---|---:|---:|
+| reclaimable at start | 4.88 | 6.20 |
+| wired baseline → peak | 3.15 → 8.15 | 2.97 → 8.01 |
+| **wired growth** | **5.00** | **5.04** |
+| process footprint peak | 4.64 | 4.58 |
+| compressor | 2.34 → 5.48 | 2.56 → 5.33 |
+| reclaimable trough | **1.35** | 1.78 |
+| **decode** | **41.35 tok/s** | **41.40 tok/s** |
+
+GiB unless stated. n=2 agreeing to **0.8%**.
+
+**Charge this model for two artifacts, not one.** The E-series loads a compiled program *and* a
+PLE table, and a memory law told only about the first is wrong by 2.87 GiB:
+
+```
+compiled blob   resources.bin              2,088,055,648 B  =  1.9447 GiB
+PLE tables      gemma4_qat_gather_raw      2,808,611,840 B  =  2.6157 GiB
+------------------------------------------------------------------------
+what the runtime actually loads                             =  4.5603 GiB
+```
+
+(The PLE figure is the binary payload the runtime binds; the published folder is 231 bytes
+larger because it also carries `meta.json`.)
+
+| denominator | wired ÷ denominator |
+|---|---:|
+| compiled blob alone | **2.581** — absurd |
+| **blob + PLE tables** | **1.101** |
+
+**1.101 sits alongside the LFM MoE's 1.092 and below the dense 12B's 1.157** — the E-series is
+not architecture-exceptional at all; it was being charged for one of the two files it opens. The
+honest requirement on this machine is `4.5603 × 1.10 + 1.25 GiB in-flight floor` = **6.27 GiB**.
+
+A related catalog error, recorded because it points the other way: `approximateBytes` for E2B
+had been taken from the published **LiteRT `.litertlm`** artifact (≈ 2.41 GB), a file the Core AI
+backend never opens. One number over-charged the download size and the other under-charged the
+memory gate, from the same root cause — pricing the wrong artifact.
+
+### Guided structured output
+
+10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, **sequential
+engine** with the PLE tables bound as static inputs, `reset()` between samples, 128-token cap.
+
+| | `gpu-pipelined/…_tbl_pf64` |
 |---|---|
 | Load | 12.1 s |
 | Guided JSON parse | **10/10** |
@@ -160,291 +218,173 @@ from `s/row`; sample 1 excluded as a cache-warm outlier.
 | Decode | **22.7–32.7 tok/s** |
 | TTFT | **0.59–4.20 s** |
 | Peak footprint | 8.26 GB |
-| Max RSS | 5.40 GB |
 | Outcome | completed all ten samples |
 
-**The S=64 `prefill` function carries the time-to-first-token.** Measured head to head on
-a 942-token prompt against a decode-only export of the same weights, exporting `prefill`
-moved TTFT 35.74 s → 4.65 s and s/row 45.04 → 8.83, a **7.7×**. Decode is untouched — the
-prefill function only serves full-width chunks. The decode-only bundle is no longer
-published; every bundle in this repo now carries `prefill`.
-
-**Enum conformance is the grammar's doing.** Unguided, the model emits an off-schema enum
-value on essentially every sample. Guided, all ten are correct, because an off-enum token
-is unsamplable. Decode throughput is essentially unchanged by the constraint; the ~25%
-extra cost of a guided sample is the sequential engine's step-synchronous prefill.
-
-For reference, upstream measured E2B at 77.0/87.1 tok/s on an M4 Max, and an unguided
-pipelined run on the 16 GB machine reached 44.3 tok/s. The 22.7–32.7 tok/s above is the
-guided, sequential-engine figure.
-
-### Unguided workload — memory-capped
+The 41.4 tok/s figure above and the 22.7–32.7 here are not in conflict: the first is the
+pipelined engine unguided, the second is the sequential engine under a grammar mask, whose
+step-synchronous prefill costs roughly 25% per sample. An earlier unguided pipelined run on the
+same machine reached 44.3 tok/s; upstream measured E2B at 77.0/87.1 tok/s on an M4 Max.
 
-Free-form generation from a fixed prompt. **Every generation length below is a memory cap,
-not a model stop** — see the known issue.
+**The S=64 `prefill` function carries the time-to-first-token.** Head to head on a 942-token
+prompt against a decode-only export of the same weights, exporting `prefill` moved TTFT
+35.74 s → 4.65 s and s/row 45.04 → 8.83, a **7.7×**. Decode is untouched. The decode-only
+bundle is no longer published.
 
-| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint |
-|---|---|---|---|---|---|---|
-| 3.4k | 3314 | 12.21 s | 7.2 tok/s | 128 (capped) | 30.4 s | **19.97 GB** |
-| 8k | 7972 | 54.01 s | 2.1 tok/s | 64 (capped) | 84.3 s | **36.71 GB** |
+**Enum conformance is the grammar's doing.** Unguided, the model emits an off-schema enum value
+on essentially every sample. Guided, all ten are correct, because an off-enum token is
+unsamplable.
 
-Output quality up to the cap is sound; the limit is memory, not capability.
+**Needle-in-haystack recall: 3/3 verbatim at 8k**, within a 64-token cap. 15k was not attempted:
+8k already cost 36.71 GB of footprint under the shape-signature defect.
 
-**Needle-in-haystack recall: 3/3 verbatim at 8k**, within a 64-token cap (three distinctive
-facts planted at 10/50/90% of the filler, strict scoring). 15k was not attempted: 8k
-already cost 36.71 GB of footprint.
+### What this model is and is not, on this stack
 
-## Status
-
-| Artifact | Status |
-|---|---|
-| `gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64` (ctx 4096) | **QUALIFIED FOR GUIDED GENERATION** — measured: 10/10 parse, 10/10 enum-clean, 6.12/4.23 s/row, 22.7–32.7 tok/s, 8.26 GB peak footprint. Not qualified for free-form generation on the current reference engine implementation. |
-| `w4a8/gemma4_e2b_qat_decode_int4lin_a8_tbl_pf64` | **EXPERIMENTAL** — built, unmeasured. Gate: a Mac-side oracle/parity check plus a device benchmark. |
-| `stable/…_tbl_pf64_stable_c16384` | **EXPERIMENTAL** — shape-stable decode contract; removes the per-token memory growth by design; host support exists; measured E2B: memory fix confirmed but ~1 tok/s decode at capacity 16384 — awaiting a smaller-capacity export before any use. |
-| `stable-smoke/…_stable_c16384_l5` | **NOT A MODEL** — 5-layer truncated proving asset for host development; produces low-quality text by design. |
-
-### About the `w4a8` bundle
+The shape-signature growth sets a hard ceiling that no cap works around: at ~81 MB/token over a
+~9.6 GB intercept, a 900-token report costs ~54 GB (measured killed) and a 600-token insight
+card ~58 GB. The 128-token ceiling that does survive is shorter than a single card. **Gemma 4 is
+an enrichment model on this stack — bounded, repeating shapes — and it is not a report, RAG or
+long-insight model. The blocker is the export, not the weights.**
 
-Same QAT int4 per-block-32 weights and the same graph as
-`gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64`; the difference is an int8
-quantize/dequantize pair on the inputs of every `F.linear`, calibrated on 128 synthetic
-samples (the corpus ships in the folder). It was built as a prefill/TTFT lever.
+### The `staticInputBuffers` residency question, settled
 
-The caveat comes from the compression toolchain's own documentation: the runtime fast path
-is `W_INT8 × A_INT8`, and a float weight path executes in floating point *regardless* of
-activation quantization. These weights use int4 scale-multiply dequant, not an int8 LUT, so
-the expected TTFT gain may be zero or negative. This bundle exists to be measured, not
-because a win is predicted.
-
-**No numerics gate has been run on any bundle in this repo.** The 10/10 results are
-behavioural (parse rate, enum conformance, clean stop); a decode oracle against an fp32
-reference has not been run.
-
-## License
+The PLE table is bound through `EngineOptions.staticInputBuffers`, and it had been assumed those
+pages stay clean and evictable. **They do not.** Forcing the mapped path
+(`COREAI_GEMMA_TABLES=mapped`) against the owned one, on this bundle:
 
-The upstream QAT checkpoint carries Apache-2.0 metadata and is ungated, but it is a Gemma
-model and **the Gemma terms apply downstream**, which is why this repo declares
-`license: gemma`. Use is subject to the
-[Gemma Terms of Use](https://ai.google.dev/gemma/terms), the
-[Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy) and the
-[Gemma 4 license](https://ai.google.dev/gemma/docs/gemma_4_license). Those obligations
-travel with any redistribution of these bundles, including the gather-table sidecar, which
-is derived from the same weights. The contribution here is the conversion, not the weights.
+| | `owned` (`makeBuffer` + read) | `mapped` (`mmap` + `bytesNoCopy`) |
+|---|---:|---:|
+| `phys_footprint` after load | **2.39 GiB** | **0.19 GiB** |
+| `vmmap` region holding the table | `IOAccelerator`, 2.2 G resident, dirty 64K | `mapped file`, dirty 0K |
+| **wired growth** | **4.67 GiB** | **4.65 GiB** |
 
+The footprint column is real and matters on iOS jetsam accounting. **The wired column is what
+the machine is about, and the two paths are identical to within 0.02 GiB.** Under pressure —
+reclaimable falling to 2.52 GiB with the compressor climbing — not one byte came back. An
+`mmap`-backed static input gets wired at the first forward pass exactly like a program constant.
 
-## `stable-smoke/` — truncated proving asset
+One qualification, added later: that test measured whether the *machine* can take the pages
+back, and it cannot. Releasing the `InferenceFunction` does give them back — the process can,
+even though the kernel cannot.
 
-> ⚠️ **EXPERIMENTAL — not a usable model.** A 5-layer truncation of the E2B decoder,
-> published only so the shape-stable engine contract below can be developed against a
-> small download. It carries real weights for the layers it keeps and produces
-> low-quality text; do not evaluate quality from it.
+## Usage
 
-| Bundle | Bytes | Context | Functions | Producer | Created |
-|---|---:|---|---|---|---|
-| `stable-smoke/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_stable_c16384_l5` | 1,171,287,831 | 16384 | main+prefill | coreai-core 1.0.0b2 | 20260818T003645Z |
-
-
-## `stable/` — shape-stable decode contract
-
-> ⚠️ **EXPERIMENTAL — built, not yet measured on a Mac.** A re-export of the same weights
-> and the same quantization onto a decode/prefill contract in which no input shape moves
-> between steps, which removes the per-generated-token memory growth described in the
-> known issue above. It requires a host that feeds `position_ids` as the **absolute
-> positions of the S new tokens only** (host support in progress); a host that feeds the
-> growing `0..N` prefix will write the KV cache at the wrong offset. Measured on host support (16 GB M2 Pro): the memory defect is
-> confirmed fixed — 6.53 GB flat peak across a 664-token free-form generation
-> (+0.011 MB/token, versus ~81 MB/token and a killed process on the default bundle) and
-> guided output byte-identical to the default bundle — but decode runs at ~1.0 tok/s
-> versus ~27–31, because every step reads the full 16,384-slot cache. Not usable as
-> shipped; a smaller-capacity export would trade window for speed. Treat as a working
-> proof of the contract, not a deployable bundle.
-
-| Bundle | Bytes | Context | Functions | Producer | Created |
-|---|---:|---|---|---|---|
-| `stable/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_stable_c16384` | 2,122,071,656 | 16384 | main+prefill | coreai-core 1.0.0b2 | 20260818T003828Z |
-
-**Contract.** Both entrypoints take four inputs and two states, all statically shaped:
+Swift Package Manager, via [CoreAIKit](https://github.com/john-rocky/coreai-kit) — a community
+package, not affiliated with Apple, requiring macOS 27 beta:
 
+```swift
+.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
+// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")
 ```
-main    IN  input_ids Int32 1x1  | position_ids Int32 1x1  | ple_table Int8 V x (L*ld) | ple_scale Float32 V
-prefill IN  input_ids Int32 1x64 | position_ids Int32 1x64 | (same statics)
-        ST  keyCache / valueCache Float16 [slots, 1, n_kv, 16384, 512]
-        OUT logits Float16 1 x S x 262144
-```
-
-`position_ids` carries the absolute position of each of the S tokens in the call, and
-`position_ids[0,0]` is also the cache slot the K/V for those tokens is written at. The
-attention mask (causality, unwritten slots, and the 512-token sliding window) is derived
-inside the graph from those positions, so there is no mask input to bind. The KV state's
-sequence extent is a literal 16,384 rather than a dynamic dimension, so a host that
-resolves its cache strategy from the state descriptor will allocate the full cache up
-front (0.50 GB for E2B, 1.61 GB for E4B) instead of growing it — that fixed cost is the
-trade for the unbounded growth it replaces.
 
-The context ceiling here is **encoded in the graph**: a different window needs a different
-export, not a manifest edit.
+An E-series bundle is **two downloads**, the decoder and its paired PLE tables, addressed as two
+paths inside this repo:
 
-**Verification.** Before conversion, the stable graph was checked against the shipped
-graph in eager torch on the same weights: greedy argmax identical on all tested decode
-steps (including positions past the 512-token sliding window, with 384 cache slots
-unwritten-and-masked), worst logits cosine 0.99998 (fp16), and the prefill path
-bit-identical in fp32. On-device behavior is unmeasured until host support lands.
+```swift
+import CoreAIKit
 
+let store = ModelStore.default
+let decoderURL = try await store.download(
+    ModelID("visible-cx/gemma-4-E2B-CoreAI",
+            path: "gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64"))
+let tablesURL = try await store.download(
+    ModelID("visible-cx/gemma-4-E2B-CoreAI",
+            path: "ios-frontend/gemma4_qat_gather_raw"))
 
+// engineVariant defaults to .pipelined here, which is the supported path for _tbl bundles.
+let runtime = try await GemmaRuntime(
+    decoderBundleAt: decoderURL, tablesAt: tablesURL, arch: .gemma4)
+```
 
-## `ring-smoke/` — truncated proving asset
+Notes that are not optional:
 
-> ⚠️ **EXPERIMENTAL — not a usable model.** A 5-layer truncation of the E2B decoder,
-> published only so the shape-stable engine contract can be developed against a small
-> download. It carries real weights for the layers it keeps and produces low-quality
-> text; do not evaluate quality from it.
+- **Do not pair a QAT bundle with non-QAT tables.**
+- **Do not enable chunked prefill.** The `…_tbl` graph is S=1 on `main`; `GemmaRuntime` sets
+  `COREAI_CHUNK_THRESHOLD=1` for you if it is unset. Leave it.
+- **Guided decoding needs the sequential engine**, and the sequential engine needs
+  `ple_table`/`ple_scale` supplied through `EngineOptions.staticInputBuffers` — it will refuse
+  by name otherwise.
+- Pass `revision:` a Hub commit hash to pin immutable bytes.
 
-| Bundle | Bytes | Context | Functions | Producer | Created |
-|---|---:|---|---|---|---|
-| `ring-smoke/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_ring_c16384_l5` | 1,171,301,346 | 16384 | main+prefill | coreai-core 1.0.0b2 | 20260818T085021Z |
+## Integrity
 
+Core AI `.aimodel` bundles are **not byte-reproducible**: the exporter is not deterministic even
+against itself. Verify by digesting the exact published bytes rather than by rebuilding. Every
+bundle carries `main.hash`, the raw 32 bytes of `sha256(main.mlirb)`; on the Hub the same value
+is recoverable from the LFS oid without fetching the file. The compiled-blob identity this
+project keys its measurements on for the published bundle is
+`73bef8155c41d512e9c6b4ab1788b7547bade134`.
 
-## `ring/` — shape-stable decode, ring sliding caches
+## Status
 
-> ⚠️ **EXPERIMENTAL — built and gated in torch, on-device measurements pending.** A
-> re-export of the same weights and the same quantization as `stable/`, onto the same
-> host contract, with the KV layout reworked. No throughput or footprint numbers have
-> been taken on a Mac yet; the structural facts below are read off the converted program,
-> not inferred.
+| Artifact | Status |
+|---|---|
+| `gpu-pipelined/…_tbl_pf64` (ctx 4096) | **QUALIFIED FOR BOUNDED-SHAPE GENERATION** — measured: 10/10 guided parse and enum-clean, 6.12/4.23 s/row, 41.4 tok/s unguided on n=2 completed watchdogged runs, 5.00–5.04 GiB wired growth, honest requirement 6.27 GiB. **Not qualified for free-form or variable-length generation** — see the known issue. |
+| `w4a8/…_a8_tbl_pf64` | **EXPERIMENTAL** — built, unmeasured. Same int4 weights and graph; the difference is an int8 quantize/dequantize pair on the inputs of every `F.linear`, calibrated on 128 synthetic samples (corpus ships in the folder). Built as a prefill/TTFT lever, with the toolchain's own caveat that the runtime fast path is `W_INT8 × A_INT8` and these weights use int4 scale-multiply dequant — so the expected gain may be zero or negative. It exists to be measured, not because a win is predicted. |
+| `stable/…_stable_c16384` | **EXPERIMENTAL** — the memory fix is confirmed (6.53 GB flat peak across a 664-token generation, +0.011 MB/token, guided output byte-identical to the default bundle) but decode runs at **~1.0 tok/s** because every step reads the full 16,384-slot cache. A working proof of the contract, not a deployable bundle. |
+| `ring/`, `ring2/`, `ring3/` | **EXPERIMENTAL — gated in torch, no on-device numbers.** Diagnostic assets for the capacity-vs-cost question. |
+| `*-smoke/` | **NOT MODELS** — 5-layer truncations for host development. |
 
-| Bundle | Bytes | Context | Functions | Producer | Created |
-|---|---:|---|---|---|---|
-| `ring/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_ring_c16384` | 2,122,101,099 | 16384 | main+prefill | coreai-core 1.0.0b2 | 20260818T085141Z |
+**No numerics gate has been run on device for any bundle in this repo.** The 10/10 results are
+behavioural (parse rate, enum conformance, clean stop); a decode oracle against an fp32
+reference has not been run.
 
-**What this is.** `stable/` removed the per-generated-token memory growth by making every
-input shape a literal, and it worked — but it gave every layer a full 16,384-slot KV cache
-and read all of it every step. `ring/` keeps that contract byte-for-byte and changes what
-sits behind it:
+## Shape-stable re-exports
 
-* **sliding layers read a ring buffer**, 576 slots (the model's own 512-token sliding
-  window plus one 64-token prefill chunk) instead of 16,384. Ring slot `r` holds the
-  newest written position with `pos % 576 == r`, and the mask that selects the written,
-  causal, in-window slots is derived inside the graph from the absolute positions.
-* **the grouped-query head expansion is folded into the query** rather than materialised.
-  The stock lowering broadcasts an index tensor to the full `[1, heads, K, head_dim]` and
-  gathers K and V through it; reshaping the query so the head counts already match
-  produces the same dot products with no index tensor and no gathered copies.
-* **both regions are packed into the same two states**, so the host still binds exactly
-  two KV states, positionally, at whatever literal extent the descriptor declares.
+`stable/`, `ring/`, `ring2/` and `ring3/` are re-exports of the same weights and the same
+quantization onto a contract in which **no input shape moves between steps**, which removes the
+per-generated-token growth by design. They need a host that feeds `position_ids` as the
+**absolute positions of the S new tokens only**; a host that feeds the growing `0..N` prefix
+will write the KV cache at the wrong offset.
 
-**Contract — unchanged from `stable/`.** Both entrypoints take four inputs and two states,
-all statically shaped:
+**Contract** (identical across all four families):
 
 ```
 main    IN  input_ids Int32 1x1  | position_ids Int32 1x1  | ple_table Int8 V x (L*ld) | ple_scale Float32 V
 prefill IN  input_ids Int32 1x64 | position_ids Int32 1x64 | (same statics)
-        ST  keyCache / valueCache Float16 [1, 1, n_kv, TOTAL, 512]
+        ST  keyCache / valueCache Float16, literal extents
         OUT logits Float16 1 x S x 262144
 ```
 
-`position_ids` carries the absolute position of each of the S tokens in the call, and
-`position_ids[0,0]` is the write position. The mask is derived in the graph, so there is
-no mask input to bind. The KV states have literal extents, so a host that resolves its
-cache strategy from the state descriptor allocates them up front rather than growing them.
+`position_ids[0,0]` is also the cache slot the K/V for those tokens is written at. The mask —
+causality, unwritten slots and the sliding window — is derived inside the graph from those
+positions, so there is no mask input to bind. The context ceiling is **encoded in the graph**: a
+different window needs a different export, not a manifest edit.
 
-**Structural facts** (dumped from the converted program):
+**`stable/` gave every layer a full 16,384-slot cache and read all of it every step**, which is
+the 1 tok/s. `ring/` keeps the contract byte for byte and changes what sits behind it: sliding
+layers read a **576-slot ring** (the model's 512-token window plus one 64-token prefill chunk)
+instead of 16,384, and the grouped-query head expansion is folded into the query rather than
+materialised.
 
-| | E2B | E4B |
+| | `stable/` | `ring/` |
 |---|---|---|
-| KV state shape | `[1, 1, 1, 56064, 512]` x2 | `[1, 1, 2, 77056, 512]` x2 |
-| KV bytes, both states | 114.8 MB | 315.6 MB |
-| same for `stable/` | 503.3 MB | 1.61 GB |
-| cache slots read per decode step | 56,064 | 77,056 |
-| same for `stable/` | 245,760 | 393,216 |
-| sliding / full layers | 12 x 576 + 3 x 16,384 | 20 x 576 + 4 x 16,384 |
-| dynamic dimensions anywhere | none | none |
-
-**One new host precondition.** A `prefill` call's first position must be a multiple of 64.
-The ring write is a fixed-width store at `p0 % 576`, and 576 is nine 64-token chunks, so a
-64-aligned chunk can never straddle the wrap; an unaligned one would write past the end of
-its own ring region. Feeding whole 64-token chunks from a 64-aligned position — and the
-remainder one token at a time through `main` — was already what the contract asked for.
-`stable/` tolerated an unaligned chunk; `ring/` does not.
-
-Unlike the `ctx*/` folders, the context ceiling is **encoded in the graph**: a different
-window needs a different export, not a manifest edit.
-
-
-## `ring2/` — the same graph at three KV capacities
-
-> **EXPERIMENTAL — diagnostic assets, not products.** These are the `ring/` recipe, byte
-> for byte, exported at a smaller full-attention capacity. They exist to measure how the
-> per-step cost of the shape-stable graph depends on that capacity. No throughput numbers
-> have been taken on device for either of them. Their context ceilings (2,048 and 8,192)
-> are smaller than the 16,384 of `ring/` and are encoded in the graph, so neither is a
-> drop-in replacement for anything already published here.
-
-| Bundle | Context | KV state shape | KV bytes, both states | Sliding / full layers |
-|---|---:|---|---:|---|
-| `ring2/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_ring_c2048` | 2,048 | `[1, 1, 1, 13056, 512]` x2 | 26.7 MB | 12 x 576 + 3 x 2,048 |
-| `ring2/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_ring_c8192` | 8,192 | `[1, 1, 1, 31488, 512]` x2 | 64.5 MB | 12 x 576 + 3 x 8,192 |
-| `ring/gpu-pipelined/..._ring_c16384` (for comparison) | 16,384 | `[1, 1, 1, 56064, 512]` x2 | 114.8 MB | 12 x 576 + 3 x 16,384 |
-
-**Contract: identical to `ring/`.** Same four inputs in the same order, same two states in
-the same positional order, `position_ids` as the absolute positions of the S tokens in the
-call, the mask derived in the graph, the same 64-alignment precondition on `prefill`. The
-only thing that differs is the literal extent of the full-attention region, and a host that
-reads its cache strategy from the state descriptor needs no change to drive them.
-
-**Equivalence.** Both were gated in eager torch against the shipped graph before conversion,
-at their own capacity: 1,600 prompt tokens as 25 chunks of 64 then 32 greedy decode steps,
-final position 1,631, so the sliding ring wraps roughly three times. fp32, 57 comparisons
-per run, **0 argmax mismatches** and worst cosine 0.999999999987 for both. No dynamic
-dimension appears in either entrypoint of either bundle.
-
-## `ring2-smoke/` — truncated proving asset
-
-> **EXPERIMENTAL — not a usable model.** A 5-layer truncation of the E2B decoder at the
-> 2,048 capacity (`[1, 1, 1, 4352, 512]` states, 4 x 576 + 1 x 2,048), published only so
-> host-side work can be developed against the contract over a smaller download. It carries
-> real weights for the layers it keeps and produces low-quality text; do not evaluate
-> quality from it.
-
-
-## `ring3/` — per-layer-type SDPA form
-
-> **EXPERIMENTAL — gated in torch, no on-device numbers yet.** Same weights, same
-> quantization, same host contract as `ring/`. One thing changes inside the graph: the
-> grouped-query expansion is folded into the query **only in the full-attention layers**,
-> and the sliding layers keep the stock gathered form.
-
-**Why the split.** Folding the GQA expansion into the query removes the gathered copies and
-the broadcast index tensor, but it also turns G independent per-head matmuls into a single
-batch-1 matmul with G times the rows. That trade is worth taking when the key length is long
-(the full-attention layers read the whole capacity) and not worth taking when it is short
-(the sliding layers read a fixed 576-slot ring, where the operation is short enough that
-losing the per-head parallelism costs more than the bytes it saves). `ring/` applied the fold
-everywhere; `ring3/` applies it only where the key length is long.
-
-Both forms compute the same dot products in the same order. The equality gate below is run
-against the shipped graph, not against `ring/`.
-
-**Contract: unchanged.** Same four inputs in the same order, same two KV states in the same
-positional order, `position_ids` carrying the absolute position of each of the S tokens in
-the call, the mask derived inside the graph, the same requirement that a `prefill` call start
-at a multiple of 64. A host driving `ring/` drives these with no change.
-
-**Equality.** Gated in eager torch against the shipped graph before conversion: 1,600 prompt
-tokens as 25 chunks of 64 then 32 greedy decode steps, final position 1,631, so the sliding
-ring wraps about three times. fp32, 57 comparisons per run, **0 argmax mismatches** on every
-bundle below. No dynamic dimension appears in either entrypoint of any of them.
-
-| Bundle | Context | KV state shape | KV bytes, both states | Sliding / full slots |
-|---|---:|---|---:|---|
-| `ring3/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_ring_c4096_gqafull` | 4,096 | `[1, 1, 1, 19200, 512]` x2 | 39.3 MB | 12 x 576 + 3 x 4,096 |
-| `ring3/gpu-pipelined/gemma4_e2b_qat_decode_int4lin_tbl_pf64_ring_c2048_gqafull` | 2,048 | `[1, 1, 1, 13056, 512]` x2 | 26.7 MB | 12 x 576 + 3 x 2,048 |
-
-E2B runs 28 sliding and 7 full attention layers; in these bundles the 28 keep the gathered
-form and the 7 use the folded one.
-
-## `ring3-smoke/` — truncated proving asset
-
-> **EXPERIMENTAL — not a usable model.** A 5-layer truncation of the E2B decoder at the 4,096
-> capacity (`[1, 1, 1, 6400, 512]` states, 4 x 576 + 1 x 4,096), published only so host-side
-> work can be developed against the contract over a smaller download. It carries real weights
-> for the layers it keeps and produces low-quality text; do not evaluate quality from it.
+| KV state shape | `[slots, 1, n_kv, 16384, 512]` ×2 | `[1, 1, 1, 56064, 512]` ×2 |
+| KV bytes, both states | 503.3 MB | **114.8 MB** |
+| cache slots read per decode step | 245,760 | **56,064** |
+| dynamic dimensions | none | none |
+
+`ring2/` is the same recipe at 2,048 and 8,192 (26.7 MB and 64.5 MB of KV); `ring3/` folds the
+GQA expansion **only in the full-attention layers**, where the key length is long enough for the
+trade to pay, and keeps the stock gathered form in the short-key sliding layers.
+
+**One host precondition new in `ring/`:** a `prefill` call's first position must be a multiple
+of 64. The ring write is a fixed-width store at `p0 % 576`, and 576 is nine 64-token chunks, so
+a 64-aligned chunk can never straddle the wrap. `stable/` tolerated an unaligned chunk; `ring/`
+does not.
+
+**Equivalence gates**, all run in eager torch against the shipped graph before conversion:
+`stable/` — greedy argmax identical on every tested decode step including past the sliding
+window, worst logits cosine 0.99998 (fp16), prefill bit-identical in fp32. `ring*/` — 1,600
+prompt tokens as 25 chunks of 64 then 32 greedy decode steps, final position 1,631 so the ring
+wraps about three times, fp32, 57 comparisons per run: **0 argmax mismatches** on every bundle,
+worst cosine 0.999999999987. No dynamic dimension appears in either entrypoint of any of them.
+
+## License
+
+Google publishes the upstream QAT checkpoint under **Apache-2.0** with a `license_link` to the
+[Gemma 4 license](https://ai.google.dev/gemma/docs/gemma_4_license), and this repo mirrors that
+declaration. Use is governed by those terms and by the
+[Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy); the
+obligations travel with any redistribution of these bundles, **including the gather-table
+sidecar**, which is derived from the same weights. The contribution here is the conversion, not
+the weights.