README: ring bundles

README.md

@@ -276,3 +276,69 @@ steps (including positions past the 512-token sliding window), worst logits cosi
 0.99999 (fp16), and the prefill path bit-identical in fp32. On-device behavior is
 unmeasured until host support lands.
 
+
+
+## `ring/` — shape-stable decode, ring sliding caches
+
+> ⚠️ **EXPERIMENTAL — built and gated in torch, on-device measurements pending.** A
+> re-export of the same weights and the same quantization as `stable/`, onto the same
+> host contract, with the KV layout reworked. No throughput or footprint numbers have
+> been taken on a Mac yet; the structural facts below are read off the converted program,
+> not inferred.
+
+| Bundle | Bytes | Context | Functions | Producer | Created |
+|---|---:|---|---|---|---|
+| `ring/gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64_ring_c16384` | 3,989,979,750 | 16384 | main+prefill | coreai-core 1.0.0b2 | 20260818T085331Z |
+
+**What this is.** `stable/` removed the per-generated-token memory growth by making every
+input shape a literal, and it worked — but it gave every layer a full 16,384-slot KV cache
+and read all of it every step. `ring/` keeps that contract byte-for-byte and changes what
+sits behind it:
+
+* **sliding layers read a ring buffer**, 576 slots (the model's own 512-token sliding
+  window plus one 64-token prefill chunk) instead of 16,384. Ring slot `r` holds the
+  newest written position with `pos % 576 == r`, and the mask that selects the written,
+  causal, in-window slots is derived inside the graph from the absolute positions.
+* **the grouped-query head expansion is folded into the query** rather than materialised.
+  The stock lowering broadcasts an index tensor to the full `[1, heads, K, head_dim]` and
+  gathers K and V through it; reshaping the query so the head counts already match
+  produces the same dot products with no index tensor and no gathered copies.
+* **both regions are packed into the same two states**, so the host still binds exactly
+  two KV states, positionally, at whatever literal extent the descriptor declares.
+
+**Contract — unchanged from `stable/`.** Both entrypoints take four inputs and two states,
+all statically shaped:
+
+```
+main    IN  input_ids Int32 1x1  | position_ids Int32 1x1  | ple_table Int8 V x (L*ld) | ple_scale Float32 V
+prefill IN  input_ids Int32 1x64 | position_ids Int32 1x64 | (same statics)
+        ST  keyCache / valueCache Float16 [1, 1, n_kv, TOTAL, 512]
+        OUT logits Float16 1 x S x 262144
+```
+
+`position_ids` carries the absolute position of each of the S tokens in the call, and
+`position_ids[0,0]` is the write position. The mask is derived in the graph, so there is
+no mask input to bind. The KV states have literal extents, so a host that resolves its
+cache strategy from the state descriptor allocates them up front rather than growing them.
+
+**Structural facts** (dumped from the converted program):
+
+| | E2B | E4B |
+|---|---|---|
+| KV state shape | `[1, 1, 1, 56064, 512]` x2 | `[1, 1, 2, 77056, 512]` x2 |
+| KV bytes, both states | 114.8 MB | 315.6 MB |
+| same for `stable/` | 503.3 MB | 1.61 GB |
+| cache slots read per decode step | 56,064 | 77,056 |
+| same for `stable/` | 245,760 | 393,216 |
+| sliding / full layers | 12 x 576 + 3 x 16,384 | 20 x 576 + 4 x 16,384 |
+| dynamic dimensions anywhere | none | none |
+
+**One new host precondition.** A `prefill` call's first position must be a multiple of 64.
+The ring write is a fixed-width store at `p0 % 576`, and 576 is nine 64-token chunks, so a
+64-aligned chunk can never straddle the wrap; an unaligned one would write past the end of
+its own ring region. Feeding whole 64-token chunks from a 64-aligned position — and the
+remainder one token at a time through `main` — was already what the contract asked for.
+`stable/` tolerated an unaligned chunk; `ring/` does not.
+
+Unlike the `ctx*/` folders, the context ceiling is **encoded in the graph**: a different
+window needs a different export, not a manifest edit.