HiramKHackenbacker commited on
Commit
668d018
·
verified ·
1 Parent(s): 8e241a6

README: stable bundles

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -256,3 +256,39 @@ model and **the Gemma terms apply downstream**, which is why this repo declares
256
  [Gemma 4 license](https://ai.google.dev/gemma/docs/gemma_4_license). Those obligations
257
  travel with any redistribution of these bundles, including the gather-table sidecar, which
258
  is derived from the same weights. The contribution here is the conversion, not the weights.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
256
  [Gemma 4 license](https://ai.google.dev/gemma/docs/gemma_4_license). Those obligations
257
  travel with any redistribution of these bundles, including the gather-table sidecar, which
258
  is derived from the same weights. The contribution here is the conversion, not the weights.
259
+
260
+
261
+ ## `stable/` — shape-stable decode contract
262
+
263
+ > ⚠️ **EXPERIMENTAL — built, not yet measured on a Mac.** A re-export of the same weights
264
+ > and the same quantization onto a decode/prefill contract in which no input shape moves
265
+ > between steps, which removes the per-generated-token memory growth described in the
266
+ > known issue above. It requires a host that feeds `position_ids` as the **absolute
267
+ > positions of the S new tokens only** (host support in progress); a host that feeds the
268
+ > growing `0..N` prefix will write the KV cache at the wrong offset. No throughput or
269
+ > footprint numbers have been taken yet.
270
+
271
+ | Bundle | Bytes | Context | Functions | Producer | Created |
272
+ |---|---:|---|---|---|---|
273
+ | `stable/gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64_stable_c16384` | 3,989,939,812 | 16384 | main+prefill | coreai-core 1.0.0b2 | 20260818T004019Z |
274
+
275
+ **Contract.** Both entrypoints take four inputs and two states, all statically shaped:
276
+
277
+ ```
278
+ main IN input_ids Int32 1x1 | position_ids Int32 1x1 | ple_table Int8 V x (L*ld) | ple_scale Float32 V
279
+ prefill IN input_ids Int32 1x64 | position_ids Int32 1x64 | (same statics)
280
+ ST keyCache / valueCache Float16 [slots, 1, n_kv, 16384, 512]
281
+ OUT logits Float16 1 x S x 262144
282
+ ```
283
+
284
+ `position_ids` carries the absolute position of each of the S tokens in the call, and
285
+ `position_ids[0,0]` is also the cache slot the K/V for those tokens is written at. The
286
+ attention mask (causality, unwritten slots, and the 512-token sliding window) is derived
287
+ inside the graph from those positions, so there is no mask input to bind. The KV state's
288
+ sequence extent is a literal 16,384 rather than a dynamic dimension, so a host that
289
+ resolves its cache strategy from the state descriptor will allocate the full cache up
290
+ front (0.50 GB for E2B, 1.61 GB for E4B) instead of growing it — that fixed cost is the
291
+ trade for the unbounded growth it replaces.
292
+
293
+ Unlike the `ctx*/` folders, the context ceiling here is **encoded in the graph**: a
294
+ different window needs a different export, not a manifest edit.