docs: stable/ folders in Status, oracle verification line, known-issue points at available fix
Browse files
README.md
CHANGED
|
@@ -37,8 +37,9 @@ produces a bare input-arity error naming `ple_table`/`ple_scale`.
|
|
| 37 |
> because previously seen lengths replay from cache at no cost. Long free-form
|
| 38 |
> generations grow without bound until the process exits; no in-process mitigation
|
| 39 |
> exists (token caps only postpone it, and shape-length reuse is the only stable state).
|
| 40 |
-
> A re-export onto a shape-stable contract
|
| 41 |
-
>
|
|
|
|
| 42 |
|
| 43 |
## Provenance
|
| 44 |
|
|
@@ -212,6 +213,7 @@ reachable. 15k was not attempted.
|
|
| 212 |
| `ctx16384/…_tbl_pf64` | **QUALIFIED BY EQUIVALENCE** — as above; free-form at this depth was not attempted. |
|
| 213 |
| `w4a8/gemma4_e4b_qat_decode_int4lin_a8_tbl_pf64` | **EXPERIMENTAL** — built, unmeasured. Gate: a Mac-side oracle/parity check plus a device benchmark. |
|
| 214 |
| `gpu-pipelined/gemma4_e4b_qat_decode_int4lin` | **BLOCKED** — does not load on any current code path. |
|
|
|
|
| 215 |
|
| 216 |
### About the `w4a8` bundle
|
| 217 |
|
|
@@ -292,3 +294,10 @@ trade for the unbounded growth it replaces.
|
|
| 292 |
|
| 293 |
Unlike the `ctx*/` folders, the context ceiling here is **encoded in the graph**: a
|
| 294 |
different window needs a different export, not a manifest edit.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
> because previously seen lengths replay from cache at no cost. Long free-form
|
| 38 |
> generations grow without bound until the process exits; no in-process mitigation
|
| 39 |
> exists (token caps only postpone it, and shape-length reuse is the only stable state).
|
| 40 |
+
> A re-export onto a shape-stable contract that removes this growth is available in this
|
| 41 |
+
> repository under `stable/` — see its section below. It requires updated host support
|
| 42 |
+
> (in progress); on current hosts, use the bundles above for guided generation only.
|
| 43 |
|
| 44 |
## Provenance
|
| 45 |
|
|
|
|
| 213 |
| `ctx16384/…_tbl_pf64` | **QUALIFIED BY EQUIVALENCE** — as above; free-form at this depth was not attempted. |
|
| 214 |
| `w4a8/gemma4_e4b_qat_decode_int4lin_a8_tbl_pf64` | **EXPERIMENTAL** — built, unmeasured. Gate: a Mac-side oracle/parity check plus a device benchmark. |
|
| 215 |
| `gpu-pipelined/gemma4_e4b_qat_decode_int4lin` | **BLOCKED** — does not load on any current code path. |
|
| 216 |
+
| `stable/…_tbl_pf64_stable_c16384` | **EXPERIMENTAL** — shape-stable decode contract; removes the per-token memory growth by design; requires host support for absolute-position inputs (in progress); no on-device measurements yet. |
|
| 217 |
|
| 218 |
### About the `w4a8` bundle
|
| 219 |
|
|
|
|
| 294 |
|
| 295 |
Unlike the `ctx*/` folders, the context ceiling here is **encoded in the graph**: a
|
| 296 |
different window needs a different export, not a manifest edit.
|
| 297 |
+
|
| 298 |
+
**Verification.** Before conversion, the stable graph was checked against the shipped
|
| 299 |
+
graph in eager torch on the same weights: greedy argmax identical on all tested decode
|
| 300 |
+
steps (including positions past the 512-token sliding window), worst logits cosine
|
| 301 |
+
0.99999 (fp16), and the prefill path bit-identical in fp32. On-device behavior is
|
| 302 |
+
unmeasured until host support lands.
|
| 303 |
+
|