Ben Graville commited on
README: w4a8 bundles
Browse files
README.md
CHANGED
|
@@ -49,3 +49,16 @@ Same weights, same graph, same recipe as the 4096 bundles above — the exporter
|
|
| 49 |
| Bundle | Source checkpoint | Context | Functions | KV dtype | Producer | Created |
|
| 50 |
|---|---|---|---|---|---|---|
|
| 51 |
| `ctx16384/gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64` | google/gemma-4-E4B-it-qat-q4_0-unquantized | 16384 | main+prefill | fp16 | coreai-core 1.0.0b2 | 20260817T204711Z |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
| Bundle | Source checkpoint | Context | Functions | KV dtype | Producer | Created |
|
| 50 |
|---|---|---|---|---|---|---|
|
| 51 |
| `ctx16384/gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64` | google/gemma-4-E4B-it-qat-q4_0-unquantized | 16384 | main+prefill | fp16 | coreai-core 1.0.0b2 | 20260817T204711Z |
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
## w4a8/ — int4 weights + INT8 activations (EXPERIMENTAL)
|
| 55 |
+
|
| 56 |
+
> **EXPERIMENTAL — not qualified.** These bundles have not passed a Mac-side
|
| 57 |
+
> oracle/parity gate or a device benchmark. Do not route production traffic to
|
| 58 |
+
> them until they have.
|
| 59 |
+
|
| 60 |
+
Same QAT int4 per-block-32 weights and same graph as gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64; the only difference is an int8 quantize/dequantize pair on the inputs of every F.linear, calibrated on 128 synthetic social-media-style samples (calibration_corpus.jsonl ships in the bundle). Intended as a prefill/TTFT lever. READ THE CAVEAT: coreai-opt documents the runtime fast path as W_INT8-A_INT8 and states a float weight path executes in floating point regardless of activation quantization. These weights use int4 scale-multiply dequant, not an int8 LUT, so the expected TTFT gain may be zero or negative. This bundle exists to be MEASURED. ctx 16384; includes chat_template.jinja, which the 4096 folders do not.
|
| 61 |
+
|
| 62 |
+
| Bundle | Source checkpoint | Context | Functions | Bytes | Producer | Created |
|
| 63 |
+
|---|---|---|---|---|---|---|
|
| 64 |
+
| `w4a8/gemma4_e4b_qat_decode_int4lin_a8_tbl_pf64` | google/gemma-4-E4B-it-qat-q4_0-unquantized | 16384 | main+prefill | 3,990,690,638 | coreai-core 1.0.0b2 | 20260817T223600Z |
|