Ben Graville commited on
Commit
36db1ae
·
verified ·
1 Parent(s): 41351c7

README: w4a8 bundles

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -49,3 +49,16 @@ Same weights, same graph, same recipe as the 4096 bundles above — the exporter
49
  | Bundle | Source checkpoint | Context | Functions | KV dtype | Producer | Created |
50
  |---|---|---|---|---|---|---|
51
  | `ctx16384/gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64` | google/gemma-4-E4B-it-qat-q4_0-unquantized | 16384 | main+prefill | fp16 | coreai-core 1.0.0b2 | 20260817T204711Z |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
  | Bundle | Source checkpoint | Context | Functions | KV dtype | Producer | Created |
50
  |---|---|---|---|---|---|---|
51
  | `ctx16384/gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64` | google/gemma-4-E4B-it-qat-q4_0-unquantized | 16384 | main+prefill | fp16 | coreai-core 1.0.0b2 | 20260817T204711Z |
52
+
53
+
54
+ ## w4a8/ — int4 weights + INT8 activations (EXPERIMENTAL)
55
+
56
+ > **EXPERIMENTAL — not qualified.** These bundles have not passed a Mac-side
57
+ > oracle/parity gate or a device benchmark. Do not route production traffic to
58
+ > them until they have.
59
+
60
+ Same QAT int4 per-block-32 weights and same graph as gpu-pipelined/gemma4_e4b_qat_decode_int4lin_tbl_pf64; the only difference is an int8 quantize/dequantize pair on the inputs of every F.linear, calibrated on 128 synthetic social-media-style samples (calibration_corpus.jsonl ships in the bundle). Intended as a prefill/TTFT lever. READ THE CAVEAT: coreai-opt documents the runtime fast path as W_INT8-A_INT8 and states a float weight path executes in floating point regardless of activation quantization. These weights use int4 scale-multiply dequant, not an int8 LUT, so the expected TTFT gain may be zero or negative. This bundle exists to be MEASURED. ctx 16384; includes chat_template.jinja, which the 4096 folders do not.
61
+
62
+ | Bundle | Source checkpoint | Context | Functions | Bytes | Producer | Created |
63
+ |---|---|---|---|---|---|---|
64
+ | `w4a8/gemma4_e4b_qat_decode_int4lin_a8_tbl_pf64` | google/gemma-4-E4B-it-qat-q4_0-unquantized | 16384 | main+prefill | 3,990,690,638 | coreai-core 1.0.0b2 | 20260817T223600Z |