cortexelus commited on
Commit
0d42b85
·
verified ·
1 Parent(s): 5d1992c

cpu-amx README: consistent <model>_<precision> filenames

Browse files
Files changed (1) hide show
  1. cpu-amx/README.md +5 -5
cpu-amx/README.md CHANGED
@@ -34,12 +34,12 @@ than fp32** — rebuild from source (`optimized/cpu-amx/build/`) for other ISAs.
34
 
35
  | file(s) | model | precision / method | notes |
36
  |---|---|---|---|
37
- | `t5gemma.*` | T5Gemma-b-b-ul2 encoder (text conditioner) | **bf16** AMX GEMMs, fp32 islands (RMSNorm/softmax/softcap/RoPE) | standard softmax attention → bf16-safe |
38
- | `dit_medium.*` + `dit_medium_kernels.tar.gz` | medium DiT (24-block rectified-flow) | **int8** — fully-fused all-integer, AOT-compiled Triton kernels + oneDNN | naive RTN (weight-PTQ *hurts* the DiT — see below) |
39
  | `same_{s,l}_decoder_bf16.*` | SAME-S (50M) / SAME-L (426M) decoder | **bf16** AMX GEMMs, fp32 differential-attention islands | highest-fidelity CPU decoders |
40
  | `same_{s,l}_decoder_int8.*` | SAME-S / SAME-L decoder | **int8 (w8a8, fused)** via **SmoothQuant α0.9 → GPTQ** ("improved" grid) | ½ size; SAME-S +1.2 dB over naive int8 on real music |
41
- | `same_{s,l}_encoder.*` | SAME-S / SAME-L encoder (audio→latent) | **bf16** AMX GEMMs, fp32 differential-attention islands | for audio-to-audio / inpainting |
42
- | `same_l_encoder_weights_f32.bin` | SAME-L encoder (optional) | **fp32** refinement mode | max-fidelity; bf16 is the default |
43
 
44
  **Quantization findings baked into these choices** (full write-up in the repo's `LESSONS.md`):
45
  - The medium **DiT stays naive-int8** — GPTQ/SmoothQuant *lower* its accuracy (its adaLN-modulated qkv +
@@ -82,4 +82,4 @@ optimized/cpu-amx/sa3-gradio # web UI
82
 
83
  Files are flat and self-describing: each engine is `<name>.so` + `<name>_weights.bin` (+ a
84
  `_manifest.txt` giving the mmap layout the loader reads). The DiT additionally needs
85
- `dit_medium_core.bin` (int8 block weights) and `dit_medium_kernels.tar.gz` (its AOT Triton kernels).
 
34
 
35
  | file(s) | model | precision / method | notes |
36
  |---|---|---|---|
37
+ | `t5gemma_bf16.*` | T5Gemma-b-b-ul2 encoder (text conditioner) | **bf16** AMX GEMMs, fp32 islands (RMSNorm/softmax/softcap/RoPE) | standard softmax attention → bf16-safe |
38
+ | `dit_medium_int8.*` + `dit_medium_int8_kernels.tar.gz` | medium DiT (24-block rectified-flow) | **int8** — fully-fused all-integer, AOT-compiled Triton kernels + oneDNN | naive RTN (weight-PTQ *hurts* the DiT — see below) |
39
  | `same_{s,l}_decoder_bf16.*` | SAME-S (50M) / SAME-L (426M) decoder | **bf16** AMX GEMMs, fp32 differential-attention islands | highest-fidelity CPU decoders |
40
  | `same_{s,l}_decoder_int8.*` | SAME-S / SAME-L decoder | **int8 (w8a8, fused)** via **SmoothQuant α0.9 → GPTQ** ("improved" grid) | ½ size; SAME-S +1.2 dB over naive int8 on real music |
41
+ | `same_{s,l}_encoder_bf16.*` | SAME-S / SAME-L encoder (audio→latent) | **bf16** AMX GEMMs, fp32 differential-attention islands | for audio-to-audio / inpainting |
42
+ | `same_l_encoder_bf16_weights_f32.bin` | SAME-L encoder (optional) | **fp32** refinement mode | max-fidelity; bf16 is the default |
43
 
44
  **Quantization findings baked into these choices** (full write-up in the repo's `LESSONS.md`):
45
  - The medium **DiT stays naive-int8** — GPTQ/SmoothQuant *lower* its accuracy (its adaLN-modulated qkv +
 
82
 
83
  Files are flat and self-describing: each engine is `<name>.so` + `<name>_weights.bin` (+ a
84
  `_manifest.txt` giving the mmap layout the loader reads). The DiT additionally needs
85
+ `dit_medium_int8_core.bin` (int8 block weights) and `dit_medium_int8_kernels.tar.gz` (its AOT Triton kernels).