- forcing the direct-conv GEMM to F32 (kernel upcast + llamafile tinyBLAS) is
slower than the F16 generic path: im2col 880->1113 ms, GEMM 2180->2614 ms,
VAE 3.8->4.6 s; the 16-bit patches halve GEMM read traffic
- neither BF16 nor F32 upcast is shipped; the further VAE win needs Winograd or
a quantized conv GEMM
stage3: record BF16 conv GEMM negative result (slower than the F16 path)
- tried converting the VAE conv kernel to BF16 to reach llamafile's 32-wide
AVX512-BF16 tinyBLAS: im2col 851->1323 ms, GEMM 2196->2481 ms, VAE 3.8->4.6 s
- reverted; the VAE weights are already F16 and the generic F16 vec-dot GEMM is
the faster path on this host
- profile the direct conv into im2col / gemm / permute (GGML_CONV_PROFILE=1);
im2col and gemm were each ~half, permute small
- rewrite im2col: hoist sy/sx and the padding check out of the inner loop, and
for the interior (dilation 1, whole knl_w run in bounds, F32 src) copy the
contiguous run directly; hoist the F32/F16 store branch out too
-> im2col 1716 ms -> 875 ms, output diff mean 0 / max 0
- add GGML_CONV_PATCH_MB diagnostic knob (capping the im2col buffer to L3 made
no difference, left unlimited)
- add stages/stage3_vae_conv notes; README + build.sh updated
- root cause: the reference Q4_K GGUF stores the 2 noise + 2 context refiner
blocks as Q8_0; on a memory-bound CPU GEMM the same M=1024 w13 matmul is
89.4 ms (Q8_0) vs 42.9 ms (Q4_K), ~-1.5 s per 8-step run
- tools/requant_refiner.py + tools/q4k.c rewrite only the 16 refiner
attention/FFN matrices as Q4_K (adaLN kept at Q8_0); build.sh now produces
z_image_turbo-Q4_K-ffn13-rq.gguf (opt-in, pixel diff mean 6.4)
- flash-attn: hoist V packing to once per head and raise tiles to 128x128;
FLASH_ATTN_EXT 4663 -> 4451 ms (-4.5%), pixel-identical
- add stages/stage_p2_refiner notes, SmartTest rq row, README findings
Pipeline stage P1: shape-level profile diagnostic, flash-attn K packing once per head, build.sh submodule reset fix
