HelloSun commited on
Commit
5ec9c4d
·
1 Parent(s): 8466782

stage3: also record F32 conv GEMM negative result

Browse files

- forcing the direct-conv GEMM to F32 (kernel upcast + llamafile tinyBLAS) is
slower than the F16 generic path: im2col 880->1113 ms, GEMM 2180->2614 ms,
VAE 3.8->4.6 s; the 16-bit patches halve GEMM read traffic
- neither BF16 nor F32 upcast is shipped; the further VAE win needs Winograd or
a quantized conv GEMM

Files changed (1) hide show
  1. stages/stage3_vae_conv/README.md +10 -4
stages/stage3_vae_conv/README.md CHANGED
@@ -49,12 +49,18 @@ as BF16):
49
 
50
  | conv GEMM dtype | im2col | GEMM | VAE decode | pixel diff vs F16 |
51
  |---|---|---|---|---|
52
- | F16 (current) | 851 ms | 2196 ms | ~3.8 s | ref |
53
  | BF16 (tinyBLAS) | 1323 ms | 2481 ms | ~4.6 s | 0.10 / 11 |
 
54
 
55
- BF16 is **slower** here, both in im2col (per-run bf16 conversion) and in the GEMM, so it
56
- is not used. The BF16 `tinyBLAS` path is not a win on this machine; the F16 generic path
57
- is already competitive. Not shipped.
 
 
 
 
 
58
 
59
  ## 4. Remaining VAE cost
60
 
 
49
 
50
  | conv GEMM dtype | im2col | GEMM | VAE decode | pixel diff vs F16 |
51
  |---|---|---|---|---|
52
+ | F16 generic (current) | ~880 ms | ~2180 ms | ~3.8 s | ref |
53
  | BF16 (tinyBLAS) | 1323 ms | 2481 ms | ~4.6 s | 0.10 / 11 |
54
+ | F32 (tinyBLAS, kernel upcast) | 1113 ms | 2614 ms | ~4.6 s | ~0 (lossless upcast) |
55
 
56
+ Both alternatives are **slower** than the F16 path: the 16-bit patches halve the GEMM
57
+ read traffic, and on this host the F16 vec-dot is already competitive with the tinyBLAS
58
+ F32/BF16 paths. Neither variant is shipped.
59
+
60
+ The VAE weights themselves load as F16, which is why the GEMM takes the generic F16
61
+ vec-dot path rather than llamafile sgemm. One earlier assumption ("F32 VAE + llamafile
62
+ sgemm is the fast path") only holds for the non-direct `IM2COL` op; inside the direct
63
+ conv the F16 kernel wins.
64
 
65
  ## 4. Remaining VAE cost
66