stage3: also record F32 conv GEMM negative result
Browse files- forcing the direct-conv GEMM to F32 (kernel upcast + llamafile tinyBLAS) is
slower than the F16 generic path: im2col 880->1113 ms, GEMM 2180->2614 ms,
VAE 3.8->4.6 s; the 16-bit patches halve GEMM read traffic
- neither BF16 nor F32 upcast is shipped; the further VAE win needs Winograd or
a quantized conv GEMM
stages/stage3_vae_conv/README.md
CHANGED
|
@@ -49,12 +49,18 @@ as BF16):
|
|
| 49 |
|
| 50 |
| conv GEMM dtype | im2col | GEMM | VAE decode | pixel diff vs F16 |
|
| 51 |
|---|---|---|---|---|
|
| 52 |
-
| F16 (current) |
|
| 53 |
| BF16 (tinyBLAS) | 1323 ms | 2481 ms | ~4.6 s | 0.10 / 11 |
|
|
|
|
| 54 |
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
## 4. Remaining VAE cost
|
| 60 |
|
|
|
|
| 49 |
|
| 50 |
| conv GEMM dtype | im2col | GEMM | VAE decode | pixel diff vs F16 |
|
| 51 |
|---|---|---|---|---|
|
| 52 |
+
| F16 generic (current) | ~880 ms | ~2180 ms | ~3.8 s | ref |
|
| 53 |
| BF16 (tinyBLAS) | 1323 ms | 2481 ms | ~4.6 s | 0.10 / 11 |
|
| 54 |
+
| F32 (tinyBLAS, kernel upcast) | 1113 ms | 2614 ms | ~4.6 s | ~0 (lossless upcast) |
|
| 55 |
|
| 56 |
+
Both alternatives are **slower** than the F16 path: the 16-bit patches halve the GEMM
|
| 57 |
+
read traffic, and on this host the F16 vec-dot is already competitive with the tinyBLAS
|
| 58 |
+
F32/BF16 paths. Neither variant is shipped.
|
| 59 |
+
|
| 60 |
+
The VAE weights themselves load as F16, which is why the GEMM takes the generic F16
|
| 61 |
+
vec-dot path rather than llamafile sgemm. One earlier assumption ("F32 VAE + llamafile
|
| 62 |
+
sgemm is the fast path") only holds for the non-direct `IM2COL` op; inside the direct
|
| 63 |
+
conv the F16 kernel wins.
|
| 64 |
|
| 65 |
## 4. Remaining VAE cost
|
| 66 |
|