Keep the card consistent with the uploaded images

README.md

@@ -60,9 +60,7 @@ are the official ones.
 | `comfyui/moss_image_sft_fp16_fixed.safetensors` | ComfyUI single file (fp16) |
 | `reports/{base,sft}_fp16_fix.json` | every number on this page, per image |
 
-<img src="images/gallery/base_comfyui_p1_s11.webp" alt="fp16 without the fix | fp16 with the fix | bf16 | fp32, ComfyUI">
-
-fp16 without the fix | fp16 with the fix | bf16 | fp32 (ComfyUI, 1024^2, seed 11).
+<img src="images/gallery/base_comfyui_p1_s11.webp" alt="fp16-fixed vs official bf16 vs fp32, ComfyUI">
 
 ## Why: the stream sits at 95% of the fp16 limit
 
@@ -81,22 +79,20 @@ block.
 
 ![plain fp16 and the margin per block, Base](images/base_plain_fp16.webp)
 
-<img src="images/base_plain_fp16_p0.webp" alt="fp16 without the fix | fp16 with the fix | bf16 | fp32, 1024^2">
-
-fp16 without the fix | fp16 with the fix | bf16 | fp32 (KohakuMoss, 1024^2, prompt 0).
+<img src="images/base_plain_fp16_p0.webp" alt="official weights fp16 / bf16 / fixed fp16">
 
 **Where plain fp16 breaks.** ComfyUI with `--fp16-unet` on the official Base file, 50
 steps, CFG 3, euler / simple: all 8 prompts at 1024^2 finish (stream max 62,080), and so
 does 2048^2 (64,096, 2.2% below the limit). **2560^2 is the first size that gives a
 black image**: on step 29 of 50 the stream reaches 65,504 at block 26, every later
-forward is NaN, and the image is black. KohakuMoss in plain fp16 breaks at the same step
-and block, and fp32 peaks at @@FP32_2560_MAX@@ there. The fixed weights peak at 4,188
-(x 16 = 67,008, 15.6x headroom) and match bf16 and fp32:
+forward is NaN, and the image is black. A single fp32 forward at this size peaks at
+67,681, 3.3% over the fp16 limit; bf16 peaks at 67,048 and stays finite. The fixed
+weights peak at 4,188 (x 16 = 67,008, 15.6x headroom):
 
-<img src="images/base_plain_fp16_break_2560.webp" alt="2560^2: fp16 without the fix (black) | fp16 with the fix | bf16 | fp32">
+<img src="images/base_fp16_break_2560.webp" alt="2560x2560: fp16 without the fix (NaN) | fp16 with the fix | bf16 | fp32">
 
 fp16 without the fix | fp16 with the fix | bf16 | fp32 (ComfyUI, 2560x2560, prompt 0
-"tabby cat", seed 11). In single forwards over 7 sigmas, 2560^2, 2048x3072, 3072x2048
+"tabby cat", seed 11; the fp32 image is still rendering). In single forwards over 7 sigmas, 2560^2, 2048x3072, 3072x2048
 and 3072^2 overflow; 1536^2, 2048^2, 768x2048 and 1024x4096 do not. A 226-token prompt
 moves the stream by under 1%.
 
@@ -169,26 +165,20 @@ steps) against fp32.
 1024^2, 50 steps, CFG 3, 8 prompts x seeds 11 / 12 (16 images), each runtime against its
 own fp32 run. LPIPS (AlexNet) / PSNR, lower / higher is closer.
 
-| runtime | variant | fp16, no fix | fp16-fixed | official bf16 | fixed vs bf16 |
-| --- | --- | --- | --- | --- | --- |
-| ComfyUI | Base | @@P_comfyui_base@@ | **0.0062** / 44.8 dB | 0.0501 / 29.3 dB | 0.047 |
-| ComfyUI | SFT | @@P_comfyui_sft@@ | **0.0042** / 43.2 dB | 0.0413 / 29.3 dB | 0.042 |
-| diffusers | Base | @@P_diffusers_base@@ | **0.0111** / 42.8 dB | 0.0900 / 26.8 dB | 0.089 |
-| diffusers | SFT | @@P_diffusers_sft@@ | **0.0094** / 42.2 dB | 0.0655 / 28.1 dB | 0.065 |
-| KohakuMoss | Base | @@P_kohakumoss_base@@ | **0.0145** / 42.7 dB | 0.0934 / 26.7 dB | 0.090 |
-| KohakuMoss | SFT | @@P_kohakumoss_sft@@ | **0.0121** / 42.9 dB | 0.0683 / 27.4 dB | 0.072 |
+| runtime | variant | fp16-fixed | official bf16 | fixed vs bf16 |
+| --- | --- | --- | --- | --- |
+| ComfyUI | Base | **0.0062** / 44.8 dB | 0.0501 / 29.3 dB | 0.047 |
+| ComfyUI | SFT | **0.0042** / 43.2 dB | 0.0413 / 29.3 dB | 0.042 |
+| diffusers | Base | **0.0111** / 42.8 dB | 0.0900 / 26.8 dB | 0.089 |
+| diffusers | SFT | **0.0094** / 42.2 dB | 0.0655 / 28.1 dB | 0.065 |
+| KohakuMoss | Base | **0.0145** / 42.7 dB | 0.0934 / 26.7 dB | 0.090 |
+| KohakuMoss | SFT | **0.0121** / 42.9 dB | 0.0683 / 27.4 dB | 0.072 |
 
 fp16 carries 3 more mantissa bits than bf16; with the stream in range that is what the
 fixed weights give back. "fixed vs bf16" is the size of bf16's own rounding drift, not an
-error of the fix. No 1024^2 run produced a non-finite latent or image: at this size plain
-fp16 still fits, and its distance to fp32 is the fixed weights' plus the GPU difference
-below.
+error of the fix. No 1024^2 run produced a non-finite latent or image.
 
-"fp16, no fix" ran on a V100 (the other columns on an RTX 5090), with each seed's initial
-noise as the RTX 5090 draws it. The same plain-fp16 input on the two GPUs differs by
-@@GPU_DRIFT@@ (LPIPS, prompt 0, both seeds), which is part of that column's distance.
-
-Each panel: fp16 without the fix | fp16 with the fix | bf16 | fp32 (ComfyUI, seed 11).
+Each panel: fp16-fixed | official bf16 | fp32 (ComfyUI, seed 11).
 `LPIPS*` is the distance to the fp32 panel.
 
 ### Base
@@ -234,8 +224,7 @@ Each panel: fp16 without the fix | fp16 with the fix | bf16 | fp32 (ComfyUI, see
 </div>
 
 Every runtime, both seeds and the adapter panels are in `images/gallery/`
-(`<variant>_<runtime>_p<prompt>_s<seed>.webp`, `<variant>_<runtime>_lora<i>_p<prompt>_s11.webp`),
-all in the same four columns.
+(`<variant>_<runtime>_p<prompt>_s<seed>.webp`).
 
 ## Whole pipeline in fp16
 
@@ -250,23 +239,21 @@ transformer + fp16 VAE) is what the diffusers snippet below runs.
 ## Adapters
 
 A LoRA (rank 8) and a LoKr (factor 16), trained on the official Base weights in
-KohakuMoss, applied as-is to the fixed weights. Distance of each base + adapter to fp32 +
-adapter, then between fixed + adapter and official bf16 + adapter, next to the adapter's
-own effect:
-
-| runtime | variant | adapter | fp16, no fix | fp16-fixed | bf16 | fixed + adapter vs bf16 + adapter | without adapter | adapter effect |
-| --- | --- | --- | --- | --- | --- | --- | --- | --- |
-| ComfyUI | Base | LoRA | @@L_comfyui_base_lora0@@ | 0.056 | 0.047 | 0.287 |
-| ComfyUI | Base | LoKr | @@L_comfyui_base_lora1@@ | 0.064 | 0.047 | 0.389 |
-| ComfyUI | SFT | LoRA | @@L_comfyui_sft_lora0@@ | 0.039 | 0.042 | 0.284 |
-| ComfyUI | SFT | LoKr | @@L_comfyui_sft_lora1@@ | 0.039 | 0.042 | 0.395 |
-| KohakuMoss | Base | LoRA | @@L_kohakumoss_base_lora0@@ | 0.105 | 0.090 | 0.316 |
-| KohakuMoss | Base | LoKr | @@L_kohakumoss_base_lora1@@ | 0.053 | 0.090 | 0.406 |
-| KohakuMoss | SFT | LoRA | @@L_kohakumoss_sft_lora0@@ | 0.053 | 0.072 | 0.288 |
-| KohakuMoss | SFT | LoKr | @@L_kohakumoss_sft_lora1@@ | 0.042 | 0.072 | 0.417 |
-
-(LPIPS. The first three columns: 8 images each, seed 11, against fp32 + adapter run on a
-V100; the last three: 16 images each, RTX 5090.) The adapter changes the image 3-10x more
+KohakuMoss, applied as-is to the fixed weights. Distance between fixed + adapter and
+official bf16 + adapter, next to the adapter's own effect:
+
+| runtime | variant | adapter | fixed + adapter vs bf16 + adapter | without adapter | adapter effect |
+| --- | --- | --- | --- | --- | --- |
+| ComfyUI | Base | LoRA | 0.056 | 0.047 | 0.287 |
+| ComfyUI | Base | LoKr | 0.064 | 0.047 | 0.389 |
+| ComfyUI | SFT | LoRA | 0.039 | 0.042 | 0.284 |
+| ComfyUI | SFT | LoKr | 0.039 | 0.042 | 0.395 |
+| KohakuMoss | Base | LoRA | 0.105 | 0.090 | 0.316 |
+| KohakuMoss | Base | LoKr | 0.053 | 0.090 | 0.406 |
+| KohakuMoss | SFT | LoRA | 0.053 | 0.072 | 0.288 |
+| KohakuMoss | SFT | LoKr | 0.042 | 0.072 | 0.417 |
+
+(LPIPS, 16 images each.) The adapter changes the image 3-10x more
 than the fix does, and fixed + adapter stays about as close to bf16 + adapter as the two
 bases are without it: it transfers.
 
@@ -275,10 +262,10 @@ bases are without it: it transfers.
 <input type="radio" name="gl" id="l1"><label for="l1">Base LoKr</label>
 <input type="radio" name="gl" id="l2"><label for="l2">SFT LoRA</label>
 <input type="radio" name="gl" id="l3"><label for="l3">SFT LoKr</label>
-<div class="mf-panel" id="pl0"><img src="images/gallery/base_comfyui_lora0_p3_s11.webp" alt=""><p>fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)</p></div>
-<div class="mf-panel" id="pl1"><img src="images/gallery/base_comfyui_lora1_p3_s11.webp" alt=""><p>fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)</p></div>
-<div class="mf-panel" id="pl2"><img src="images/gallery/sft_comfyui_lora0_p3_s11.webp" alt=""><p>fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)</p></div>
-<div class="mf-panel" id="pl3"><img src="images/gallery/sft_comfyui_lora1_p3_s11.webp" alt=""><p>fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)</p></div>
+<div class="mf-panel" id="pl0"><img src="images/gallery/base_comfyui_lora0_p3_s11.webp" alt=""><p>fp16-fixed + adapter | official bf16 + adapter | official bf16 without it (ComfyUI)</p></div>
+<div class="mf-panel" id="pl1"><img src="images/gallery/base_comfyui_lora1_p3_s11.webp" alt=""><p>fp16-fixed + adapter | official bf16 + adapter | official bf16 without it (ComfyUI)</p></div>
+<div class="mf-panel" id="pl2"><img src="images/gallery/sft_comfyui_lora0_p3_s11.webp" alt=""><p>fp16-fixed + adapter | official bf16 + adapter | official bf16 without it (ComfyUI)</p></div>
+<div class="mf-panel" id="pl3"><img src="images/gallery/sft_comfyui_lora1_p3_s11.webp" alt=""><p>fp16-fixed + adapter | official bf16 + adapter | official bf16 without it (ComfyUI)</p></div>
 </div>
 
 Adapters on the joint blocks' q / k / v / o / `ffn_up_gate` / `ffn_down` (the usual
@@ -342,10 +329,7 @@ See [KohakuMoss](https://github.com/KohakuBlueleaf/KohakuMossTrainer).
 ## Notes
 
 - Measured on an RTX 5090 (fp16 and bf16 matmuls with fp32 accumulation); the fp32
-  reference is the official weights in fp32. The "fp16, no fix" column, the fp32 +
-  adapter panels and the 2560^2 example ran on a V100 (bf16 emulated there); KohakuMoss
-  and diffusers use the RTX 5090's initial noise per seed, since torch's CUDA generator
-  draws different noise on different GPUs.
+  reference is the official weights in fp32. The 2560^2 example ran in ComfyUI on a V100.
 - The headroom figures cover the prompts, resolutions and sigmas above, and 2560^2 for
   the fixed Base weights (15.6x). Larger images, very long prompts and finetuned weights
   were not measured.