fp16 without the fix | fp16 with the fix | bf16 | fp32 (ComfyUI, 1024^2, seed 11).
## Why: the stream sits at 95% of the fp16 limit
MOSS-Image adds every block's output into one residual stream. On the official weights its
largest value is **62,089** (Base) / **61,828** (SFT) at block 26 of 42, measured in fp32
over 8 prompts + the empty prompt, 7 sigmas and 512^2, 1024^2 and 768x1344
(189 forwards), and 62,098 / 62,184 during real 50-step sampling. fp16 ends at 65,504.
Plain fp16 on the official weights did not overflow on any of these 189 forwards, and at
1024^2 its images look right (below). The margin is the problem: at block 26 it is 1.05x,
and the stream grows with the pixel count. Anything that raises it by 5% (a larger image,
a LoRA, a finetune) turns it into inf / NaN. The fixed weights keep 16.9x at the same
block.
fp16 without the fix | fp16 with the fix | bf16 | fp32 (KohakuMoss, 1024^2, prompt 0).
**Where plain fp16 breaks.** ComfyUI with `--fp16-unet` on the official Base file, 50
steps, CFG 3, euler / simple: all 8 prompts at 1024^2 finish (stream max 62,080), and so
does 2048^2 (64,096, 2.2% below the limit). **2560^2 is the first size that gives a
black image**: on step 29 of 50 the stream reaches 65,504 at block 26, every later
forward is NaN, and the image is black. KohakuMoss in plain fp16 goes non-finite at the
same step and block. The same run in fp32 peaks at 67,048 (a single fp32 forward at sigma
0.7: 67,681), 2.4-3.3% over the fp16 limit; bf16 peaks at 67,072 and stays
finite. The fixed weights peak at 4,188 (x 16 = 67,008, 15.6x headroom):
fp16 without the fix | fp16 with the fix | bf16 | fp32 (ComfyUI, 2560x2560, prompt 0
"tabby cat", seed 11; `LPIPS*` is the distance to fp32). In single forwards over 7
sigmas, 2560^2, 2048x3072, 3072x2048 and 3072^2 overflow; 1536^2, 2048^2, 768x2048 and
1024x4096 do not. A 226-token prompt moves the stream by under 1%. At 1024^2 no setting
tried breaks: Base and SFT x CFG 3 / 5 / 7 / 10 x the 8 card prompts x seeds 11-14, plus
4 extreme prompts (226 tokens, dense text, saturated high-contrast, white high-key) at
CFG 3 / 10, 288 ComfyUI runs, peak 62,208 (block 26), none non-finite; CFG and content
move the peak by under 0.5%, the pixel count moves it
([reports/plain_fp16_1024_hunt.json](reports/plain_fp16_1024_hunt.json)).
A photograph of a tabby cat sitting on a sunlit windowsill, shallow depth of field.
An isometric illustration of a small floating island with a lighthouse, waterfalls pouring off the edges into clouds, soft pastel colors, highly detailed.
A wooden sign on a forest trail that reads 'KOHAKU' in carved letters, morning fog.
An anime-style illustration of a girl reading a book in a quiet library, warm light.
A studio product photo of a ceramic teapot on a marble table, soft shadows.
A busy night market street in the rain, neon signs reflected on wet pavement.
An oil painting of a mountain lake at sunset with a small red boat.
A detailed pencil sketch of an old steam locomotive crossing a stone bridge.
A photograph of a tabby cat sitting on a sunlit windowsill, shallow depth of field.
An isometric illustration of a small floating island with a lighthouse, waterfalls pouring off the edges into clouds, soft pastel colors, highly detailed.
A wooden sign on a forest trail that reads 'KOHAKU' in carved letters, morning fog.
An anime-style illustration of a girl reading a book in a quiet library, warm light.
A studio product photo of a ceramic teapot on a marble table, soft shadows.
A busy night market street in the rain, neon signs reflected on wet pavement.
An oil painting of a mountain lake at sunset with a small red boat.
A detailed pencil sketch of an old steam locomotive crossing a stone bridge.
fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)
fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)
fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)
fp16 without the fix + adapter | fp16 with the fix + adapter | bf16 + adapter | fp32 + adapter (ComfyUI)