MOSS-Image Base & heal on Turing (INT8 + FP16)

MOSS-Image-Base and MOSS-Image-heal don't runs at full speed on a Turing GPU (Ex: RTX 2060 SUPER) because they don't have native BF16. On Turing, BF16 is emulated via FP32, and ComfyUI's default loader keeps INT8 linears on INT8 but runs everything else (attention, norms, RoPE, residual) in FP32. I collected the info and weights needed to run it as INT8 weights + FP16 everything else, and what to expect in terms of visual diff.

Big thanks to KBlueLeaf for KBlueLeaf/MOSS-Image-FP16-fixed & KBlueLeaf/MOSS-Image-heal.


Files involved

File Origin Contents Role
moss_image_base_int8_convrot.safetensors Comfy-Org 168Γ— I8 (layers.N.proj.{qkv,o,ffn_up_gate,ffn_down}), rest BF16 official base INT8 (unfolded)
moss_image_base_fp16_fixed.safetensors KBlueLeaf/MOSS-Image-FP16-fixed all F16 (base weights folded Γ—c = 2⁻⁴) base-folded weights; outputs identical to the official base up to FP16 rounding
moss_image_base_int8_fp16.safetensors local splice 168Γ— I8 official + 210 F16 from FP16-fixed official base (folded) INT8 + FP16 β€” stream peak ~3.9k
moss_image_heal_int8_convrot.safetensors KBlueLeaf/MOSS-Image-heal 168Γ— I8, rest BF16 heal INT8
moss_image_heal_fp16.safetensors KBlueLeaf/MOSS-Image-heal all F16 (no fold; global max abs 12.69) heal weights
moss_image_heal_int8_fp16.safetensors local splice 168Γ— I8 from heal INT8 + 210 F16 from heal FP16 heal INT8 + FP16 β€” stream peak ~3.1k

Local splice (recipe)

INT8 official + non-quantized part taken from the FP16 of the same model β€” byte-level offset copy (no re-quantization):

I8 codes + weight_scale F32 + comfy_quant U8 + comfy_attention.config U8: from official INT8
210 BF16 β†’ F16 + rope.bands F32: from the corresponding FP16

Technical notes:

  • The fold (c=2⁻⁴) does not touch the 168 INT8 linears β†’ the official INT8 codes are exactly the codes of the folded base.
  • rope.bands is bit-identical in all variants.
  • The INT8 tensors are rotated by ConvRot (Hadamard 256, H^4 of H4/2, entries Β±1/16). The correct comparison is against the dense weight with the same rotation applied; with that rotation, INT8-vs-dense error is 0.85–0.94 % rel. L2 β€” the expected INT8 quantization error. (A direct, unrotated comparison gives ~1.37 and is not an error.)
  • video_embedder.weight drops to 0.111 in the base splice vs 1.77 in the official INT8 β†’ confirms the resulting model carries the folded weights (stream peak ~3.9k).

Benchmarks (same test for all runs)

Model Loader s/it Comment
moss_image_heal_int8_fp16 MOSS Image Loader (FP16) node 4.76 full fast path
moss_image_heal_int8_convrot default 6.58 native INT8 linears, rest in FP32
moss_image_base_int8_convrot default 6.64 official base, compute FP32
moss_image_base_int8_convrot MOSS Image Loader (FP16) node + clamp ON 4.80 official (unfolded) weights in FP16, residual clamp saturates at 65504
moss_image_base_int8_fp16 MOSS Image Loader (FP16) node + clamp ON 4.78 folded weights (stream peak ~3.9k), clamp not needed

Visual diff

Common setup for all tests: 25 steps Β· seed 614001632771110 Β· 1936Γ—1088 (16:9, 2 MP) Β· CFG 3 Β· scheduler simple Β· sampler ER-SDE (max-stage 3, eta 1.0, s_noise 1.10) Β· SLG off Β· no LoRAs Β· text encoder qwen3.5_9b_w4a8_gptq Β· VAE flux2-vae_small.

Prompt A (dog - day)

base INT8 official (BF16 -> FP32) base INT8 FP16 + clamp (node) base INT8_FP16 fold (node) heal INT8 (BF16 -> FP32) heal INT8 + FP16 (node)
file moss_image_t2i_00027_.webp moss_image_t2i_00032_.webp moss_image_t2i_00036_.webp moss_image_t2i_00026_.webp moss_image_t2i_00025_.webp
image base INT8 base INT8 FP16 clamp base INT8_FP16 folded heal INT8 heal INT8+FP16

Prompt B (ducks - night)

base INT8 official (BF16 -> FP32) base INT8 FP16 + clamp (node) base INT8_FP16 fold (node) heal INT8 (BF16 -> FP32) heal INT8 + FP16 (node)
file moss_image_t2i_00028_.webp moss_image_t2i_00033_.webp moss_image_t2i_00035_.webp moss_image_t2i_00029_.webp moss_image_t2i_00030_.webp
image base INT8 base INT8 FP16 clamp base INT8_FP16 folded heal INT8 heal INT8+FP16

MOSS Loader (FP16) node

Node: https://github.com/mkrgg/ComfyUI-MossFP16

The node ComfyUI-MossFP16 replaces MOSSImage.supported_inference_dtypes during load (equivalent to the dtypes half of ComfyUI #17684, without editing files) and registers its own cached_patcher_init so reloads keep the override. Optional clamp_FP16 toggle that adds a per-block hook doing x.clamp_(-65504, 65504) β€” a safety net for the unfolded base weights (not needed for FP16-fixed / heal weights).

When does MOSS go NaN (black image) in FP16?

Only with moss_image_base_int8_convrot in FP16 mode. Three measured points:

Resolution MP Stream peak Status
1024Β² 1.05 61.9k ok (288 runs, none non-finite)
1536Β² 2.36 62.6k ok
1936Γ—1088 2.11 ~62.6k (fit) ok
2048Β² 4.19 64.3k ok (1.8% margin)
2240Β² 5.02 64.9k (fit) ok (1%)
2368Β² 5.61 65.3k (fit) risky depending on seed
2496Β² 6.23 65.8k (fit) OVERFLOW
2560Β² 6.55 65,504 β†’ black at step 29 of 50 (44/100 non-finite forwards) OVERFLOW
3072Β² 9.44 68.3k (fit) OVERFLOW

Least-squares linear fit: peak β‰ˆ 60.97k + 0.773k Γ— MP (residuals Β±0.2k). Critical point 5.86 MP (2420Β²). The first non-finite forward lands at roughly 56–58% of the schedule (29/50 β‰ˆ 58%, 56/100 β‰ˆ 56%) β†’ with 25 ER-SDE steps expect it around step 14–15.

Caveats:

  • Variance of Β±0.5–1% by seed/prompt/stochasticity β†’ the boundary moves by a few dozen pixels per side.
  • CFG barely affects it (<0.5%); the area is what matters. Long prompts also don't matter (Β±1%).
  • With clamp: no NaN β€” the stream saturates at Β±65,504 β†’ artifacts, not black.
  • LoRA / finetunes raise the stream: the 2048Β² margin is just 1.8%, a LoRA adding even +5–10% can push it to black; LoRA + large images β†’ go with the folded base or the heal.
  • With the folded base (moss_image_base_INT8_FP16) there is ~16Γ— headroom β€” this failure mode does not exist.

References

AI

The custom node and the README was created using Inklink 2 Small.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for mkrgg/MOSS-Image-Base-INT8-FP16-Turing

Finetuned
(1)
this model