MOSS-Image-SFT: quantized GGUF (study results)
GGUF quantizations of OpenMOSS-Team/MOSS-Image-SFT (revision
f972bd4): the 5.08 B DiT and its 9 B text encoder (stock Qwen3.5-9B). They come from a study that asked one question:
what is the smallest model whose images stay as close as possible to the released model's, for the same prompt and
seed?
The repo holds all 37 recipes the study produced, the ones that passed and the ones that didn't, with the measured result for each. Most people want one of the two picks:
| pick | DiT | text encoder | total | what it keeps |
|---|---|---|---|---|
| Faithful | dit/moss_image_sft-dit-w8-gptq.gguf (5.40 GB) |
text_encoder/moss_image_sft-te-w8-gptq.gguf (8.45 GB) |
13.8 GB (53% of 26 GB) | Same pictures as the released model. Closer to FP32 than bf16 inference itself is (0.43x its drift). |
| Equivalent quality | dit/moss_image_sft-dit-w8-gptq.gguf (5.40 GB) |
text_encoder/moss_image_sft-te-cal-mix6.gguf (5.97 GB) |
11.4 GB (44%) | Same quality and text rendering; a few images on "fork" prompts change composition. |
No DiT below 8 bits is faithful. That includes our own Q6_K versions (imatrix and GPTQ) and the community Q6_K and Q4_K_M files. One passes the quality bar: the community Q6_K DiT (molbal's, plain rounding) with our W8 text encoder, 13.0 GB. That is larger than the quality pick. The text encoder can go lower than the DiT.
1024², 50 steps, seed 0, three of the held-out prompts. Columns: FP32 compute on the released weights (the reference), the released model in bf16 compute, and the two picks. LPIPS to the reference, mean over these three: bf16 0.044, faithful pick 0.013 (0.30x), quality pick 0.031 (0.70x). Every quoted string is rendered in all four.
Where smaller files break
512², 20 steps, held-out prompts, seeds as labelled. LPIPS to the FP32 reference under each image. Below 8 bits in the DiT, or below the cal-mix6 text encoder, the first thing to go is small text. On the calligraphy prompt the red seal should read "墨香"; our Q6_K (imatrix) DiT, the community Q4_K_M DiT and the 9.3 GB pair all turn it into an unreadable stamp, although the rest of the image barely changes (row 2). Q4_K_M also writes the traditional "書" for "书" and redraws whole scenes (rows 4-5). The two-person prompt (row 3) sits near a fork: bf16 itself drifts (0.15). There the quality pick (11.4 GB) draws two children instead of a girl and a man. That is why it is a quality pick and not a faithful one. The faithful pick stays at 0.01-0.03 on every row.
GB means 10⁹ bytes here. The VAE (FLUX.2 small decoder, 0.25 GB) is not included; use the original one.
Using the files
- ComfyUI with ComfyUI-GGUF: load the DiT with Unet Loader (GGUF) and the
text encoder with the GGUF CLIP loader (Qwen3.5 /
qwen35support). The DiT files use the same tensor layout as existing MOSS-Image GGUFs (e.g. molbal's). Files whereq/k/vshare one type (uniform and rule recipes) have fusedqkv; the mixes keep them separate because they differ in type. The text-encoder files use llama.cpp'sqwen35layout with separateattn_qkv/attn_gate/ssm_alpha/ssm_beta, no LM head, and Qwen3.5-9B's tokenizer and metadata. - Tensor types are standard ggml types only: Q3_K, Q4_0, Q4_K, Q5_K, Q6_K, Q8_0, BF16, F32. No custom kernels are needed. GPTQ and imatrix change which codes are stored, not the format.
- Low-VRAM GPUs: run the text encoder once per prompt, then the DiT, then the VAE. On an 8 GB card the faithful DiT (5.40 GB) fits. The 8.45 GB text encoder does not, so run it on the CPU or offload it.
What was and wasn't checked
- Every file was read back after writing and compared with the quantized weights the study measured: exact for
every tensor. Two small exceptions, both listed in the per-file reports: scales of plain round-to-nearest DiT layers
are stored as fp16 (relative error ≤ 5.2e-8). In text encoders with K-quant
out_proj(ssm_out) tensors, those are re-quantized, because llama.cpp's head order shifts the 256-wide blocks. Their codes differ from the measured ones, at the same error level (e.g. 1.41e-3 vs 1.41e-3 relative). This affects 3-23 tensors per file, including 12 in the quality pick'ste-cal-mix6. The faithful pick'ste-w8-gptqhas none. - DiT end to end: images made from the exported
dit-w8-gptqanddit-cal-mix6files are pixel-identical to the ones measured. - Quality pick's text encoder end to end: the text encoder was loaded from
te-cal-mix6.ggufitself (237 tensors bit-identical to the measured weights, the 12 re-quantized ones taken from the file). Its embeddings are as close to FP32 as the measured ones (hidden-state error 3.326e-3 vs 3.328e-3). They differ from them by 1.1e-4 relative, less than bf16 compute adds. Eight images (seed 0 on all six prompts, plus the two closest calls) stay within 0.001-0.036 LPIPS of the measured ones. All quoted strings are kept, and the "墨香" seal still reads. The composition change on the two-person scene (seed 1) is still there: 0.298 to FP32, measured 0.301, limit 0.301. That is the same picture either side of the line, so the pick's verdicts stand. The other text-encoder files were checked by read-back only. - Not checked: running these files inside ComfyUI, or llama.cpp itself. All measurements used a PyTorch harness that dequantizes each weight and computes in FP32. ComfyUI-GGUF also dequantizes on the fly, but computes in bf16/fp16, so its images will differ slightly, as the released model's bf16 images differ from FP32.
How "close" was measured
- Reference: FP32 compute on the released bf16 weights. This is also exactly how the model runs on GPUs without bf16 (e.g. Pascal). Yardstick: the released model in bf16 compute, i.e. what most people run. Its own drift from FP32 sets the bar.
- Images: 6 held-out prompts (photo, English sign text, Chinese calligraphy with a seal, watercolour, a JSON layout prompt, a two-person scene) x seeds 0-3 = 24 paired images per variant, 512², 20 steps. The picks were confirmed at 1024², 50 steps. The prompts were written after the recipes were fixed and were never used to tune them.
- Faithful (set before results): mean LPIPS to the reference ≤ 1.5x bf16's; each image ≤ max(2x bf16's on that image, 0.1); and OCR still finds every quoted string the reference renders. As a scale: the same prompt with another seed is 0.36-0.66 LPIPS; a changed composition ("flip") lands at 0.25-0.33.
- Equivalent quality (reference-free, as quantization papers report it): ImageReward delta to the reference, lower end of its 95% CI ≥ -0.05, no image below -0.3, and all quoted strings kept (OCR, every miss checked by eye; 3 of 10 were false alarms). CLIPScore was measured too; it separated nothing here.
- Screening (files marked "screened only" below): the error each variant adds to the DiT's predicted velocity (DiTs), or to the text encoder's hidden states (TEs), relative to what bf16 compute adds. That is a filter, not a verdict: it predicted the clear failures but not which images flip.
Caveats: 24 images on 6 prompts is a small sample, and the per-image limit is fragile on prompts where the model sits near a fork (bf16 itself changes one of the 24 compositions). The rule was never loosened after results came in.
All files
"x bf16" is LPIPS to the reference relative to bf16's (0.048); flips = images with a changed composition out of 24. IR = ImageReward delta, mean ± 95% CI and worst image.
DiT (dit/)
| file | GB | types (linear layers) | result |
|---|---|---|---|
| dit-w8-gptq | 5.40 | Q8_0 (GPTQ codes) | Faithful pass with te-w8-gptq: 0.43x bf16, 0 flips, IR -0.007 ± 0.014 (worst -0.11). Community Q8_0 (plain rounding) for comparison: 0.48x, also passes. |
| dit-rule-w5-w8 | 4.18 | Q5_K body, Q8_0 sensitive layers, small embedders BF16 | Screened only: 2.1x bf16's velocity error. |
| dit-q6k-ik | 4.17 | Q6_K (imatrix) | Fail: with te-w8-gptq 1.58x, 3 flips; the "墨香" seal lost on one image. IR -0.005 ± 0.024. Community Q6_K (plain rounding) for comparison: 1.86x, fails the faithful rule but keeps the seal and passes the quality bar with te-w8-gptq (13.0 GB). |
| dit-q6k-gptq | 4.17 | Q6_K (GPTQ codes) | Fail: 1.64x, IR worst -0.56. With te-cal-mix6: 1.72x, seal garbled. |
| dit-mix6.25 | 3.97 | mix Q4_K-Q8_0 | Fail: with te-w8-gptq 2.43x, 5 flips, seal lost. |
| dit-cal-mix6 | 3.81 | mix Q4_0-Q8_0 (calibrated) | Fail: with te-cal-mix5.5 (9.3 GB) 2.06x, 4 flips, seal garbled; IR within noise (-0.009 ± 0.037). |
| dit-mix6 | 3.81 | mix Q4_K-Q8_0 | Screened only: 1.2x bf16's velocity error. |
| dit-mix5.75 | 3.65 | mix Q4_K-Q8_0 | Screened only: 1.6x. |
| dit-rule-w4-w6 | 3.50 | Q4_K body, Q6_K sensitive layers, small embedders BF16 | Screened only: 7.2x. |
| dit-cal-mix5.5 | 3.50 | mix (calibrated) | Fail: redraws the sign prompt (seed 0, LPIPS 0.23). |
| dit-mix5.5 | 3.49 | mix | Screened only: 2.0x. |
| dit-cal-mix5 | 3.18 | mix (calibrated) | Fail: redraws the sign prompt (0.36). |
| dit-mix5 | 3.18 | mix | Screened only: 3.5x. |
| dit-cal-mix4.5 | 2.86 | mix Q3_K-Q8_0, mostly Q4_0 (calibrated) | Fail: redraws the sign prompt (0.33). |
Text encoder (text_encoder/)
All results below are with the W8 DiT unless noted. The token table counts in the size.
| file | GB | types | result |
|---|---|---|---|
| te-mix11 | 10.92 | Q8_0 / BF16 | Screened only: 0.45x bf16's hidden-state error. |
| te-mix10 | 9.93 | Q8_0 / BF16 | Screened only: 0.57x. |
| te-rule-w8-bf16 | 9.53 | Q8_0, sensitive layers BF16 | Screened only: 0.78x. |
| te-mix9 | 8.94 | Q8_0 / BF16 | Screened only: 0.73x. |
| te-mix8.75 | 8.69 | Q8_0 / BF16 / Q6_K | Screened only: 1.7x. |
| te-w8-gptq | 8.45 | Q8_0 (GPTQ codes), 6 tiny tensors BF16 | Faithful pass (see the DiT pick). Hidden-state error below bf16's. |
| te-uniform-q8 | 8.45 | Q8_0 (plain rounding) | Images on 2 prompts only: same pictures. Use te-w8-gptq instead: same size, ~4x lower hidden-state error. |
| te-mix8.5 | 8.45 | Q8_0 / BF16 / Q6_K | Screened only: 1.6x. |
| te-mix8.25 | 8.20 | Q8_0 / BF16 / Q6_K | Screened only: 1.8x. |
| te-rule-w6-bf16 | 8.13 | Q6_K, sensitive layers BF16 | Screened only: 9.3x. |
| te-mix8 | 7.95 | Q8_0 / Q6_K / BF16 | Screened only: 2.6x. |
| te-cal-mix7 | 6.96 | Q8_0 / Q5_K (calibrated) | Quality pass, faithful miss by one image (1.18x, 2 flips; one image at 0.312 vs its limit 0.301). IR +0.010 ± 0.017. |
| te-mix7 | 6.96 | Q8_0 / Q6_K / BF16 / Q5_K | Screened only: 5.9x. |
| te-cal-mix6.5 | 6.46 | Q8_0 / Q5_K (calibrated) | Screened only (through the DiT: between cal-mix6 and cal-mix7). |
| te-rule-w5-w8 | 6.28 | Q5_K, sensitive layers Q8_0 | Screened only: 41x. |
| te-cal-mix6 | 5.97 | Q5_K / Q8_0 (calibrated) | Quality pass, the smallest: 1.30x, 2 flips; misses the faithful rule on two images of the two-person scene. IR +0.006 ± 0.021. Same pictures at 1024². |
| te-mix6 | 5.97 | Q5_K / Q6_K / Q8_0 | Screened only: 23x; failed the through-DiT screen. |
| te-cal-mix5.5 | 5.47 | Q5_K / Q8_0 / Q4_0 (calibrated) | Fail: 1.78x, the "墨香" seal garbled, IR worst -0.43. |
| te-cal-mix5 | 4.97 | Q4_0 / Q5_K / Q8_0 (calibrated) | Fail: with dit-cal-mix6, one image over its limit (seed 0, pose changed). |
| te-mix5 | 4.97 | Q5_K / Q4_K / ... | Screened only: 83x. |
| te-e8w4 | 4.99 | Q4_0 (GPTQ codes), token table Q8_0 | Fail: clean images, but different pictures on every prompt tried (LPIPS 0.33-0.43). |
| te-cal-mix4.5 | 4.48 | Q4_0 / Q5_K / Q8_0 (calibrated) | Fail: 2.42x, 6 flips, seal garbled (IR still +0.041: a smaller TE changes the picture, not its quality). |
| te-cal-mix4 | 3.98 | Q4_0 / Q3_K / Q5_K / Q8_0 (calibrated) | Fail: with dit-cal-mix6, redraws the notebook prompt (seed 0). |
Recipe names
w8-gptq: every linear W8 GPTQ, group 32 (= Q8_0 layout).mixN: a per-tensor mix from a damage scan, average N bits per weight.cal-mixN: the same, with GPTQ / imatrix-calibrated levels (48 calibration prompts, disjoint from the test prompts).rule-A-B: a fixed rule (most layers A, the sensitive ones B).q6k-ik/q6k-gptq: uniform Q6_K with imatrix or GPTQ codes.e8w4: W4 GPTQ body, 8-bit token table.
Each file's exact per-tensor types are in its GGUF header.
License
Apache-2.0, as the source model (MOSS-Image-SFT; its text encoder is Qwen3.5-9B, also Apache-2.0). These are derived weights. All credit for the model goes to the OpenMOSS team and the Qwen team.
- Downloads last month
- 5,078
16-bit
Model tree for nynefold/MOSS-Image-SFT-quant-GGUF
Base model
OpenMOSS-Team/MOSS-Image-Base
