README.md

---
license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Image-SFT
base_model_relation: quantized
pipeline_tag: text-to-image
library_name: gguf
tags: [gguf, rocmfpx, amd, rocm, moss-image, stable-diffusion.cpp, quantized, text-to-image]
---

# MOSS-Image-SFT — ROCmFPX GGUF

The 5B diffusion transformer of [OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT)
for [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), in `Q8_0_ROCMFPX`, the 8-bit
[ROCmFPX](https://github.com/charlie12345/ROCmFPX) format for AMD GPUs.

> This file needs stable-diffusion.cpp built against the ROCmFPX fork of ggml. Stock sd.cpp, ComfyUI-GGUF and other
> runners do not know its type. The patch is in this repo (`sdcpp-rocmfpx.patch`).
>
> **Not on AMD with ROCmFPX?** Use a stock `Q8_0`, for example from
> [leejet/MOSS-Image-SFT-GGUF](https://huggingface.co/leejet/MOSS-Image-SFT-GGUF). In the test below a stock `Q8_0`
> made the same way as this file is closer to BF16 and only 0.15 GB larger. This file is for trying the ROCmFPX
> kernels on AMD.
>
> Not an official release of OpenMOSS. All credit for the model goes to the OpenMOSS team.

![BF16 vs Q8_0_ROCMFPX, same prompt and seed](comparison.jpg)

## File

| File | Size | |
|---|---:|---|
| `MOSS-Image-SFT-Q8_0_ROCMFPX.gguf` | 5.56 GB | the diffusion transformer only; text encoder and VAE come from the original repo |

Only the 252 main projections are quantized (attention q/k/v/o, `ffn_up_gate`, `ffn_down` in every block). Embedders,
norms, modulation, time conditioning and the output layer stay BF16, as in the reference MOSS GGUFs.

## Measured quality

8 prompts (photos, illustration, two with written text), 1024×1024, Euler, 30 steps, CFG 3.0, flow shift 5, one fixed
seed per prompt. Every image is compared with the BF16 image of the same prompt and seed. PSNR (dB) and SSIM: higher
is closer. LPIPS: lower is closer (perceptual distance). Diffusion amplifies small weight changes over the steps, so a
quantized model can give a different take of the same prompt (for example other words on a sign) without any loss of
quality.

| Weights | Size | PSNR | SSIM | LPIPS | |
|---|---:|---:|---:|---:|---|
| BF16 loaded directly (floor of this test) | 10.2 GB | 33.4 | 0.974 | 0.022 | original |
| **`Q8_0_ROCMFPX`** | 5.56 GB | 26.6 | 0.916 | 0.074 | **this repo** |
| `Q8_0` (stock ggml, same tensors) | 5.71 GB | 28.9 | 0.949 | 0.043 | baseline, not published here |
| `Q6_0_ROCMFPX` | 4.52 GB | 22.1 | 0.850 | 0.126 | not published |
| `Q4_0_ROCMFP4` | 3.33 GB | 16.5 | 0.665 | 0.294 | not published |
| `Q4_0_ROCMFP4_FAST` | 3.18 GB | 15.5 | 0.655 | 0.323 | not published |
| `Q4_0_ROCMI4` | 3.18 GB | 16.1 | 0.639 | 0.326 | not published |

The 6- and 4-bit files change the images too much (mostly different compositions, and text that changes) to be
worth publishing for a model this small.

## How this was made

1. stable-diffusion.cpp (`b13a482`, which supports MOSS-Image) built on the ROCmFPX ggml (`fb08d7c`) with
   `sdcpp-rocmfpx.patch`.
2. Importance matrix from the BF16 transformer over 16 calibration prompts (different from the test prompts), same
   settings.
3. `sd-cli -M convert --tensor-type-rules '^layers\..*\.proj\.(q|k|v|o|ffn_up_gate|ffn_down)\.weight$=q8_0_rocmfpx' --imat-in imatrix.dat`
4. Quality measured on an NVIDIA GPU with every transformer weight expanded back to full precision and rendered with
   sd.cpp, so the numbers measure the stored weights. The fork's own GPU kernels for these types were not used (they
   are wrong on NVIDIA). Text encoder for all rows: the original BF16 Qwen3.5 9B from the MOSS repo. AMD GPU kernels
   compute the same math in a different order, so results on an AMD GPU can differ slightly.
   **This file has not been tested on AMD hardware yet.**

## Run

```bash
git clone https://github.com/charlie12345/ROCmFPX
git clone https://github.com/leejet/stable-diffusion.cpp sdcpp && cd sdcpp && git checkout b13a482
git submodule update --init --recursive && git apply ../sdcpp-rocmfpx.patch
cmake -B build -DSD_USE_UPSTREAM_GGML=ON -DSD_GGML_SOURCE_DIR=$PWD/../ROCmFPX/ggml -DSD_ROCMFPX=ON -DSD_VULKAN=ON
cmake --build build -j
build/bin/sd-cli --diffusion-model MOSS-Image-SFT-Q8_0_ROCMFPX.gguf \
  --llm Qwen3.5-9B-Q8_0.gguf --vae flux2_ae.safetensors --tokenizer tokenizer.json \
  --cfg-scale 3.0 --steps 30 -W 1024 -H 1024 -p "a lovely cat holding a sign that says 'ROCmFPX'" -o cat.png
```

Text encoder, VAE and tokenizer: see
[sd.cpp's MOSS-Image guide](https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/moss_image.md)
(Qwen3.5 9B GGUF, the FLUX.2 VAE, and `tokenizer.json` from the MOSS repo; the external tokenizer is required).
Use image sizes divisible by 16.

## License

Apache-2.0, same as the base model (`LICENSE`).