| --- |
| license: apache-2.0 |
| base_model: OpenMOSS-Team/MOSS-Image-SFT |
| base_model_relation: quantized |
| pipeline_tag: text-to-image |
| library_name: gguf |
| tags: [gguf, rocmfpx, amd, rocm, moss-image, stable-diffusion.cpp, quantized, text-to-image] |
| --- |
| |
| # MOSS-Image-SFT — ROCmFPX GGUF |
|
|
| The 5B diffusion transformer of [OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT) |
| for [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), in `Q8_0_ROCMFPX`, the 8-bit |
| [ROCmFPX](https://github.com/charlie12345/ROCmFPX) format for AMD GPUs. |
|
|
| > This file needs stable-diffusion.cpp built against the ROCmFPX fork of ggml. Stock sd.cpp, ComfyUI-GGUF and other |
| > runners do not know its type. The patch is in this repo (`sdcpp-rocmfpx.patch`). |
| > |
| > **Not on AMD with ROCmFPX?** Use a stock `Q8_0`, for example from |
| > [leejet/MOSS-Image-SFT-GGUF](https://huggingface.co/leejet/MOSS-Image-SFT-GGUF). In the test below a stock `Q8_0` |
| > made the same way as this file is closer to BF16 and only 0.15 GB larger. This file is for trying the ROCmFPX |
| > kernels on AMD. |
| > |
| > Not an official release of OpenMOSS. All credit for the model goes to the OpenMOSS team. |
|
|
|  |
|
|
| ## File |
|
|
| | File | Size | | |
| |---|---:|---| |
| | `MOSS-Image-SFT-Q8_0_ROCMFPX.gguf` | 5.56 GB | the diffusion transformer only; text encoder and VAE come from the original repo | |
|
|
| Only the 252 main projections are quantized (attention q/k/v/o, `ffn_up_gate`, `ffn_down` in every block). Embedders, |
| norms, modulation, time conditioning and the output layer stay BF16, as in the reference MOSS GGUFs. |
|
|
| ## Measured quality |
|
|
| 8 prompts (photos, illustration, two with written text), 1024×1024, Euler, 30 steps, CFG 3.0, flow shift 5, one fixed |
| seed per prompt. Every image is compared with the BF16 image of the same prompt and seed. PSNR (dB) and SSIM: higher |
| is closer. LPIPS: lower is closer (perceptual distance). Diffusion amplifies small weight changes over the steps, so a |
| quantized model can give a different take of the same prompt (for example other words on a sign) without any loss of |
| quality. |
|
|
| | Weights | Size | PSNR | SSIM | LPIPS | | |
| |---|---:|---:|---:|---:|---| |
| | BF16 loaded directly (floor of this test) | 10.2 GB | 33.4 | 0.974 | 0.022 | original | |
| | **`Q8_0_ROCMFPX`** | 5.56 GB | 26.6 | 0.916 | 0.074 | **this repo** | |
| | `Q8_0` (stock ggml, same tensors) | 5.71 GB | 28.9 | 0.949 | 0.043 | baseline, not published here | |
| | `Q6_0_ROCMFPX` | 4.52 GB | 22.1 | 0.850 | 0.126 | not published | |
| | `Q4_0_ROCMFP4` | 3.33 GB | 16.5 | 0.665 | 0.294 | not published | |
| | `Q4_0_ROCMFP4_FAST` | 3.18 GB | 15.5 | 0.655 | 0.323 | not published | |
| | `Q4_0_ROCMI4` | 3.18 GB | 16.1 | 0.639 | 0.326 | not published | |
|
|
| The 6- and 4-bit files change the images too much (mostly different compositions, and text that changes) to be |
| worth publishing for a model this small. |
|
|
| ## How this was made |
|
|
| 1. stable-diffusion.cpp (`b13a482`, which supports MOSS-Image) built on the ROCmFPX ggml (`fb08d7c`) with |
| `sdcpp-rocmfpx.patch`. |
| 2. Importance matrix from the BF16 transformer over 16 calibration prompts (different from the test prompts), same |
| settings. |
| 3. `sd-cli -M convert --tensor-type-rules '^layers\..*\.proj\.(q|k|v|o|ffn_up_gate|ffn_down)\.weight$=q8_0_rocmfpx' --imat-in imatrix.dat` |
| 4. Quality measured on an NVIDIA GPU with every transformer weight expanded back to full precision and rendered with |
| sd.cpp, so the numbers measure the stored weights. The fork's own GPU kernels for these types were not used (they |
| are wrong on NVIDIA). Text encoder for all rows: the original BF16 Qwen3.5 9B from the MOSS repo. AMD GPU kernels |
| compute the same math in a different order, so results on an AMD GPU can differ slightly. |
| **This file has not been tested on AMD hardware yet.** |
|
|
| ## Run |
|
|
| ```bash |
| git clone https://github.com/charlie12345/ROCmFPX |
| git clone https://github.com/leejet/stable-diffusion.cpp sdcpp && cd sdcpp && git checkout b13a482 |
| git submodule update --init --recursive && git apply ../sdcpp-rocmfpx.patch |
| cmake -B build -DSD_USE_UPSTREAM_GGML=ON -DSD_GGML_SOURCE_DIR=$PWD/../ROCmFPX/ggml -DSD_ROCMFPX=ON -DSD_VULKAN=ON |
| cmake --build build -j |
| build/bin/sd-cli --diffusion-model MOSS-Image-SFT-Q8_0_ROCMFPX.gguf \ |
| --llm Qwen3.5-9B-Q8_0.gguf --vae flux2_ae.safetensors --tokenizer tokenizer.json \ |
| --cfg-scale 3.0 --steps 30 -W 1024 -H 1024 -p "a lovely cat holding a sign that says 'ROCmFPX'" -o cat.png |
| ``` |
|
|
| Text encoder, VAE and tokenizer: see |
| [sd.cpp's MOSS-Image guide](https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/moss_image.md) |
| (Qwen3.5 9B GGUF, the FLUX.2 VAE, and `tokenizer.json` from the MOSS repo; the external tokenizer is required). |
| Use image sizes divisible by 16. |
|
|
| ## License |
|
|
| Apache-2.0, same as the base model (`LICENSE`). |
|
|
