OzzyGT/MOSS_Image_SFT_sdnq_dynamic_8bit

MOSS-Image SFT SDNQ INT8

bf16	INT8

Left: original bf16 · Right: this INT8 model (same prompt and seed).

This is a dynamic 8-bit quantized version of OpenMOSS-Team/MOSS-Image-SFT using SDNQ. Both the transformer and the Qwen3.5 text encoder are quantized; the VAE is kept unquantized.

Note: You need SDNQ v0.2.8 or superior and diffusers from main (0.41.0.dev0 or later).

Usage

You can find ready-to-use scripts in the diffusers-recipes repository.

MOSS-Image runs as a Modular Diffusers pipeline built from the MOSS-Image custom blocks, which load the transformer code shipped in this repository (hence trust_remote_code and trust_components_code). The VAE runs in float32, like the original.

Note: both flags run Python code downloaded from the Hub: trust_remote_code=True runs the code of the blocks repository, and trust_components_code=True runs the code shipped in the components_repo. Only use them with repositories you trust, and review their code before passing different ones.

Import sdnq before loading so the quantized weights are registered. The whole pipeline fits on one GPU: the sample below peaked at 19.7 GiB of allocated VRAM at 2048x1152 (29.4 GB in nvidia-smi).

For prompting, recommended settings and the layout format, see the original model card and the user guide. The sample below uses a layout prompt: boxes are [y_min, x_min, y_max, x_max] in pixels with the origin at the center of the canvas.

Note: the samples use 28 steps, which is also the default of the custom blocks, because it already gives good results in about half the time. The official recommendation is 50 steps or more, so raise num_inference_steps if you want the best quality.

Sample image

The INT8 image above was generated with the following prompt and settings (seed 7):

import json

import sdnq  # noqa: F401
import torch
from diffusers import ModularPipelineBlocks


blocks = ModularPipelineBlocks.from_pretrained(
    "OzzyGT/moss_image_custom_blocks",
    trust_remote_code=True,
    components_repo="OzzyGT/MOSS_Image_SFT_sdnq_dynamic_8bit",
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
pipe.load_components(dtype={"default": torch.bfloat16, "vae": torch.float32})
pipe.to("cuda")

layout = {
    "summary": {
        "meta": ["photo"],
        "canvas": "single",
        "caption": (
            "A photograph of the inside of a small bicycle repair workshop, shot on a Sony A7 IV with a 35mm f/1.8 "
            "lens at f/2.8, 1/125 s, ISO 1600, handheld at eye level, wide 16:9 frame. Low late-afternoon sunlight "
            "comes through a large front window on the right in hard shafts with dust in the air, the rest of the "
            "room is lit by bare fluorescent tubes on the ceiling. Shallow depth of field with the back wall slightly "
            "out of focus, visible sensor grain. A red road bicycle is clamped in a repair stand in the center while "
            "an adult mechanic in a grease-stained navy work apron adjusts its rear derailleur. A painted wooden sign "
            'on the back wall reads "SPOKE & CHAIN". A pegboard of hand tools hangs on the left wall above an A-frame '
            "chalkboard with a price list, and an adult black cat sleeps curled up on a metal stool by the window. "
            "Oil stains on a grey concrete floor, spare tires hanging from hooks, scuffed white walls."
        ),
    },
    "items": [
        {
            "label": "text",
            "caption": "A long painted wooden sign on the back wall, dark green with cream serif capital letters.",
            "text": "SPOKE & CHAIN",
            "bbox": [-496, -464, -346, 236],
        },
        {
            "label": "object",
            "caption": "A perforated pegboard with wrenches, screwdrivers, tire levers and a chain whip on hooks.",
            "bbox": [-396, -984, 124, -544],
        },
        {
            "label": "object",
            "caption": "A red steel road bicycle with drop handlebars, clamped by its seat post in a grey repair stand.",
            "bbox": [-196, -504, 329, 371],
        },
        {
            "label": "object",
            "caption": (
                "An adult man in his forties with short grey hair, a grey t-shirt and a grease-stained navy work "
                "apron, standing and turning a small screwdriver on the rear derailleur."
            ),
            "bbox": [-371, 396, 504, 736],
        },
        {
            "label": "text",
            "caption": "A black A-frame chalkboard on the floor with a handwritten price list in white chalk.",
            "text": "FLAT REPAIR $12\nFULL TUNE-UP $45\nOPEN 8-6",
            "bbox": [184, -964, 524, -604],
        },
        {
            "label": "object",
            "caption": "An adult black cat sleeping curled up on the seat of a metal shop stool in a patch of sunlight.",
            "bbox": [104, 766, 209, 981],
        },
    ],
    "bbox_coord_space": "abs_centralized",
    "bbox_format": "yxyx",
}
prompt = json.dumps(layout, ensure_ascii=False)

image = pipe(
    prompt=prompt,
    num_inference_steps=28,
    height=1152,
    width=2048,
    generator=torch.Generator("cuda").manual_seed(7),
    output="images",
)[0]
image.save("moss_image_int8.png")

License

This model is released under the Apache-2.0 license, the same as the original.