---
base_model: OpenMOSS-Team/MOSS-Image-Base
base_model_relation: adapter
language:
- en
- zh
license: apache-2.0
library_name: peft
pipeline_tag: text-to-image
tags:
- moss-image
- diffusers
- lora
- text-to-image
---
# moss_pht_lora

A photo-oriented LoRA (rank 128) for
[OpenMOSS-Team/MOSS-Image-Base](https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base).
It adapts only the transformer; the text encoder (Qwen3.5-9B), VAE (FLUX.2 small
decoder), tokenizer, scheduler and inference pipeline are the base release's,
unchanged. Load those from their official repositories, or swap in any
compatible ones (for example a quantized text encoder).

| File | Format |
| --- | --- |
| `transformer/` | PEFT LoRA adapter for the Diffusers `MOSSImageTransformer` |
| `loras/moss_pht_lora_bf16.safetensors` | ComfyUI LoRA (kohya layout on the release's module names) |

Both files hold the same weights.

## Diffusers

```bash
pip install "diffusers>=0.38" "transformers>=5.3" "accelerate>=1.10" "safetensors>=0.6" "peft>=0.21" "Pillow>=10"
```

```python
import torch
from diffusers import AutoencoderKLFlux2
from diffusers.utils.dynamic_modules_utils import get_class_from_dynamic_module
from peft import PeftModel
from transformers import AutoModel, AutoTokenizer

BASE = "OpenMOSS-Team/MOSS-Image-Base"


def moss_class(name):
    return get_class_from_dynamic_module(
        BASE, "pipeline_moss_image.py", class_name=name, trust_remote_code=True
    )


transformer = moss_class("MOSSImageTransformer").from_pretrained(
    BASE, subfolder="transformer", torch_dtype=torch.bfloat16
)
transformer = PeftModel.from_pretrained(
    transformer, "data-archetype/moss_pht_lora", subfolder="transformer"
).get_base_model()

pipe = moss_class("MOSSImagePipeline")(
    transformer=transformer,
    scheduler=moss_class("MOSSImageScheduler").from_pretrained(BASE, subfolder="scheduler"),
    text_encoder=AutoModel.from_pretrained(BASE, subfolder="text_encoder", dtype=torch.bfloat16),
    tokenizer=AutoTokenizer.from_pretrained(BASE, subfolder="tokenizer"),
    vae=AutoencoderKLFlux2.from_pretrained(BASE, subfolder="vae", torch_dtype=torch.float32),
)
pipe.to("cuda")  # or pipe.enable_model_cpu_offload() on a smaller GPU

image = pipe(
    "A candid street photograph of a cyclist crossing a rain-soaked intersection at dusk",
    height=1024,
    width=1024,
    num_inference_steps=50,
    guidance_scale=3.0,
    uncond_skip_layers=[22],
    generator=torch.Generator("cpu").manual_seed(42),
).images[0]
```

The adapter is applied unmerged, at strength 1. Sampling settings, prompt
formats (natural language, tags and JSON layouts) and requirements are the base
model's; see its [model card](https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base)
and [user guide](https://github.com/OpenMOSS/MOSS-Image/blob/main/USERGUIDE.md).

## ComfyUI

Use ComfyUI's native MOSS-Image support with the
[Comfy-Org/MOSS-Image](https://huggingface.co/Comfy-Org/MOSS-Image) base model,
text encoder and VAE, and add the LoRA after Load Diffusion Model.

| File | Put in | Loader |
| --- | --- | --- |
| `loras/moss_pht_lora_bf16.safetensors` (this repo) | `models/loras/` | LoraLoaderModelOnly |
| `diffusion_models/moss_image_base_bf16.safetensors`, from Comfy-Org/MOSS-Image | `models/diffusion_models/` | Load Diffusion Model |
| `text_encoders/qwen3.5_9b_bf16.safetensors` (or its int8 / w4a8 variants), from Comfy-Org/MOSS-Image | `models/text_encoders/` | Load CLIP, type `moss_image` |
| `vae/flux2-vae_small.safetensors`, from Comfy-Org/MOSS-Image | `models/vae/` | Load VAE |

## Recommended: a DMD LoRA

Like the base checkpoint, generations with the standard multi-step sampler can be
a bit unstable. Using a DMD LoRA distilled from MOSS-Image-Base is recommended: it
sharpens the output distribution and stabilises generations.

## Training

| Setting | Value |
| --- | --- |
| Base | MOSS-Image-Base, frozen (bf16); text encoder and VAE frozen |
| Adapted modules | `q`, `k`, `v`, `o`, `ffn_up_gate` and `ffn_down` of all 42 transformer blocks; the text embedder is not adapted |
| LoRA | Rank 128, alpha 128 (scale 1), A Kaiming-uniform and B zero at initialization, no dropout; `ffn_up_gate` carries one rank-128 pair for each of its up and gate halves |
| Steps | 5,500 at 32 images per step |
| Resolution | About 1 megapixel, aspect-ratio buckets |
| Text | Captions 80% natural language, 20% JSON layouts (MOSS-Image's layout prompt format); prompts up to 2,048 tokens; 10% of captions dropped to the empty prompt |
| Objective | The base's flow-matching objective; noise levels from a truncated logit-normal (μ 1.09, σ 0.95, truncated at 0.95), plus 1% uniform and 1% pure noise |
| Optimizer | AdamW on the LoRA factors (fp32), β = (0.9, 0.9975), ε 1e-8 |
| Learning rate | 1e-4 |
| Schedule | 150-step linear warmup, constant to step 4,500, then a 1,000-step 1 − √ cooldown to 0 |
| Weight decay | 0.01 (decoupled) on all LoRA factors |
| Gradient clipping | Global norm 1.0 |
| Released weights | EMA of the LoRA factors (decay 0.999, fp32, updated every step) at step 5,500, stored in bfloat16 |

## License

The weights are released under the base model's license, Apache 2.0
([LICENSE](LICENSE), copied from OpenMOSS-Team/MOSS-Image-Base).

We thank the OpenMOSS team for MOSS-Image.