README.md

metadata

base_model: OpenMOSS-Team/MOSS-Image-Base
base_model_relation: adapter
language:
  - en
  - zh
license: apache-2.0
library_name: peft
pipeline_tag: text-to-image
tags:
  - moss-image
  - diffusers
  - lora
  - text-to-image

moss_pht_lora

A photo-oriented LoRA (rank 128) for OpenMOSS-Team/MOSS-Image-Base. It adapts only the transformer; the text encoder (Qwen3.5-9B), VAE (FLUX.2 small decoder), tokenizer, scheduler and inference pipeline are the base release's, unchanged. Load those from their official repositories, or swap in any compatible ones (for example a quantized text encoder).

File	Format
transformer/	PEFT LoRA adapter for the Diffusers MOSSImageTransformer
loras/moss_pht_lora_bf16.safetensors	ComfyUI LoRA (kohya layout on the release's module names)

Both files hold the same weights.

Diffusers

pip install "diffusers>=0.38" "transformers>=5.3" "accelerate>=1.10" "safetensors>=0.6" "peft>=0.21" "Pillow>=10"

import torch
from diffusers import AutoencoderKLFlux2
from diffusers.utils.dynamic_modules_utils import get_class_from_dynamic_module
from peft import PeftModel
from transformers import AutoModel, AutoTokenizer

BASE = "OpenMOSS-Team/MOSS-Image-Base"


def moss_class(name):
    return get_class_from_dynamic_module(
        BASE, "pipeline_moss_image.py", class_name=name, trust_remote_code=True
    )


transformer = moss_class("MOSSImageTransformer").from_pretrained(
    BASE, subfolder="transformer", torch_dtype=torch.bfloat16
)
transformer = PeftModel.from_pretrained(
    transformer, "data-archetype/moss_pht_lora", subfolder="transformer"
).get_base_model()

pipe = moss_class("MOSSImagePipeline")(
    transformer=transformer,
    scheduler=moss_class("MOSSImageScheduler").from_pretrained(BASE, subfolder="scheduler"),
    text_encoder=AutoModel.from_pretrained(BASE, subfolder="text_encoder", dtype=torch.bfloat16),
    tokenizer=AutoTokenizer.from_pretrained(BASE, subfolder="tokenizer"),
    vae=AutoencoderKLFlux2.from_pretrained(BASE, subfolder="vae", torch_dtype=torch.float32),
)
pipe.to("cuda")  # or pipe.enable_model_cpu_offload() on a smaller GPU

image = pipe(
    "A candid street photograph of a cyclist crossing a rain-soaked intersection at dusk",
    height=1024,
    width=1024,
    num_inference_steps=50,
    guidance_scale=3.0,
    uncond_skip_layers=[22],
    generator=torch.Generator("cpu").manual_seed(42),
).images[0]

The adapter is applied unmerged, at strength 1. Sampling settings, prompt formats (natural language, tags and JSON layouts) and requirements are the base model's; see its model card and user guide.

ComfyUI

Use ComfyUI's native MOSS-Image support with the Comfy-Org/MOSS-Image base model, text encoder and VAE, and add the LoRA after Load Diffusion Model.

File	Put in	Loader
loras/moss_pht_lora_bf16.safetensors (this repo)	models/loras/	LoraLoaderModelOnly
diffusion_models/moss_image_base_bf16.safetensors, from Comfy-Org/MOSS-Image	models/diffusion_models/	Load Diffusion Model
text_encoders/qwen3.5_9b_bf16.safetensors (or its int8 / w4a8 variants), from Comfy-Org/MOSS-Image	models/text_encoders/	Load CLIP, type moss_image
vae/flux2-vae_small.safetensors, from Comfy-Org/MOSS-Image	models/vae/	Load VAE

Recommended: a DMD LoRA

Like the base checkpoint, generations with the standard multi-step sampler can be a bit unstable. Using a DMD LoRA distilled from MOSS-Image-Base is recommended: it sharpens the output distribution and stabilises generations.

Training

Setting	Value
Base	MOSS-Image-Base, frozen (bf16); text encoder and VAE frozen
Adapted modules	q, k, v, o, ffn_up_gate and ffn_down of all 42 transformer blocks; the text embedder is not adapted
LoRA	Rank 128, alpha 128 (scale 1), A Kaiming-uniform and B zero at initialization, no dropout; ffn_up_gate carries one rank-128 pair for each of its up and gate halves
Steps	5,500 at 32 images per step
Resolution	About 1 megapixel, aspect-ratio buckets
Text	Captions 80% natural language, 20% JSON layouts (MOSS-Image's layout prompt format); prompts up to 2,048 tokens; 10% of captions dropped to the empty prompt
Objective	The base's flow-matching objective; noise levels from a truncated logit-normal (μ 1.09, σ 0.95, truncated at 0.95), plus 1% uniform and 1% pure noise
Optimizer	AdamW on the LoRA factors (fp32), β = (0.9, 0.9975), ε 1e-8
Learning rate	1e-4
Schedule	150-step linear warmup, constant to step 4,500, then a 1,000-step 1 − √ cooldown to 0
Weight decay	0.01 (decoupled) on all LoRA factors
Gradient clipping	Global norm 1.0
Released weights	EMA of the LoRA factors (decay 0.999, fp32, updated every step) at step 5,500, stored in bfloat16

License

The weights are released under the base model's license, Apache 2.0 (LICENSE, copied from OpenMOSS-Team/MOSS-Image-Base).

We thank the OpenMOSS team for MOSS-Image.