MOSS-Image-IDM-LoRA

A community implementation of the data recipe behind Intrinsic Distribution Matching (IDM), applied to MOSS-Image-SFT.

The IDM paper's ablations show that training on a large dense-text, model-generated dataset — with plain flow-matching and none of the IDM losses — already lifts LongText-Bench scores from 78.5 → 92.7 (CN) at 2 NFE on Z-Image. The DMD + moment-matching losses add the final ~1.5 points. This LoRA replicates the data-driven part of that recipe on MOSS-Image-SFT: it is not the official IDM model and contains no DMD or moment-matching training (IDM weights and training code are not yet released).

v2: 2-step OPD distillation — the paper's actual regime

The paper's headline is few-step text rendering (2 NFE). v2 attacks it directly: on-policy distillation (OPD) to 2 steps with our dense-text dataset, so the few-step generator itself learns text. Teacher: MOSS-Image-SFT @ 50 steps CFG 3. Student: 2 steps CFG 1, LoRA r=64. Checkpoints in opd_8step/ (folder name is a leftover — it holds the 2-step run).

Status (2026-10-30): 249 steps in, the 2-step student has learned composition/color/layout but text is still faint — see grid below (left: base @ 2 steps = smudge; mid: OPD @ 2 steps; right: 50-step reference). For reference, the community 8-step distillation used ~2600 updates; continuation training is in progress. Use opd_8step/step-249.safetensors at num_inference_steps=2, cfg_scale=1.0.

---

v1: SFT LoRA on the 50-step base — what it does — and an honest eval

Trained to improve legible text rendering (menus, posters, signs, labels, sticky notes — English and Chinese) on MOSS-Image-SFT.

v1 result: marginal — read this before using. Side-by-side evals (left: base, right: +LoRA, same seed, 12 held-out prompts):

At 50 steps the base model is already excellent at text (structured-prompt text rendering is MOSS-Image's core training spec), so the delta from this 100-image LoRA is small. At 8 steps (below), both base and LoRA garble text equally — few-step failure is a distillation problem, not something SFT data fixes. The IDM paper's big data-driven jump (78→93 LongText) started from a base that was weak at text; that gap doesn't exist here.

We're publishing it anyway: the checkpoints, data, and evals are all open, and a documented null result on a hyped paper's recipe is useful. See Limitations for the v2 plan.

Training

Base	OpenMOSS-Team/MOSS-Image-SFT (Apache-2.0)
Method	LoRA r=64, all linear layers, plain flow-matching SFT
Data	100 dense-text images self-generated by MOSS-Image-SFT (30 steps, CFG 1.0) — menus/posters/signs/receipts/whiteboards, ~60% EN / 40% ZH: akhaliq/moss-textboost-data
Steps	500 (checkpoints: step-250.safetensors, step-500.safetensors)
LR	1e-4, AdamW, grad accum 4, bf16, 1024×1024
Trainer	DiffSynth-Studio stock MOSS-Image LoRA trainer
Cost	~15 min on 1×A100, trained 2026-10-30

Usage (DiffSynth) — v1 SFT LoRA

from diffsynth.pipelines.moss_image import MOSSImagePipeline, ModelConfig
import torch

pipe = MOSSImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16, device="cuda",
    model_configs=[
        ModelConfig(model_id="OpenMOSS-Team/MOSS-Image-SFT", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="OpenMOSS-Team/MOSS-Image-SFT", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="OpenMOSS-Team/MOSS-Image-SFT", origin_file_pattern="vae/diffusion_pytorch_model.safetensors", computation_dtype=torch.float32),
    ],
    tokenizer_config=ModelConfig(model_id="OpenMOSS-Team/MOSS-Image-SFT", origin_file_pattern="tokenizer/"),
)
pipe.load_lora(pipe.dit, ModelConfig(model_id="akhaliq/MOSS-Image-IDM-LoRA", origin_file_pattern="step-500.safetensors"), alpha=1)

image = pipe('A cafe chalkboard menu with the heading "MORNING COFFEE" and three lines: "Americano $4", "Latte $5", "Mocha $6".',
             height=1024, width=1024, seed=42, num_inference_steps=50, cfg_scale=3.0)

Limitations & v2 plan

• v1 experiment: only 100 training images and 500 steps, and the eval shows no meaningful visible delta over the base at 50 steps.
• No LongText-Bench score; evaluation is visual (see eval/).
• Trained for the 50-step base model; stacking with few-step distilled adapters (8-step DMD, 4-step adapter) does not fix their garbled text — that needs training on the distilled generator, which is the v2 plan: scale the dataset (~1–2K images, the full 249 already in moss-textboost-data plus more) and train the boost on the few-step stack where the text gap actually is.

Citation

@misc{liu2026intrinsicdistributionmatching,
  title={Intrinsic Distribution Matching},
  author={Deyuan Liu and Yansen Han and Kaiwen Zheng and Jun Xie and Jiacheng Liu and Zhenglin Cheng and Peng Sun and Jintao Zhang and Jianguo Li and Tao Lin},
  year={2026}, eprint={2610.39633}, archivePrefix={arXiv}, primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.39633},
}