import torch
from diffusers import AutoPipelineForInpainting
from diffusers.utils import load_image
# switch to "mps" for apple devices
pipe = AutoPipelineForInpainting.from_pretrained("visualseed/RORem-merged", dtype=torch.float16, device_map="cuda")
img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"
image = load_image(img_url).resize((1024, 1024))
mask_image = load_image(mask_url).resize((1024, 1024))
prompt = "a tiger sitting on a park bench"
generator = torch.Generator(device="cuda").manual_seed(0)
image = pipe(
prompt=prompt,
image=image,
mask_image=mask_image,
guidance_scale=8.0,
num_inference_steps=20, # steps between 15 and 30 work well for us
strength=0.99, # make sure to use `strength` below 1.0
generator=generator,
).images[0]RORem (merged)
SDXL - RORem (CVPR 2025) repackaged as a plain diffusers
StableDiffusionXLInpaintPipeline: the LetsThink/RORem
weights with their LCM-distillation LoRA already merged into the UNet, so
conversion pipelines install straight from this repository with no adapter
step. Nothing was retrained — the merge is a plain B@A bake at scale 1
(788 adapters), numerically equivalent to diffusers' own fuse_lora.
Inference contract
Verified by instrumenting the reference inference_RORem_4S.py call
(diffusers 0.41):
pipe(prompt="", image=image, mask_image=mask, guidance_scale=1.0,
num_inference_steps=4, strength=0.99) # LCMScheduler
The scheduler places [999, 759, 499, 259] and the strength slice drops the head — three denoiser passes at [759, 499, 259], with the photo noised to 759. Guidance is off and the prompt is empty: RORem is prompt-free.
Provenance
Derived from LetsThink/RORem (Apache-2.0), the LCM LoRA from the RORem-LCM release, on the CreativeML Open RAIL++-M SDXL inpainting base. Project page: github.com/leeruibin/RORem.
- Downloads last month
- 13
Model tree for visualseed/RORem-merged
Base model
LetsThink/RORem