austin-k-wang/DiffusionLAIR-SDXL

Model details

• Base model: stabilityai/stable-diffusion-xl-base-1.0
• Base-model revision: 462165984030d82259a11f4367a4eed129e94a7b
• Architecture: Stable Diffusion XL
• Resolution: 1024 × 1024
• Fine-tuned component: UNet
• Weight format: sharded Safetensors
• Weight precision: float32
• Training objective: Diffusion LAIR
• Reward/preference model: PickScore
• Training dataset: Pick-a-Pic v2

License

This model is distributed under the CreativeML Open RAIL++-M license. Users must comply with the use-based restrictions specified in the license.

Usage

import torch
from diffusers import (
    AutoencoderKL,
    StableDiffusionXLPipeline,
    UNet2DConditionModel,
)

model_id = "austin-k-wang/DiffusionLAIR-SDXL"
base_model_id = "stabilityai/stable-diffusion-xl-base-1.0"

unet = UNet2DConditionModel.from_pretrained(
    model_id,
    subfolder="unet",
    torch_dtype=torch.float16,
    use_safetensors=True,
)

vae = AutoencoderKL.from_pretrained(
    "madebyollin/sdxl-vae-fp16-fix",
    torch_dtype=torch.float16,
)

pipe = StableDiffusionXLPipeline.from_pretrained(
    base_model_id,
    unet=unet,
    vae=vae,
    variant="fp16",
    torch_dtype=torch.float16,
    use_safetensors=True,
).to("cuda")

generator = torch.Generator(device="cuda").manual_seed(42)

image = pipe(
    "a cinematic photograph of a futuristic city at sunset",
    generator=generator,
    guidance_scale=5.0,
    num_inference_steps=30,
).images[0]

image.save("output.png")

For lower-VRAM inference, replace .to("cuda") with:

pipe.enable_model_cpu_offload()