HelloSun/animagine-xl-4.0_Lightning_ov

Animagine XL 4.0 SDXL-Lightning OpenVINO (CPU Optimized)

This repository provides an optimized version of Animagine XL 4.0 merged with SDXL-Lightning weights, specifically converted to OpenVINO IR (INT8) format for highly efficient inference on Intel CPUs.

By utilizing SDXL-Lightning's 4-step distillation and OpenVINO's INT8 weight compression, this model achieves fast inference speeds on standard Intel CPUs without requiring a dedicated GPU, while maintaining high-quality anime-style image generation.

📦 Installation

To use this model, you need to install the Hugging Face diffusers, transformers, and Intel's optimum-intel libraries.

pip install "optimum-intel[openvino,diffusers]>=1.27.0" \
            "openvino>=2025.2.0" \
            "diffusers>=0.31.0" \
            "transformers>=4.45.0" \
            "accelerate" \
            "nncf"

🔄 Conversion Macro (轉換巨集) If you want to convert the PyTorch model to OpenVINO IR format yourself, you can use the following script. This script downloads the base model, merges it with SDXL-Lightning weights, and exports it to OpenVINO INT8 format.

import os
import gc
import shutil
import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel, EulerDiscreteScheduler
from optimum.intel import OVStableDiffusionXLPipeline

base = "cagliostrolab/animagine-xl-4.0"
repo = "ByteDance/SDXL-Lightning"
ckpt = "sdxl_lightning_4step_unet.safetensors"
ov_output_dir = "./animagine_sdxl_lightning_ov"

# 1. Load Lightning UNet
unet = UNet2DConditionModel.from_config(base, subfolder="unet")
unet.load_state_dict(load_file(hf_hub_download(repo, ckpt), device="cpu"), strict=True)

# 2. Assemble PyTorch Pipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
    base, unet=unet, torch_dtype=torch.float32, use_safetensors=True
)
pipe.scheduler = EulerDiscreteScheduler.from_config(
    pipe.scheduler.config, timestep_spacing="trailing",
    beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear"
)

# Save temporary PyTorch model
tmp_pt_dir = "./tmp_pt"
pipe.save_pretrained(tmp_pt_dir, safe_serialization=True)
del unet, pipe; gc.collect()

# 3. Convert to OpenVINO IR (INT8 for CPU RAM optimization)
ov_pipe = OVStableDiffusionXLPipeline.from_pretrained(
    tmp_pt_dir,
    export=True,
    compile=False,
    weight_format="int8", # Recommended for Intel CPU to save RAM and speed up inference
    ov_config={
        "PERFORMANCE_HINT": "LATENCY",
        "NUM_STREAMS": "1",
        "INFERENCE_NUM_THREADS": "4",
        "DYNAMIC_QUANTIZATION_GROUP_SIZE": "32"
    }
)

# 4. Save OpenVINO model
ov_pipe.save_pretrained(ov_output_dir)
shutil.rmtree(tmp_pt_dir)
print("Export completed successfully!")

🎨 Inference Macro (推論巨集) Use the following script to load the converted OpenVINO model and generate images on an Intel CPU.

import os
import torch
from optimum.intel import OVStableDiffusionXLPipeline
from diffusers import EulerDiscreteScheduler

# Optimize OpenMP for Intel CPU
os.environ["OMP_WAIT_POLICY"] = "PASSIVE"
os.environ["KMP_BLOCKTIME"] = "0"

model_dir = "./animagine_sdxl_lightning_ov"

# 1. Load OpenVINO model
ov_pipe = OVStableDiffusionXLPipeline.from_pretrained(
    model_dir, 
    device="CPU",
    ov_config={
        "PERFORMANCE_HINT": "LATENCY",
        "NUM_STREAMS": "1",
        "INFERENCE_NUM_THREADS": "4",
        "DYNAMIC_QUANTIZATION_GROUP_SIZE": "32"
    }
)

# 2. Set Scheduler (Required for SDXL-Lightning)
ov_pipe.scheduler = EulerDiscreteScheduler.from_config(
    ov_pipe.scheduler.config,
    timestep_spacing="trailing",
    beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear"
)

# 3. Run inference
prompt = "1girl, smiling, looking at viewer, anime style, high quality, masterpiece"
negative_prompt = "lowres, bad anatomy, bad hands, text, error, worst quality, low quality"

image = ov_pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=4, # Lightning model requires 4 steps
    guidance_scale=0.0,    # Lightning model recommends guidance_scale=0
    width=1024,
    height=1024,
    generator=torch.Generator("cpu").manual_seed(42)
).images[0]

image.save("output_openvino.png")
print("Image saved to output_openvino.png")