metadata
library_name: diffusers
pipeline_tag: text-to-image
base_model: ideogram-ai/ideogram-4-nf4-diffusers
tags:
- text-to-image
- image-to-image
- differential-diffusion
- modular-diffusers
- diffusion
- ideogram
license: apache-2.0
Ideogram4 custom modular blocks
Custom Modular Diffusers blocks that
extend Ideogram4 with image-to-image and Differential Diffusion, plus a unified AutoBlocks that folds
text-to-image, img2img, and differential diffusion into a single pipeline, the workflow is chosen automatically
from which inputs you pass.
prompt -> text2image
prompt + image -> image2image (optional strength)
prompt + image + diffdiff_map -> differential diffusion
Loading & running
import torch
from diffusers import ModularPipeline
pipe = ModularPipeline.from_pretrained("OzzyGT/ideogram4-modular", trust_remote_code=True)
pipe.load_components(
names=["text_encoder", "tokenizer", "transformer", "unconditional_transformer", "vae", "scheduler"],
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
image = pipe(prompt="a photo of a red apple on a black background", output="images")[0]
image.save("out.png")
image-to-image
Pass an image (and optional strength) to the same pipe:
from diffusers.utils import load_image
image = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png")
result = pipe(
prompt="a photo of a snowy mountain landscape at sunset, dramatic clouds",
image=image,
strength=0.6,
output="images",
)[0]
result.save("img2img.png")
differential diffusion
Add a grayscale change map (diffdiff_map) alongside the image — darker regions change more, brighter regions stay closer to the reference:
from diffusers.utils import load_image
image = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png")
diffdiff_map = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/gradient_mask.png")
result = pipe(
prompt="a photo of a snowy mountain landscape at sunset, dramatic clouds",
image=image,
diffdiff_map=diffdiff_map,
output="images",
)[0]
result.save("diffdiff.png")