README.md

---
library_name: diffusers
pipeline_tag: text-to-image
base_model: ideogram-ai/ideogram-4-nf4-diffusers
tags:
  - text-to-image
  - image-to-image
  - differential-diffusion
  - modular-diffusers
  - diffusion
  - ideogram
license: apache-2.0
---

# Ideogram4 custom modular blocks

Custom [Modular Diffusers](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) blocks that
extend Ideogram4 with **image-to-image** and **Differential Diffusion**, plus a **unified `AutoBlocks`** that folds
text-to-image, img2img, and differential diffusion into a single pipeline, the workflow is chosen automatically
from which inputs you pass.

```
prompt                          -> text2image
prompt + image                  -> image2image        (optional strength)
prompt + image + diffdiff_map   -> differential diffusion
```

## Loading & running


```python
import torch
from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("OzzyGT/ideogram4-modular", trust_remote_code=True)
pipe.load_components(
    names=["text_encoder", "tokenizer", "transformer", "unconditional_transformer", "vae", "scheduler"],
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda")

image = pipe(prompt="a photo of a red apple on a black background", output="images")[0]
image.save("out.png")
```

### image-to-image

Pass an `image` (and optional `strength`) to the same pipe:

```python
from diffusers.utils import load_image

image = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png")

result = pipe(
    prompt="a photo of a snowy mountain landscape at sunset, dramatic clouds",
    image=image,
    strength=0.6,
    output="images",
)[0]
result.save("img2img.png")
```

### differential diffusion

Add a grayscale change map (`diffdiff_map`) alongside the `image` — darker regions change more, brighter regions stay closer to the reference:

```python
from diffusers.utils import load_image

image = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png")
diffdiff_map = load_image("https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/gradient_mask.png")

result = pipe(
    prompt="a photo of a snowy mountain landscape at sunset, dramatic clouds",
    image=image,
    diffdiff_map=diffdiff_map,
    output="images",
)[0]
result.save("diffdiff.png")
```