creative-graphic-design
/

collage-diffusion

@@ -1,7 +1,493 @@
-# Collage Diffusion source assets
-Apart from this model card and repository metadata, the only content currently published here is the original source material linked by the authors of [Collage Diffusion](https://github.com/VSAnimator/collage-diffusion), archived under [`source-assets/`](https://huggingface.co/creative-graphic-design/collage-diffusion/tree/main/source-assets) with a provenance and checksum inventory.
-Publishing the pipeline configuration is a later step and requires separate approval. No pipeline configuration or model weights are published here.
-The files under `source-assets/` have no clearly stated per-file license. The Apache-2.0 metadata on the separate `creative-graphic-design/documentation-images` dataset does not apply to these files. This repository therefore does not declare license metadata for these assets.

+---
+language:
+  - en
+library_name: "diffusers"
+pipeline_tag: "image-to-image"
+tags:
+  - "collage-diffusion"
+  - "image-generation"
+  - "wacv-2024"
+datasets: []
+base_model: "sd2-community/stable-diffusion-2-1-base"
+---
+<!-- --8<-- [start:card] -->
+# Model Card for Collage Diffusion
+[![arXiv](https://img.shields.io/static/v1?label=arXiv&message=2303.00262&color=b31b1b&style=flat-square&logo=arxiv&logoColor=white)](https://arxiv.org/abs/2303.00262)
+![venue](https://img.shields.io/static/v1?label=venue&message=WACV+2024&color=purple&style=flat-square)
+![license](https://img.shields.io/static/v1?label=license&message=Apache-2.0&color=green&style=flat-square&logo=apache&logoColor=white)
+![framework](https://img.shields.io/static/v1?label=framework&message=diffusers&color=red&style=flat-square&logo=huggingface&logoColor=white)
+![base](https://img.shields.io/static/v1?label=base&message=diffusers&color=blue&style=flat-square&logo=huggingface&logoColor=white)
+![task](https://img.shields.io/static/v1?label=task&message=image-to-image&color=purple&style=flat-square)
+![vendor-parity](https://img.shields.io/static/v1?label=vendor-parity&message=tolerance-verified&color=success&style=flat-square)
+![hub](https://img.shields.io/static/v1?label=hub&message=published-code-only&color=brightgreen&style=flat-square&logo=huggingface&logoColor=white)
+This package exposes Collage Diffusion as a [`🧨diffusers`](https://huggingface.co/docs/diffusers/index) pipeline that harmonizes text-described RGBA layers into a single image.
+## Model Details
+### Model Description
+Collage Diffusion combines a scene prompt with ordered RGBA image layers. Visible alpha masks guide cross-attention to each layer's text, while per-layer noise levels control how much of each source layer is preserved during diffusion. The pipeline can load caller-provided textual-inversion embeddings. It uses the upstream Stable Diffusion 2.1 base checkpoint and does not train, store, or publish model-specific weights.
+- **Developed by:** Vishnu Sarukkai, Linden Li, Arden Ma, Christopher Ré, and Kayvon Fatahalian.
+- **Shared by:** creative-graphic-design.
+- **Model type:** image generation; task: image-to-image; conditioning: layers, text.
+- **Language(s) (NLP):** English scene and layer prompts.
+- **License:** Apache-2.0 for this package; the base checkpoint and input assets retain their own terms.
+- **Finetuned from:** No Collage Diffusion checkpoint; inference loads `sd2-community/stable-diffusion-2-1-base` at the pinned revision below under the CreativeML Open RAIL++-M terms declared by its model card.
+### Model Sources
+- **Repository:** [Collage Diffusion source repository](https://github.com/VSAnimator/collage-diffusion); it has no declared repository license. Its notebook is used as a behavioral reference; its code is not included here. Notebook-linked image layers and textual-inversion embeddings are archived, with source links and integrity records, in the [model repository's source-assets directory](https://huggingface.co/creative-graphic-design/collage-diffusion/tree/main/source-assets).
+- **Paper:** [Collage Diffusion](https://arxiv.org/abs/2303.00262).
+- **Related implementation:** [Collage Diffusion UI](https://github.com/linden-li/collage-diffusion-ui) was inspected as a secondary reference. Its repository also has no declared license; its code is not included here.
+- **Released weights:** The package does not contain its own weights. It loads the [Stable Diffusion 2.1 base mirror](https://huggingface.co/sd2-community/stable-diffusion-2-1-base) at a pinned revision.
+### Base Checkpoint Provenance and Integrity
+The authors' notebook names `stabilityai/stable-diffusion-2-1-base`, but that Hub repository now returns HTTP 404. The comparison and package examples use `sd2-community/stable-diffusion-2-1-base` at revision `4e63672c03103b6c636b8fb4119ba982469b2955`. The model card declares the CreativeML Open RAIL++-M license; users must review and follow those terms.
+The main weight files were checked across three available copies. Their SHA-256 values match byte-for-byte:
+| Copy                                         | Revision                                   | UNet SHA-256                                                       | VAE SHA-256                                                        | Text encoder SHA-256                                               |
+| -------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------ | ------------------------------------------------------------------ | ------------------------------------------------------------------ |
+| `sd2-community/stable-diffusion-2-1-base`    | `4e63672c03103b6c636b8fb4119ba982469b2955` | `6dfae3e5f7d459b50f4b0850ead945972c75bb0e1897628933e169eb43974214` | `a1d993488569e928462932c8c38a0760b874d166399b14414135bd9c42df5815` | `cce6febb0b6d876ee5eb24af35e27e764eb4f9b1d0b7c026c8c3333d4cfc916c` |
+| `Manojb/stable-diffusion-2-1-base`           | `0094d483a120f3f33dafbd187ea4aa60d10de75c` | same                                                               | same                                                               | same                                                               |
+| `LanguageMachines/stable-diffusion-2-1-base` | `2579f99c90b5a3019fc115ee9566d5ec4f17fccf` | same                                                               | same                                                               | same                                                               |
+SHA-256 values for every file used from the selected pinned snapshot:
+| File                                         | SHA-256                                                            |
+| -------------------------------------------- | ------------------------------------------------------------------ |
+| `model_index.json`                           | `518553e098fdaa4153d9a92139e344e44f53bb329c22f383293e3099c4f1bad2` |
+| `feature_extractor/preprocessor_config.json` | `2a1da83b5e1032aaeef397552ddb408dca0d8cd1dc58f61bf6abf38d6f33a0a2` |
+| `scheduler/scheduler_config.json`            | `11ac5627d7df0fa344b875c4b5722b1767a8a2aa1684c2cf8b4d614300127234` |
+| `text_encoder/config.json`                   | `7e33bfc475f6cf76e69b28b101993ca834d1f22ff4d35caefde32d63d4f42fba` |
+| `text_encoder/model.safetensors`             | `cce6febb0b6d876ee5eb24af35e27e764eb4f9b1d0b7c026c8c3333d4cfc916c` |
+| `tokenizer/merges.txt`                       | `9fd691f7c8039210e0fced15865466c65820d09b63988b0174bfe25de299051a` |
+| `tokenizer/special_tokens_map.json`          | `f118ab3a983206e4f32583448de6bd6aae4ee21869135cef1f5848a753cdaab6` |
+| `tokenizer/tokenizer_config.json`            | `d562b18a0c6d32a168fce3d7a342fce64579ebacffb98fb71b0affc494fc32d8` |
+| `tokenizer/vocab.json`                       | `e089ad92ba36837a0d31433e555c8f45fe601ab5c221d4f607ded32d9f7a4349` |
+| `unet/config.json`                           | `ce0c6d379e3b1d3e1f79338de70c80a1e36f17a6372439a8249b6fb0dfa1b608` |
+| `unet/diffusion_pytorch_model.safetensors`   | `6dfae3e5f7d459b50f4b0850ead945972c75bb0e1897628933e169eb43974214` |
+| `vae/config.json`                            | `424117cb534ce03497c41305ed868980123917b2b6abba4bbaa615e968772903` |
+| `vae/diffusion_pytorch_model.safetensors`    | `a1d993488569e928462932c8c38a0760b874d166399b14414135bd9c42df5815` |
+Both sides of the notebook comparison use this same pinned snapshot and the file hashes listed above. The selected model copy and its weights are not redistributed by this package.
+## Supported Checkpoints
+| Artifact                  | Hub ID                                      | Status                                    |
+| ------------------------- | ------------------------------------------- | ----------------------------------------- |
+| `diffusers` pipeline code | `creative-graphic-design/collage-diffusion` | planned, code only; no learned checkpoint |
+When publishing this pipeline to the Hub, its `model_index.json` and `scheduler/scheduler_config.json` must specify `EulerAncestralDiscreteScheduler`.
+## Uses
+### Direct Use
+Use the pipeline to harmonize a layered collage while guiding object placement through masks and controlling source preservation through per-layer noise levels.
+```python
+from PIL import Image
+from diffusers import EulerAncestralDiscreteScheduler
+from collage_diffusion import CollageDiffusionPipeline, CollageLayer
+base = "sd2-community/stable-diffusion-2-1-base"
+revision = "4e63672c03103b6c636b8fb4119ba982469b2955"
+# Cross-attention guidance uses the scheduler's sigma values.
+scheduler = EulerAncestralDiscreteScheduler.from_pretrained(
+    base, subfolder="scheduler", revision=revision
+)
+pipe = CollageDiffusionPipeline.from_pretrained(
+    base, revision=revision, scheduler=scheduler
+)
+layers = [
+    CollageLayer(
+        image=Image.open("toy.png").convert("RGBA"),
+        prompt="a small wooden toy",
+        position=(64, 96),
+        noise_level=0.35,
+        positive_attention_weight=0.5,
+        negative_attention_weight=0.5,
+    ),
+]
+result = pipe(
+    prompt="a small wooden toy on a studio table",
+    layers=layers,
+    negative_prompt="collage",
+    seed=0,
+    num_inference_steps=20,
+)
+result.images[0].save("harmonized.png")
+print("Saved harmonized.png")
+```
+```text
+Saved harmonized.png
+```
+Textual-inversion embeddings are caller-supplied with a token-to-file mapping, and the matching token is set on its `CollageLayer`. ControlNet and textual-inversion training are outside this package's current scope.
+### Examples
+#### Shared setup
+These examples follow the [Collage Diffusion notebook](https://github.com/VSAnimator/collage-diffusion/blob/main/CollageDiffusionDemo.ipynb) and its [Google Drive image layers](https://drive.google.com/drive/folders/1EfoLfWbn5fl7c092uk96zHtJOVHRlIqI?usp=sharing) and [textual-inversion assets](https://drive.google.com/drive/folders/13Gg8y-0FxdPcuhrPOUw6Ze9wa0Ruu-cI?usp=sharing). Because the Drive files may disappear, copies and their source URLs, retrieval dates, sizes, and SHA-256 values are recorded in the [model repository's source-assets directory](https://huggingface.co/creative-graphic-design/collage-diffusion/tree/main/source-assets). The original author, rights holder, and per-file license are unknown; the documentation-images dataset's Apache-2.0 license does not apply to those files.
+Each image places the input collage on the left and the generated result on the right. All scenes use `sd2-community/stable-diffusion-2-1-base` at revision `4e63672c03103b6c636b8fb4119ba982469b2955`, `EulerAncestralDiscreteScheduler`, seed `0`, 512 × 512 resolution, guidance scale `7.5`, and 50 requested inference steps. The active step count is 40 at strength `0.8` and 37 at `0.75`. The base checkpoint uses the [CreativeML Open RAIL++-M License](https://huggingface.co/sd2-community/stable-diffusion-2-1-base). The code uses every CPU thread available to its runtime; the generated examples used **2 threads**.
+#### Toy Box
+Prompt: `a teddy bear, a wood train, and an american football, in front of a tan background`. Negative prompt: `collage`. Strength: `0.8`; `noise_blur=20`; 50 requested (40 active) steps.
+![Toy Box input collage on the left and Collage Diffusion output on the right.](https://huggingface.co/datasets/creative-graphic-design/documentation-images/resolve/main/collage-diffusion/toy-box-diffusers-comparison-99b4a70.png)
+#### Bento Box
+Prompt: `a bento box with rice, edamame, ginger, and sushi`. Negative prompt: `collage`. Strength: `0.8`; `noise_blur=20`; 50 requested (40 active) steps.
+![Bento Box input collage on the left and Collage Diffusion output on the right.](https://huggingface.co/datasets/creative-graphic-design/documentation-images/resolve/main/collage-diffusion/bento-box-diffusers-comparison-99b4a70.png)
+#### Cake Table
+Prompt: `a wood table with two white chairs behind, two pink cakes on top, maroon bookshelves behind, and winter window`. Negative prompt: `collage, warped, cupcake`. Strength: `0.75`; `noise_blur=10`; 50 requested (37 active) steps.
+![Cake Table input collage on the left and Collage Diffusion output on the right.](https://huggingface.co/datasets/creative-graphic-design/documentation-images/resolve/main/collage-diffusion/cake-table-diffusers-comparison-99b4a70.png)
+The following script builds each scene from its archived image layers and textual-inversion embeddings, then saves the side-by-side examples above. It generates all three by default; set `COLLAGE_SCENES` to a comma-separated subset of scene names to run only selected examples.
+```python
+import gc
+import os
+import time
+import torch
+from diffusers import EulerAncestralDiscreteScheduler
+from diffusers.utils import load_image, make_image_grid
+from huggingface_hub import hf_hub_download
+from collage_diffusion import CollageDiffusionPipeline, CollageLayer, compose_collage_layers
+torch.set_num_threads(os.cpu_count() or 1)
+print(f"CPU threads: {torch.get_num_threads()}")
+base = "sd2-community/stable-diffusion-2-1-base"
+base_revision = "4e63672c03103b6c636b8fb4119ba982469b2955"
+asset_repo_id = "creative-graphic-design/collage-diffusion"
+asset_revision = "3d958ad645d9d4641126c8d8a3e74116f4a99421"
+asset_root = "source-assets"
+asset_url = "https://huggingface.co/creative-graphic-design/collage-diffusion/resolve/3d958ad645d9d4641126c8d8a3e74116f4a99421/source-assets"
+scenes = [
+    {
+        "name": "Toy Box",
+        "folder": "toyBox2",
+        "output": "toy-box-diffusers-comparison-99b4a70.png",
+        "prompt": "a teddy bear, a wood train, and an american football, in front of a tan background",
+        "negative_prompt": "collage",
+        "strength": 0.8,
+        "noise_blur": 20,
+        "layers": [
+            ("_0004_background.png", "background", 0.8, 0.0, 0.5, None, None),
+            ("_0003_football.png", "american football", 0.2, 0.5, 0.5, None, None),
+            ("_0002_teddy_bear.png", "teddy bear", 0.8, 0.2, 0.5, "<fda>", "asset_library/toyBox2-teddy_bear/lora_weight_e999_s1000.ti.pt"),
+            ("_0000_wood_train.png", "wood train", 0.5, 0.2, 0.5, "<xtc>", "asset_library/toyBox2-wood_train/lora_weight_e999_s1000.ti.pt"),
+        ],
+    },
+    {
+        "name": "Bento Box",
+        "folder": "bentoBox",
+        "output": "bento-box-diffusers-comparison-99b4a70.png",
+        "prompt": "a bento box with rice, edamame, ginger, and sushi",
+        "negative_prompt": "collage",
+        "strength": 0.8,
+        "noise_blur": 20,
+        "layers": [
+            ("_0004_bento_box.png", "bento box", 0.5, 0.0, 0.5, "<krk>", "asset_library/bentoBox-bento_box/lora_weight_e999_s1000.ti.pt"),
+            ("_0003_rice.png", "rice", 0.5, 0.4, 0.5, "<lol>", "asset_library/bentoBox-rice/lora_weight_e499_s500.ti.pt"),
+            ("_0002_edamame.png", "edamame", 0.6, 0.0, 0.5, "<fda>", "asset_library/bentoBox-edamame/lora_weight_e999_s1000.ti.pt"),
+            ("_0001_ginger.png", "ginger", 0.8, 0.6, 0.5, "<xtc>", "asset_library/bentoBox-ginger/lora_weight_e999_s1000.ti.pt"),
+            ("_0000_sushi.png", "sushi", 0.8, 0.2, 0.5, "<kyt>", "asset_library/bentoBox-sushi/lora_weight_e999_s1000.ti.pt"),
+        ],
+    },
+    {
+        "name": "Cake Table",
+        "folder": "cakeTable2",
+        "output": "cake-table-diffusers-comparison-99b4a70.png",
+        "prompt": "a wood table with two white chairs behind, two pink cakes on top, maroon bookshelves behind, and winter window",
+        "negative_prompt": "collage, warped, cupcake",
+        "strength": 0.75,
+        "noise_blur": 10,
+        "layers": [
+            ("_0006_winter_window.png", "winter window", 0.8, 0.45, 1.0, "<krk>", "asset_library/cakeTable2-winter_window/lora_weight_e499_s500.ti.pt"),
+            ("_0005_maroon_bookshelves.png", "maroon bookshelves", 0.8, 0.35, 1.0, "<lol>", "asset_library/cakeTable2-maroon_bookshelves/lora_weight_e999_s1000.ti.pt"),
+            ("_0004_white_chairs.png", "white chairs", 0.8, 0.5, 1.0, "<fda>", "asset_library/cakeTable2-white_chairs/lora_weight_e999_s1000.ti.pt"),
+            ("_0003_wood_table.png", "wood table", 0.6, 0.1, 1.0, "<xtc>", "asset_library/cakeTable2-wood_table/lora_weight_e999_s1000.ti.pt"),
+            ("_0002_decorated_cakes.png", "pink cakes", 0.6, 0.3, 2.0, "<kyt>", "asset_library/cakeTable2-decorated_cakes/lora_weight_e99_s100.ti.pt"),
+        ],
+    },
+]
+selected_names = os.environ.get("COLLAGE_SCENES")
+if selected_names:
+    selected_names = {name.strip() for name in selected_names.split(",")}
+    scenes = [scene for scene in scenes if scene["name"] in selected_names]
+    if not scenes:
+        raise ValueError("COLLAGE_SCENES must name at least one configured scene")
+for scene in scenes:
+    scene_started = time.perf_counter()
+    layers = []
+    textual_inversion_embeddings = {}
+    for filename, prompt, noise, positive, negative, token, embedding in scene["layers"]:
+        layers.append(
+            CollageLayer(
+                image=load_image(
+                    f"{asset_url}/image_layers/{scene['folder']}/layers/{filename}",
+                    convert_method=lambda image: image.convert("RGBA"),
+                ),
+                prompt=prompt,
+                noise_level=noise,
+                positive_attention_weight=positive,
+                negative_attention_weight=negative,
+                textual_inversion_token=token,
+            )
+        )
+        if token is not None and embedding is not None:
+            textual_inversion_embeddings[token] = hf_hub_download(
+                repo_id=asset_repo_id,
+                repo_type="model",
+                revision=asset_revision,
+                filename=f"{asset_root}/{embedding}",
+            )
+    input_collage, _ = compose_collage_layers(layers, width=512, height=512)
+    scheduler = EulerAncestralDiscreteScheduler.from_pretrained(
+        base, subfolder="scheduler", revision=base_revision
+    )
+    pipe = CollageDiffusionPipeline.from_pretrained(
+        base, revision=base_revision, scheduler=scheduler
+    )
+    result = pipe(
+        prompt=scene["prompt"],
+        layers=layers,
+        textual_inversion_embeddings=textual_inversion_embeddings,
+        negative_prompt=scene["negative_prompt"],
+        strength=scene["strength"],
+        num_inference_steps=50,
+        guidance_scale=7.5,
+        noise_blur=scene["noise_blur"],
+        seed=0,
+    ).images[0]
+    make_image_grid([input_collage, result], rows=1, cols=2).save(scene["output"])
+    elapsed_minutes = (time.perf_counter() - scene_started) / 60
+    print(f"Saved {scene['output']} in {elapsed_minutes:.1f} minutes")
+    del pipe, scheduler, result, input_collage, layers, textual_inversion_embeddings
+    gc.collect()
+```
+### Downstream Use
+Generated images may be used for design exploration after human review of object placement, appearance, and the licenses attached to all input layers and model weights.
+### Out-of-Scope Use
+This pipeline does not guarantee semantic accuracy, rights clearance for caller-provided images or embeddings, or suitability for safety-critical applications.
+## Bias, Risks, and Limitations
+Results inherit the capabilities and biases of Stable Diffusion 2.1 and the caller's prompts, layers, masks, and textual-inversion embeddings. The original authors and rights holders of the notebook-linked example images and embeddings, and the license for each file, are unknown. The [source-assets inventory](https://huggingface.co/creative-graphic-design/collage-diffusion/blob/main/source-assets/README.md) records the retrieval URLs and integrity data. The paper's ControlNet extension and textual-inversion training are not implemented here.
+### Recommendations
+Review each output and confirm rights to every input layer, embedding, and generated asset before publication or commercial use.
+## How to Get Started with the Model
+Install the pipeline package and `laygen` from commit-pinned GitHub source archives. The repository README uses the `main` archive URL so the command stays current; the published Hub card pins both packages to the commit that published its pipeline code. The archive includes the selected package directory without fetching vendor submodules.
+```bash
+pip install \
+  "laygen @ https://github.com/creative-graphic-design/design-generators/archive/9fff847022eeff922429df2fdd299e21408a0a40.zip#subdirectory=lib/laygen" \
+  "collage-diffusion @ https://github.com/creative-graphic-design/design-generators/archive/9fff847022eeff922429df2fdd299e21408a0a40.zip#subdirectory=models/collage-diffusion"
+```
+The Hub pipeline loads code from `creative-graphic-design/collage-diffusion` and weights from `sd2-community/stable-diffusion-2-1-base`. Before setting `trust_remote_code=True`, review the pipeline code in the [Collage Diffusion Hub repository](https://huggingface.co/creative-graphic-design/collage-diffusion); `diffusers` executes that code when it loads the custom pipeline.
+```python
+from PIL import Image
+from diffusers import DiffusionPipeline
+from collage_diffusion import CollageLayer
+base = "sd2-community/stable-diffusion-2-1-base"
+base_revision = "4e63672c03103b6c636b8fb4119ba982469b2955"
+pipe = DiffusionPipeline.from_pretrained(
+    base,
+    revision=base_revision,
+    custom_pipeline="creative-graphic-design/collage-diffusion",
+    trust_remote_code=True,
+)
+layer = CollageLayer(
+    image=Image.open("toy.png").convert("RGBA"),
+    prompt="a small wooden toy",
+    position=(64, 96),
+    noise_level=0.35,
+    positive_attention_weight=0.5,
+    negative_attention_weight=0.5,
+)
+result = pipe(
+    prompt="a small wooden toy on a studio table",
+    layers=[layer],
+    negative_prompt="collage",
+    seed=0,
+    num_inference_steps=20,
+)
+result.images[0].save("harmonized.png")
+print(type(pipe).__name__)
+```
+```text
+CollageDiffusionPipeline
+```
+The Hub card pins both source archives to the repository commit that published the pipeline. This package has no learned checkpoints; the pipeline reads base weights from the upstream Stable Diffusion repository and accepts caller-provided layers and optional textual-inversion embeddings.
+```bash
+git clone https://github.com/creative-graphic-design/design-generators.git
+cd design-generators
+uv sync --package collage-diffusion
+uv run --package collage-diffusion python
+```
+```python
+from diffusers import EulerAncestralDiscreteScheduler
+from collage_diffusion import CollageDiffusionPipeline
+base = "sd2-community/stable-diffusion-2-1-base"
+revision = "4e63672c03103b6c636b8fb4119ba982469b2955"
+# Cross-attention guidance uses the scheduler's sigma values.
+scheduler = EulerAncestralDiscreteScheduler.from_pretrained(
+    base, subfolder="scheduler", revision=revision
+)
+pipe = CollageDiffusionPipeline.from_pretrained(
+    base, revision=revision, scheduler=scheduler
+)
+print(type(pipe).__name__)
+```
+## Training Details
+### Training Data
+No training dataset is packaged. Layer images and optional textual-inversion embeddings are caller inputs.
+### Training Procedure
+This package contains no training procedure and no Collage Diffusion checkpoint.
+#### Training Hyperparameters
+- **Training regime:** Not applicable; the pipeline uses an upstream inference checkpoint.
+## Evaluation
+### Testing Data, Factors & Metrics
+#### Testing Data
+Ordinary tests use small synthetic RGBA layers. The comparison uses the Bento Box, Cake Table, and Toy Box scenes and their notebook-linked embeddings; public archive copies are linked above.
+#### Factors
+The reference comparison holds the input layers, preprocessing, textual-inversion embedding, scheduler, seed, and number of denoising steps constant between the notebook path and this package.
+#### Metrics
+Compare normalized final RGB arrays with RMSE, maximum absolute difference, and PSNR, and compare each post-scheduler, post-preservation latent with RMSE. The six comparisons use three 512 × 512 scenes and seeds 0 and 1, the same pinned checkpoint, each scene's layers and embeddings, Euler Ancestral, and 10 requested steps (7 or 8 active steps according to strength). Bento Box and Toy Box use strength `0.8` with `noise_blur=20`; Cake Table uses strength `0.75` with `noise_blur=10`.
+The notebook example omits `num_inference_steps`, so it uses the pipeline default of 50 steps (40 active at strength `0.8` and 37 at `0.75`). The parity run requests 10 steps to limit comparison cost (8 or 7 active); the notebook and package use identical settings within each comparison pair.
+### Parity Results
+An independent acceptance rerun passed all six scene-and-seed conditions against the authors' notebook. This is tolerance-verified agreement, not bitwise equality. The reference uses `diffusers` `0.10.0`, Torch `2.0.0+cpu`, torchvision `0.15.1+cpu`, and [`🤗transformers`](https://huggingface.co/docs/transformers/index) `4.28.1`; the package uses `diffusers` `0.39.0`, Torch `2.13.0+cpu`, torchvision `0.28.0+cpu`, and `transformers` `5.13.0`. Both comparison environments used CPU with one thread. The ordinary Stable Diffusion Img2Img controls and comparison plan are recorded in [issue #484](https://github.com/creative-graphic-design/design-generators/issues/484).
+Each threshold uses `T = 2·Bmax − Bmedian` over the six ordinary Stable Diffusion controls: RGB RMSE has `Bmax=6.00713e-5`, `Bmedian=1.61880e-5`, and `T=1.03955e-4`; maximum per-step latent RMSE has `Bmax=1.01644e-4`, `Bmedian=4.42415e-5`, and `T=1.59046e-4`.
+Each RMSE is compared with zero using `rtol=0`; the RGB criterion uses `atol=1.03955e-4` and the latent criterion uses `atol=1.59046e-4`.
+The implementation-run measurements below are recorded in this [issue #484 comment](https://github.com/creative-graphic-design/design-generators/issues/484#issuecomment-6636816601). Each row reports the final normalized RGB RMSE, PSNR (peak 1.0), maximum absolute RGB difference, maximum per-step post-scheduler/post-preservation latent RMSE, and first U-Net output difference (maximum absolute difference and RMSE).
+| Scene      | Seed | Active steps |           RGB RMSE |       PSNR (dB) |        RGB max abs | Max per-step latent RMSE | First U-Net max abs |   First U-Net RMSE |
+| ---------- | ---: | -----------: | -----------------: | --------------: | -----------------: | -----------------------: | ------------------: | -----------------: |
+| Bento Box  |    0 |            8 | `3.00031813940e-5` | `90.4566538465` | `4.99486923218e-4` |       `6.88358887783e-5` |  `2.07424163818e-5` | `6.67105892839e-7` |
+| Bento Box  |    1 |            8 | `5.48811255684e-6` | `105.211539803` | `8.64267349243e-5` |       `5.32430305242e-6` |  `1.99079513550e-5` | `6.69578823937e-7` |
+| Cake Table |    0 |            7 | `1.26997889517e-5` | `97.9240699240` | `3.05831432343e-4` |       `1.02028460064e-5` |  `9.88245010376e-5` | `1.79971479210e-6` |
+| Cake Table |    1 |            7 | `5.10831144860e-6` | `105.834452642` | `6.95586204529e-5` |       `7.19363812652e-6` |  `9.45329666138e-5` | `1.75942494680e-6` |
+| Toy Box    |    0 |            8 | `1.04906613643e-5` | `99.5839426331` | `1.33097171783e-4` |       `7.98777295022e-6` |  `3.38554382324e-5` | `9.17125873657e-7` |
+| Toy Box    |    1 |            8 | `1.17187871964e-5` | `98.6223466421` | `4.33802604675e-4` |       `1.76268145823e-5` |  `3.25739383698e-5` | `9.24976044259e-7` |
+The independent reviewer measured these per-condition results. Values are reported as RGB RMSE / maximum per-step latent RMSE; each is below the fixed RGB and latent thresholds above.
+| Scene      | Seed |           RGB RMSE | Maximum per-step latent RMSE |
+| ---------- | ---: | -----------------: | ---------------------------: |
+| Bento Box  |    0 | `9.85132492492e-6` |           `9.11307774482e-6` |
+| Bento Box  |    1 | `1.78408690247e-5` |           `5.76063468136e-6` |
+| Cake Table |    0 | `1.43160028885e-5` |           `7.43683456343e-6` |
+| Cake Table |    1 | `1.20999262219e-5` |            `7.7582077503e-6` |
+| Toy Box    |    0 | `2.03771846015e-5` |           `1.37336492006e-5` |
+| Toy Box    |    1 | `1.27287351725e-5` |           `1.90641440735e-5` |
+The Cake Table checkpoint-step control kept the notebook's per-layer inversion steps `[500, 1000, 1000, 1000, 100]` in layer order. Setting all five to `1000` changed the seed-0 RGB RMSE to `0.120`, showing that the selected checkpoint step for each layer affects the result.
+## Reproducibility
+See [REPRODUCING.md](https://github.com/creative-graphic-design/design-generators/blob/main/models/collage-diffusion/REPRODUCING.md) for CPU test and notebook-comparison steps.
+## Environmental Impact
+No model training is performed. CPU validation cost depends on the selected Colab memory profile, dependency installation, and comparison resolution.
+## Technical Specifications
+### Model Architecture and Objective
+The pipeline uses Stable Diffusion image-to-image components with a `diffusers` attention processor. It composites layers in back-to-front order, derives visible alpha masks, adjusts cross-attention logits for layer tokens, and blends the initial latent into each denoising step according to the inverted per-layer noise map.
+### Compute Infrastructure
+CPU is sufficient for small contract tests. Image generation uses the upstream Stable Diffusion checkpoint and is substantially slower on CPU than on a GPU.
+#### Hardware
+The requested implementation checks and image comparison run on a named Colab CPU session. No GPU is required for the planned comparison.
+#### Software
+Use Python 3.11 or newer, `diffusers`, `transformers`, PyTorch, Pillow, NumPy, and the repository's `laygen` workspace library. The pipeline uses the same workspace dependency configuration as other model packages.
+## License
+Package code follows the repository's Apache-2.0 license. The upstream notebook and UI source repositories do not declare a repository license; their implementations are referenced but not copied or distributed. Stable Diffusion 2.1 base is subject to its upstream CreativeML Open RAIL++-M terms. The original authors and rights holders of the notebook-linked example images and embeddings, and the license applicable to each file, are unknown. The archive records the URL from which each file was retrieved and its SHA-256; these records identify retrieval locations, not rights holders or license grants.
+## Citation
+```bibtex
+@article{sarukkai2023collagediffusion,
+  title = {Collage Diffusion},
+  author = {Sarukkai, Vishnu and Li, Linden and Ma, Arden and Ré, Christopher and Fatahalian, Kayvon},
+  journal = {arXiv preprint arXiv:2303.00262},
+  year = {2023},
+  eprint = {2303.00262},
+  archivePrefix = {arXiv},
+  primaryClass = {cs.CV},
+  url = "https://arxiv.org/abs/2303.00262",
+}
+```
+<!-- --8<-- [end:card] -->

pipeline.py ADDED Viewed

	@@ -0,0 +1,807 @@

+"""Diffusers pipeline and layer helpers for collage harmonization."""
+from __future__ import annotations
+import math
+from collections.abc import Callable, Mapping, Sequence
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Literal, Protocol
+import numpy as np
+import torch
+import torch.nn.functional as F
+from diffusers import StableDiffusionImg2ImgPipeline
+from diffusers.models.attention_processor import Attention
+from diffusers.pipelines.stable_diffusion.pipeline_output import (
+    StableDiffusionPipelineOutput,
+)
+from jaxtyping import Float
+from laygen.common.randomness import randn, resolve_torch_generator
+from PIL import Image, ImageFilter
+class TokenizerProtocol(Protocol):
+    """The tokenizer operation needed to locate layer phrases in a prompt."""
+    def encode(self, text: str, *, add_special_tokens: bool) -> list[int]:
+        """Encode text into token ids."""
+@dataclass(frozen=True, slots=True)
+class CollageLayer:
+    """One positioned RGBA layer and the controls associated with its prompt.
+    Args:
+        image: RGBA image for the layer.
+        prompt: Layer description that also appears in the scene prompt.
+        position: Pixel ``(x, y)`` position on the output canvas.
+        noise_level: Layer-specific amount of source preservation, in ``[0, 1]``.
+        positive_attention_weight: Attention-logit boost inside the visible mask.
+        negative_attention_weight: Attention-logit penalty outside the visible mask.
+        textual_inversion_token: Optional token inserted before ``prompt``.
+    Raises:
+        TypeError: If the image or position has the wrong type.
+        ValueError: If the image mode, prompt, or numeric control is invalid.
+    """
+    image: Image.Image
+    prompt: str
+    position: tuple[int, int] = (0, 0)
+    noise_level: float = 0.7
+    positive_attention_weight: float = 0.0
+    negative_attention_weight: float = 0.0
+    textual_inversion_token: str | None = None
+    def __post_init__(self) -> None:
+        """Validate layer input modes and scalar controls."""
+        if not isinstance(self.image, Image.Image):
+            raise TypeError("image must be a PIL Image")
+        if self.image.mode != "RGBA":
+            raise ValueError("image must use RGBA mode")
+        if not self.prompt.strip():
+            raise ValueError("prompt must be a non-empty layer description")
+        if len(self.position) != 2 or any(
+            not isinstance(value, int) or isinstance(value, bool)
+            for value in self.position
+        ):
+            raise TypeError("position must contain two integer pixel coordinates")
+        if not math.isfinite(self.noise_level) or not 0.0 <= self.noise_level <= 1.0:
+            raise ValueError("noise_level must be finite and between 0 and 1")
+        for name, weight in (
+            ("positive_attention_weight", self.positive_attention_weight),
+            ("negative_attention_weight", self.negative_attention_weight),
+        ):
+            if not math.isfinite(weight) or weight < 0:
+                raise ValueError(f"{name} must be finite and non-negative")
+        if (
+            self.textual_inversion_token is not None
+            and not self.textual_inversion_token.strip()
+        ):
+            raise ValueError("textual_inversion_token must be non-empty when provided")
+def _placed_alpha(
+    layer: CollageLayer,
+    *,
+    width: int,
+    height: int,
+) -> Float[np.ndarray, "height width"]:
+    canvas = Image.new("RGBA", (width, height), (0, 0, 0, 0))
+    # Using the RGBA image as its own paste mask multiplies partial alpha by itself.
+    canvas.paste(layer.image, layer.position, layer.image)
+    # Keep alpha arithmetic at float64 until conversion to the 8-bit noise map.
+    return np.asarray(canvas.getchannel("A"), dtype=np.float64) / 255.0
+def compose_collage_layers(
+    layers: Sequence[CollageLayer],
+    *,
+    width: int = 512,
+    height: int = 512,
+    background_color: tuple[int, int, int] = (150, 150, 150),
+) -> tuple[Image.Image, list[Float[np.ndarray, "height width"]]]:
+    """Composite layers back-to-front and return their visible alpha masks.
+    Args:
+        layers: Layers ordered from back to front.
+        width: Output canvas width in pixels.
+        height: Output canvas height in pixels.
+        background_color: RGB color used outside all layers.
+    Returns:
+        The RGB collage and one visible alpha mask per input layer.
+    Raises:
+        ValueError: If there are no layers or the canvas is invalid.
+    Examples:
+        >>> layer = CollageLayer(Image.new("RGBA", (2, 2), (255, 0, 0, 255)), "red square")
+        >>> image, masks = compose_collage_layers([layer], width=2, height=2)
+        >>> image.getpixel((0, 0)), float(masks[0][0, 0])
+        ((255, 0, 0), 1.0)
+    """
+    if not layers:
+        raise ValueError("layers must contain at least one RGBA image")
+    if width < 1 or height < 1:
+        raise ValueError("canvas width and height must be positive")
+    if len(background_color) != 3 or any(
+        not 0 <= channel <= 255 for channel in background_color
+    ):
+        raise ValueError("background_color must contain three byte values")
+    composite = Image.new("RGB", (width, height), background_color)
+    for layer in layers:
+        # Use each RGBA layer's alpha as its paste mask while compositing.
+        composite.paste(layer.image, layer.position, layer.image)
+    visible_masks: list[Float[np.ndarray, "height width"]] = []
+    for index, layer in enumerate(layers):
+        layer_alpha = _placed_alpha(layer, width=width, height=height)
+        top_canvas = Image.new("RGBA", (width, height), (0, 0, 0, 0))
+        for top_layer in layers[index + 1 :]:
+            top_canvas.paste(top_layer.image, top_layer.position, top_layer.image)
+        top_alpha = np.asarray(top_canvas.getchannel("A"), dtype=np.float64) / 255.0
+        visible_masks.append(layer_alpha * (1.0 - top_alpha))
+    return composite, visible_masks
+def make_layer_noise_map(
+    visible_masks: Sequence[Float[np.ndarray, "height width"]],
+    layers: Sequence[CollageLayer],
+    *,
+    strength: float,
+    blur_radius: float = 10.0,
+) -> Float[np.ndarray, "height width"]:
+    """Blend per-layer noise controls over the global image-to-image strength.
+    Args:
+        visible_masks: Visible alpha masks in back-to-front layer order.
+        layers: Layers whose ``noise_level`` values drive the map.
+        strength: Global image-to-image strength used outside layer masks.
+        blur_radius: Gaussian blur radius for smooth layer boundaries.
+    Returns:
+        A grayscale float map with values in ``[0, 1]``.
+    Raises:
+        ValueError: If masks, layers, or controls are inconsistent.
+    """
+    if len(visible_masks) != len(layers) or not layers:
+        raise ValueError("visible_masks and layers must have the same non-zero length")
+    if not math.isfinite(strength) or not 0.0 < strength <= 1.0:
+        raise ValueError("strength must be finite and between 0 and 1")
+    if not math.isfinite(blur_radius) or blur_radius < 0:
+        raise ValueError("blur_radius must be finite and non-negative")
+    noise_map = np.full_like(visible_masks[0], int(strength * 255), dtype=np.uint8)
+    for mask, layer in zip(visible_masks, layers, strict=True):
+        if mask.shape != noise_map.shape:
+            raise ValueError("all visible masks must have the same shape")
+        if not np.isfinite(mask).all() or np.any(mask < 0.0) or np.any(mask > 1.0):
+            raise ValueError("visible masks must contain finite values between 0 and 1")
+        layer_values = Image.fromarray(
+            (mask * layer.noise_level * 255).astype(np.uint8)
+        )
+        layer_mask = Image.fromarray((mask * 255).astype(np.uint8))
+        collage_map = Image.fromarray(noise_map)
+        collage_map.paste(layer_values, (0, 0), layer_mask)
+        noise_map = np.asarray(collage_map, dtype=np.uint8)
+    grayscale = Image.fromarray(noise_map)
+    if blur_radius > 0:
+        grayscale = grayscale.filter(ImageFilter.GaussianBlur(blur_radius))
+    return np.asarray(grayscale, dtype=np.float32) / 255.0
+def _find_token_span(
+    prompt_token_ids: Sequence[int],
+    layer_token_ids: Sequence[int],
+) -> tuple[int, ...]:
+    if not layer_token_ids:
+        raise ValueError("layer descriptions must produce at least one tokenizer token")
+    token_count = len(layer_token_ids)
+    for start in range(1, len(prompt_token_ids) - token_count + 1):
+        if tuple(prompt_token_ids[start : start + token_count]) == tuple(
+            layer_token_ids
+        ):
+            return tuple(range(start, start + token_count))
+    raise ValueError("each layer description must appear as a token span in the prompt")
+def _prompt_token_spans(
+    tokenizer: TokenizerProtocol,
+    prompt: str,
+    layers: Sequence[CollageLayer],
+) -> tuple[tuple[int, ...], ...]:
+    prompt_ids = tokenizer.encode(prompt, add_special_tokens=True)
+    spans: list[tuple[int, ...]] = []
+    textual_inversion_enabled = any(
+        layer.textual_inversion_token is not None for layer in layers
+    )
+    for layer in layers:
+        try:
+            layer_ids = tokenizer.encode(f" {layer.prompt}", add_special_tokens=False)
+            span = _find_token_span(prompt_ids, layer_ids)
+        except ValueError:
+            layer_ids = tokenizer.encode(layer.prompt, add_special_tokens=False)
+            span = _find_token_span(prompt_ids, layer_ids)
+        if textual_inversion_enabled:
+            # Extend each span's start one token left to include a prepended TI token.
+            span = tuple(range(span[0] - 1, span[-1] + 1))
+        spans.append(span)
+    return tuple(spans)
+def _inject_textual_inversion_tokens(
+    prompt: str,
+    layers: Sequence[CollageLayer],
+    embedding_sources: Mapping[str, str | Path] | None,
+) -> str:
+    requested_tokens = {
+        layer.textual_inversion_token
+        for layer in layers
+        if layer.textual_inversion_token is not None
+    }
+    supplied_tokens = set() if embedding_sources is None else set(embedding_sources)
+    unused_tokens = supplied_tokens - requested_tokens
+    if unused_tokens:
+        raise ValueError(
+            f"textual inversion embeddings have no matching layer: {sorted(unused_tokens)}"
+        )
+    for layer in layers:
+        token = layer.textual_inversion_token
+        if token is None:
+            continue
+        if prompt.count(layer.prompt) != 1:
+            raise ValueError(
+                "each layer description must occur exactly once in the scene prompt"
+            )
+        prompt = prompt.replace(layer.prompt, f"{token} {layer.prompt}", 1)
+    return prompt
+def _resize_mask_to_query(
+    mask: Float[np.ndarray, "height width"],
+    *,
+    size: tuple[int, int],
+    device: torch.device,
+) -> Float[torch.Tensor, "query"]:
+    """Resize a visible mask with the reference tensor-sampling semantics."""
+    mask_tensor = torch.as_tensor(mask, device=device, dtype=torch.float64)
+    batched_mask = mask_tensor.unsqueeze(0).unsqueeze(0)
+    resized_mask = F.interpolate(
+        batched_mask,
+        size=size,
+        mode="bilinear",
+        align_corners=False,
+        antialias=False,
+    )
+    resized_mask = resized_mask.squeeze(0).squeeze(0)
+    query_length = size[0] * size[1]
+    return resized_mask.reshape(query_length)
+class _CollageCrossAttentionProcessor:
+    """Apply visible-layer logit guidance through Diffusers' processor API."""
+    def __init__(
+        self,
+        *,
+        visible_masks: Sequence[Float[np.ndarray, "height width"]],
+        token_spans: Sequence[Sequence[int]],
+        layers: Sequence[CollageLayer],
+        output_size: tuple[int, int],
+        batch_size: int,
+        classifier_free_guidance: bool,
+    ) -> None:
+        self.visible_masks = tuple(visible_masks)
+        self.token_spans = tuple(tuple(span) for span in token_spans)
+        self.layers = tuple(layers)
+        self.output_size = output_size
+        self.batch_size = batch_size
+        self.classifier_free_guidance = classifier_free_guidance
+        self.sigma = 1.0
+    def __call__(
+        self,
+        attn: Attention,
+        hidden_states: Float[torch.Tensor, "batch sequence channels"],
+        encoder_hidden_states: Float[torch.Tensor, "batch tokens channels"]
+        | None = None,
+        attention_mask: Float[torch.Tensor, "..."] | None = None,
+        temb: Float[torch.Tensor, "..."] | None = None,
+        scale: float | None = None,
+    ) -> Float[torch.Tensor, "batch sequence channels"]:
+        # Add per-layer score bias before softmax; CFG leaves unconditional rows unchanged.
+        del scale
+        residual = hidden_states
+        spatial_dimensions: tuple[int, int, int] | None = None
+        if attn.spatial_norm is not None:
+            hidden_states = attn.spatial_norm(hidden_states, temb)
+        input_ndim = hidden_states.ndim
+        if input_ndim == 4:
+            batch_size, channel, height, width = hidden_states.shape
+            spatial_dimensions = (channel, height, width)
+            hidden_states = hidden_states.view(batch_size, channel, height * width)
+            hidden_states = hidden_states.transpose(1, 2)
+        batch_size, sequence_length, _ = (
+            hidden_states.shape
+            if encoder_hidden_states is None
+            else encoder_hidden_states.shape
+        )
+        if attention_mask is not None:
+            attention_mask = attn.prepare_attention_mask(
+                attention_mask, sequence_length, batch_size
+            )
+            attention_mask = attention_mask.view(
+                batch_size, attn.heads, -1, attention_mask.shape[-1]
+            )
+        if attn.group_norm is not None:
+            hidden_states = hidden_states.transpose(1, 2)
+            hidden_states = attn.group_norm(hidden_states)
+            hidden_states = hidden_states.transpose(1, 2)
+        query = attn.to_q(hidden_states)
+        if encoder_hidden_states is None:
+            encoder_hidden_states = hidden_states
+        elif attn.norm_cross:
+            encoder_hidden_states = attn.norm_encoder_hidden_states(
+                encoder_hidden_states
+            )
+        to_k = attn.to_k
+        to_v = attn.to_v
+        to_out = attn.to_out
+        if to_k is None or to_v is None or to_out is None:
+            raise ValueError(
+                "cross-attention requires key, value, and output projections"
+            )
+        key = to_k(encoder_hidden_states)
+        value = to_v(encoder_hidden_states)
+        head_dim = key.shape[-1] // attn.heads
+        query = query.view(batch_size, -1, attn.heads, head_dim)
+        query = query.transpose(1, 2)
+        key = key.view(batch_size, -1, attn.heads, head_dim)
+        key = key.transpose(1, 2)
+        value = value.view(batch_size, -1, attn.heads, head_dim)
+        value = value.transpose(1, 2)
+        if attn.norm_q is not None:
+            query = attn.norm_q(query)
+        if attn.norm_k is not None:
+            key = attn.norm_k(key)
+        score_query = query.float() if attn.upcast_attention else query
+        score_key = key.float() if attn.upcast_attention else key
+        scores = torch.matmul(score_query, score_key.transpose(-1, -2)) * attn.scale
+        if attention_mask is not None:
+            scores = scores + attention_mask
+        self._apply_layer_guidance(scores)
+        softmax_scores = scores.float() if attn.upcast_softmax else scores
+        attention_probs = softmax_scores.softmax(dim=-1).to(value.dtype)
+        output = torch.matmul(attention_probs, value)
+        output = output.transpose(1, 2)
+        output = output.reshape(batch_size, -1, attn.heads * head_dim)
+        output = output.to(query.dtype)
+        output = to_out[0](output)
+        output = to_out[1](output)
+        if spatial_dimensions is not None:
+            channel, height, width = spatial_dimensions
+            output = output.transpose(-1, -2)
+            output = output.reshape(batch_size, channel, height, width)
+        if attn.residual_connection:
+            output = output + residual
+        return output / attn.rescale_output_factor
+    def _apply_layer_guidance(
+        self, scores: Float[torch.Tensor, "batch heads query tokens"]
+    ) -> None:
+        query_length = scores.shape[-2]
+        output_height, output_width = self.output_size
+        target_height = max(
+            1, int(round(math.sqrt(query_length * output_height / output_width)))
+        )
+        while query_length % target_height:
+            target_height -= 1
+        target_width = query_length // target_height
+        latent_masks = [
+            _resize_mask_to_query(
+                mask,
+                size=(target_height, target_width),
+                device=scores.device,
+            )
+            for mask in self.visible_masks
+        ]
+        # CFG batches unconditional rows before conditional rows; guide only the latter.
+        cfg_offset = self.batch_size if self.classifier_free_guidance else 0
+        sigma_scale = math.log(math.log(max(self.sigma, 0.0) + 1.01) + 1.0)
+        for batch_index in range(self.batch_size):
+            conditional_index = batch_index + cfg_offset
+            for mask, span, layer in zip(
+                latent_masks, self.token_spans, self.layers, strict=True
+            ):
+                for token_index in span:
+                    if token_index >= scores.shape[-1]:
+                        raise ValueError(
+                            "layer token span exceeds the text encoder sequence length"
+                        )
+                    token_scores = scores[conditional_index, :, :, token_index]
+                    max_score = token_scores.max()
+                    visible_mask = mask.unsqueeze(0)
+                    non_visible_mask = 1.0 - visible_mask
+                    positive_bias = visible_mask * layer.positive_attention_weight
+                    negative_bias = non_visible_mask * layer.negative_attention_weight
+                    bias = (positive_bias - negative_bias) * sigma_scale * max_score
+                    scores[conditional_index, :, :, token_index] += bias
+class CollageDiffusionPipeline(StableDiffusionImg2ImgPipeline):
+    """Harmonize positioned image layers with layer-aware diffusion controls.
+    The pipeline loads the standard Stable Diffusion image-to-image components and
+    replaces only the sampling orchestration needed for visible-layer attention
+    and noise guidance.
+    Examples:
+        >>> from PIL import Image
+        >>> from collage_diffusion import CollageLayer
+        >>> layer = CollageLayer(Image.new("RGBA", (16, 16), (220, 80, 30, 255)), "orange toy")
+        >>> layer.prompt
+        'orange toy'
+    """
+    @torch.no_grad()
+    def __call__(
+        self,
+        prompt: str,
+        layers: Sequence[CollageLayer],
+        *,
+        negative_prompt: str | None = None,
+        strength: float = 0.7,
+        num_inference_steps: int = 50,
+        guidance_scale: float = 7.5,
+        num_images_per_prompt: int = 1,
+        generator: torch.Generator | None = None,
+        seed: int | None = None,
+        textual_inversion_embeddings: Mapping[str, str | Path] | None = None,
+        height: int = 512,
+        width: int = 512,
+        noise_blur: float = 10.0,
+        eta: float = 0.0,
+        output_type: Literal["pil", "np"] = "pil",
+        return_dict: bool = True,
+        step_callback: Callable[
+            [int, float, Float[torch.Tensor, "batch channels height width"]], None
+        ]
+        | None = None,
+    ) -> (
+        StableDiffusionPipelineOutput
+        | tuple[
+            list[Image.Image] | Float[np.ndarray, "batch height width channels"],
+            list[bool] | None,
+        ]
+    ):
+        """Generate a harmonized collage from an ordered set of RGBA layers.
+        Args:
+            prompt: Global scene description containing every layer description.
+            layers: Back-to-front RGBA layers with descriptions and controls.
+            negative_prompt: Optional text to suppress during classifier-free guidance.
+            strength: Global image-to-image strength in ``(0, 1]``.
+            num_inference_steps: Number of scheduler steps before strength adjustment.
+            guidance_scale: Classifier-free guidance scale.
+            num_images_per_prompt: Number of images to generate for the prompt.
+            generator: Explicit random generator; takes precedence over ``seed``.
+            seed: Optional local seed used when ``generator`` is absent.
+            textual_inversion_embeddings: Map from layer token to caller-provided weights.
+            height: Output canvas height; must be divisible by the VAE scale factor.
+            width: Output canvas width; must be divisible by the VAE scale factor.
+            noise_blur: Gaussian blur radius for the blended per-layer noise map.
+            eta: Optional scheduler noise parameter used by DDIM-compatible schedulers.
+            output_type: ``"pil"`` or ``"np"`` image output format.
+            return_dict: Whether to return ``StableDiffusionPipelineOutput``.
+            step_callback: Optional callback receiving step index, timestep, and latents.
+        Returns:
+            A Diffusers image output or its standard tuple representation.
+        Raises:
+            TypeError: If a layer sequence contains an unsupported value.
+            ValueError: If the scheduler does not expose sigmas or generation inputs are invalid.
+        """
+        # Add layer-aware attention guidance and restore source regions during denoising.
+        # 1. Validate scheduler, prompts, layers, and generation settings.
+        if getattr(self.scheduler, "sigmas", None) is None:
+            raise ValueError(
+                "scheduler must expose `sigmas` for cross-attention guidance; "
+                "use EulerAncestralDiscreteScheduler"
+            )
+        if not prompt.strip():
+            raise ValueError("prompt must be non-empty")
+        if not layers:
+            raise ValueError("layers must contain at least one layer")
+        if any(not isinstance(layer, CollageLayer) for layer in layers):
+            raise TypeError("layers must contain CollageLayer values")
+        if not math.isfinite(strength) or not 0.0 < strength <= 1.0:
+            raise ValueError("strength must be finite and between 0 and 1")
+        if num_inference_steps < 1 or num_images_per_prompt < 1:
+            raise ValueError(
+                "num_inference_steps and num_images_per_prompt must be positive"
+            )
+        if not math.isfinite(guidance_scale) or guidance_scale < 0.0:
+            raise ValueError("guidance_scale must be finite and non-negative")
+        if height < 1 or width < 1:
+            raise ValueError("height and width must be positive")
+        if output_type not in {"pil", "np"}:
+            raise ValueError("output_type must be 'pil' or 'np'")
+        device = self._execution_device
+        vae_scale_factor = self.vae_scale_factor
+        if height % vae_scale_factor or width % vae_scale_factor:
+            raise ValueError(
+                "height and width must be divisible by the VAE scale factor"
+            )
+        # 2. Prepare layer prompts, collage image, visible masks, and noise map.
+        generator = resolve_torch_generator(generator=generator, seed=seed)
+        prompt = _inject_textual_inversion_tokens(
+            prompt, layers, textual_inversion_embeddings
+        )
+        if textual_inversion_embeddings is not None:
+            for token, embedding_source in textual_inversion_embeddings.items():
+                self.load_textual_inversion(embedding_source, token=token)
+        token_spans = _prompt_token_spans(self.tokenizer, prompt, layers)
+        collage, visible_masks = compose_collage_layers(
+            layers, width=width, height=height
+        )
+        noise_map = make_layer_noise_map(
+            visible_masks, layers, strength=strength, blur_radius=noise_blur
+        )
+        image = self.image_processor.preprocess(collage, height=height, width=width)
+        image = image.to(device=device, dtype=self.unet.dtype)
+        # 3. Encode the prompt and batch both guidance branches when guidance_scale > 1.
+        do_classifier_free_guidance = guidance_scale > 1.0
+        prompt_embeds, negative_prompt_embeds = self.encode_prompt(
+            prompt,
+            device,
+            num_images_per_prompt,
+            do_classifier_free_guidance,
+            negative_prompt=negative_prompt,
+        )
+        if do_classifier_free_guidance:
+            if negative_prompt_embeds is None:
+                raise ValueError(
+                    "classifier-free guidance requires negative prompt embeddings"
+                )
+            # Concatenate both branches so the U-Net handles them in one call.
+            prompt_embeds = torch.cat([negative_prompt_embeds, prompt_embeds])
+        # 4. Encode the collage and prepare its initial latent and noise.
+        encoded = self.vae.encode(image)
+        init_latents = encoded.latent_dist.sample(generator=generator)
+        init_latents = init_latents * self.vae.config.scaling_factor
+        init_latents = init_latents.repeat(num_images_per_prompt, 1, 1, 1)
+        initial_noise = randn(
+            init_latents.shape,
+            generator=generator,
+            device=device,
+            dtype=init_latents.dtype,
+        )
+        # 5. Set img2img timesteps and build the latent-resolution preservation mask.
+        self.scheduler.set_timesteps(num_inference_steps, device=device)
+        scheduler_offset = int(self.scheduler.config.get("steps_offset", 0))
+        init_timestep = min(
+            int(num_inference_steps * strength) + scheduler_offset,
+            num_inference_steps,
+        )
+        timestep_start = max(
+            num_inference_steps - init_timestep + scheduler_offset,
+            0,
+        )
+        timesteps = self.scheduler.timesteps[timestep_start:]
+        effective_steps = num_inference_steps - timestep_start
+        if len(timesteps) == 0:
+            raise ValueError(
+                "strength and num_inference_steps produced no denoising steps"
+            )
+        latent_timestep = timesteps[:1].repeat(num_images_per_prompt)
+        latents = self.scheduler.add_noise(init_latents, initial_noise, latent_timestep)
+        noise_mask_pixels = np.rint(noise_map * 255.0).astype(np.uint8)
+        # Quantize the source strength map before resizing it to the latent grid.
+        noise_mask_image = Image.fromarray(noise_mask_pixels)
+        noise_mask_image = noise_mask_image.resize(
+            (latents.shape[-1], latents.shape[-2]),
+            resample=Image.Resampling.NEAREST,
+        )
+        noise_mask_values = np.asarray(noise_mask_image, dtype=np.float32)
+        noise_mask_values = noise_mask_values.copy()
+        normalized_noise_mask = noise_mask_values / 255.0
+        preservation_mask = 1.0 - normalized_noise_mask
+        preservation_mask_tensor = torch.as_tensor(
+            preservation_mask, device=device, dtype=latents.dtype
+        )
+        batched_preservation_mask = preservation_mask_tensor.unsqueeze(0).unsqueeze(0)
+        noise_mask = batched_preservation_mask.expand(
+            num_images_per_prompt, 1, *latents.shape[-2:]
+        )
+        # 6. Install layer-aware attention processors and prepare scheduler arguments.
+        previous_processors = dict(self.unet.attn_processors)
+        controlled_processors = {
+            name: _CollageCrossAttentionProcessor(
+                visible_masks=visible_masks,
+                token_spans=token_spans,
+                layers=layers,
+                output_size=(height, width),
+                batch_size=num_images_per_prompt,
+                classifier_free_guidance=do_classifier_free_guidance,
+            )
+            if name.endswith(".attn2.processor")
+            else processor
+            for name, processor in previous_processors.items()
+        }
+        controlled_attention_processors = tuple(
+            processor
+            for processor in controlled_processors.values()
+            if isinstance(processor, _CollageCrossAttentionProcessor)
+        )
+        attention_control_enabled = any(
+            layer.positive_attention_weight or layer.negative_attention_weight
+            for layer in layers
+        )
+        if attention_control_enabled and not controlled_attention_processors:
+            raise ValueError("the U-Net exposes no cross-attention processors")
+        if attention_control_enabled:
+            self.unet.set_attn_processor(controlled_processors)
+        extra_step_kwargs = self.prepare_extra_step_kwargs(generator, eta)
+        self._num_timesteps = len(timesteps)
+        try:
+            # 7. Denoise, applying attention guidance and layer-specific preservation.
+            with self.progress_bar(total=effective_steps) as progress_bar:
+                for step_index, timestep in enumerate(timesteps):
+                    if attention_control_enabled:
+                        sigma = float(self.scheduler.sigmas[step_index].item())
+                        for processor in controlled_attention_processors:
+                            processor.sigma = sigma
+                    latent_model_input = (
+                        torch.cat([latents] * 2)
+                        if do_classifier_free_guidance
+                        else latents
+                    )
+                    latent_model_input = self.scheduler.scale_model_input(
+                        latent_model_input, timestep
+                    )
+                    noise_prediction = self.unet(
+                        latent_model_input,
+                        timestep,
+                        encoder_hidden_states=prompt_embeds,
+                    ).sample
+                    if do_classifier_free_guidance:
+                        # Standard CFG: unconditional + scale * (conditional - unconditional).
+                        unconditional, conditional = noise_prediction.chunk(2)
+                        noise_prediction = unconditional + guidance_scale * (
+                            conditional - unconditional
+                        )
+                    # Advance x_t to x_(t-1); the mask below restores source regions.
+                    latents = self.scheduler.step(
+                        noise_prediction,
+                        timestep,
+                        latents,
+                        **extra_step_kwargs,
+                    ).prev_sample
+                    step_noise = randn(
+                        latents.shape,
+                        generator=generator,
+                        device=device,
+                        dtype=latents.dtype,
+                    )
+                    timestep_batch = timestep.expand(latents.shape[0])
+                    preserved_latents = self.scheduler.add_noise(
+                        init_latents, step_noise, timestep_batch
+                    )
+                    timestep_value = float(timestep.item())
+                    # Apply the timestep-dependent preservation threshold and spread it to neighbors.
+                    mask_t = (noise_mask > 1.0 - timestep_value / 1000.0).to(
+                        latents.dtype
+                    )
+                    dilation = int(4.0 * timestep_value / 1000.0)
+                    if dilation > 0:
+                        mask_t = F.max_pool2d(
+                            mask_t,
+                            kernel_size=dilation * 2 + 1,
+                            stride=1,
+                            padding=dilation,
+                        )
+                    latents = preserved_latents * mask_t + latents * (1.0 - mask_t)
+                    if step_callback is not None:
+                        step_callback(step_index, timestep_value, latents)
+                    progress_bar.update()
+        finally:
+            if attention_control_enabled:
+                self.unet.set_attn_processor(previous_processors)
+        # 8. Decode the final latent and return the standard Diffusers output.
+        decoded = self.vae.decode(
+            latents / self.vae.config.scaling_factor, return_dict=False
+        )[0]
+        decoded, has_nsfw_concept = self.run_safety_checker(
+            decoded, device, prompt_embeds.dtype
+        )
+        do_denormalize = (
+            [not flag for flag in has_nsfw_concept]
+            if has_nsfw_concept is not None
+            else None
+        )
+        images = self.image_processor.postprocess(
+            decoded, output_type=output_type, do_denormalize=do_denormalize
+        )
+        self.maybe_free_model_hooks()
+        if not return_dict:
+            return images, has_nsfw_concept
+        return StableDiffusionPipelineOutput(
+            images=images,
+            nsfw_content_detected=has_nsfw_concept,
+        )