README.md

---

title: Workflow1111 Diffusion Studio
emoji: 🎨
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.22.0
app_file: app.py
pinned: false
license: mit
hf_oauth: true
hf_oauth_scopes:
  - inference-api
short_description: Automatic1111-style studio on one gr.Workflow canvas
---


> **Sign in with Hugging Face** (button at the top right) before running
> anything. This Space carries no token of its own — every `model`, `space` and
> inference-calling `fn` node runs on *your* token and your own inference quota.

# 05 · Workflow1111 — a Diffusion Studio built from `gr.Workflow`

An **Automatic1111-shaped image studio expressed as one canvas graph** instead
of a tabbed UI: **64 nodes, 10 pipelines, 18 independently runnable outputs**,
all inside a single `gr.Workflow`.

```bash

pip install -r apps/05_workflow1111/requirements.txt

hf auth login                       # or: set HF_TOKEN=hf_xxx

python apps/05_workflow1111/app.py

```

---

## Why it isn't tabs

`gr.Workflow` raises if you construct it inside a `gr.Blocks` context:

```python

if Context.root_block is not None:

    raise ValueError("gr.Workflow cannot be created inside another gr.Blocks context.")

```

So an A1111 clone genuinely *cannot* be a `gr.Tabs` layout here — the graph **is**
the UI. The analogue of a tab is a **subject group**: a connected cluster of
outputs, which gradio also publishes as its own REST endpoint. Ten pipelines sit
side by side on one canvas, and you run whichever output you want.

---

## What's on the canvas

| # | Pipeline | A1111 equivalent | Nodes |
|---|----------|------------------|-------|
| 1 | **txt2img** | txt2img tab | prompt builder → negative builder → sampler → FLUX.1-schnell → post-processing → image + params |
| 2 | **Hires fix** | Hires. fix | txt2img result → prep → FLUX.1-Kontext re-render |
| 3 | **img2img** | img2img tab | upload → prep → FLUX.1-Kontext edit → post-processing |
| 4 | **Prompt magic** | — | idea → instruction → Qwen3-4B → cleanup |
| 5 | **Interrogate** | CLIP interrogate | image → Qwen2.5-VL → prompt; + ViT classification |
| 6 | **Detect & mask** | inpaint masking | DETR → annotated boxes → feathered inpaint mask |
| 7 | **Prompt matrix** | X/Y/Z plot | 4 variants → 4 parallel renders → contact sheet |
| 8 | **Extras** | Extras tab | local Lanczos upscale · AuraSR ×4 · background removal |
| 9 | **Annotators** | ControlNet preprocessors | Canny · line art · sketch · luma-depth · posterize · threshold |
| 10 | **PNG Info** | PNG Info tab | read generation parameters back out of a file |

**The txt2img node has the real control surface** — negative prompt, sampling
steps, CFG scale, seed (with `-1` = random), width/height with aspect presets,
and a `model_id` box that acts as the checkpoint selector — not just a prompt
box. Keeping that surface intact in the browser is exactly what gotcha #1 below
is about.

**The generation-parameters loop closes.** `postprocess` writes the A1111
parameter block into the PNG's `parameters` text chunk; the PNG Info pipeline
parses it back out. Images this app makes round-trip through a real A1111
install too.

---

## Architecture

```

14 references  →  32 operators  →  18 subjects        73 edges

                  ├─ 28 fn      19 pure-local · 9 calling InferenceClient

                  ├─  2 model   HF Inference Providers

                  └─  2 space   Gradio Spaces on the Hub

```

19 of the 28 `fn` nodes are pure local Pillow/numpy — all the prompt logic,
post-processing, annotators, masking, grid composition and metadata parsing —
so most of the app keeps working with no token, no quota and no network. 13
nodes in total leave the machine.

### Files

| File | What it is |
|---|---|
| `app.py` | Entry point — 12 lines of actual wiring |
| `nodes.py` | The 21 bound functions (the `fn` node library) |
| `build_workflow.py` | **Generates + verifies** `workflow.json` |
| `workflow.json` | The committed graph |
| `test_nodes.py` | 53 offline unit tests (~2s) |
| `test_pipelines.py` | Runs all 18 outputs through the real `WorkflowExecutor` |
| `test_api.py` | Drives the 9 generated REST endpoints against a running app |
| `make_samples.py` | Regenerates the shipped sample images |
| `deploy_space.py` | Stages + uploads the Space |
| `layout.json` | The curated node positions |
| `samples/` | Sample images used as reference-node defaults |

`build_workflow.py` derives each `fn` node's input ports from the bound
function's own signature via `inspect`, so **port order can never drift from

the Python argument order** — the executor passes `fn` arguments positionally,
in port order, and that mismatch is the easiest bug to introduce by hand. It
then refuses to write the file unless every edge resolves, every type matches,
every required input is wired or defaulted, no input has two incoming edges,
and no node is orphaned.

---

## Six gotchas this app is built around

All six were found by probing gradio 6.22.0 / huggingface_hub 1.26.0

directly, not from the docs. They are the difference between "renders on the

canvas" and "actually runs".



### 1. The canvas rewrites `model` node ports — silently



This is the big one, and it is invisible until you open the graph in a browser.

The canvas normalizes every `model` node's input ports to the endpoint's

canonical schema in `_INFERENCE_ENDPOINT_SCHEMAS`, **saves the result back over

`workflow.json`**, and leaves the now-dangling edges in place.

`text_to_image`'s schema is just `["prompt"]`. So a txt2img node carrying
`negative_prompt`, `num_inference_steps`, `guidance_scale`, `seed`, `width` and
`height` — which works perfectly through the headless executor and the REST API
— loses all six the instant a browser loads it. No error, no warning; the image
just quietly ignores every setting. `chat_completion` normalizes to
`["image", "text"]`, which had the same effect on the prompt LLM (it started
replying "Hello! It seems like your message might be missing something").

The fix is to stop using `model` nodes wherever the control surface is richer
than the schema: `txt2img`, `chat_llm` and `interrogate` are `fn` nodes that
call `InferenceClient` themselves. `fn` ports are never rewritten. The two
remaining `model` nodes (`image_to_image`) have ports exactly equal to their
schema, and
`build_workflow.py` now **refuses to build** if that ever stops being true.

A useful side effect: an `fn` node can validate. `interrogate` requires its
image, because the `model` node version cheerfully described an image it was
never given — a fabricated result that looked entirely successful.

### 2. Image ports do not chain uniformly

`model` nodes emit `{"path", "url": "/gradio_api/file=<abs path>", "is_file": true}`,
and `_img_url()` prefers the `url` key — which means nothing to a remote
provider:

| chain | works? |
|---|---|
| model image → another model's `image_to_image` | ❌ `File not found at \gradio_api\file=...` |
| `data:` URI → model task endpoint | ✅ |
| `data:` URI → **space** | ❌ `call_space` only calls `handle_file` on dicts |
| `{"path": p}` (no `url`) → model or space | ✅ |
| model image → `chat_completion` VLM | ✅ (`_chat_image_url` strips the prefix) |
| uploaded reference → space or model | ✅ |

So: **`fn` nodes emit `{"path", "url"}` carrying a real file *and* a `data:`

URI** (see gotcha #5), **`prep_image` sits between any two model nodes**, and

**`space` nodes only ever take an uploaded image**.



### 3. Several obvious backends simply don't work



Probed live against this account's enabled providers:



- `guidance_scale=0` on FLUX → **422**, fal-ai requires `>= 1`. The sampler node clamps it.

- `stable-diffusion-3.5-large-turbo` → 504 after ~120s.

- `Salesforce/blip-image-captioning-large`, `Qwen2.5-VL-7B`, `Llama-3.2-*`,

  `Mistral-7B-Instruct-v0.3` → no enabled provider.

- **depth-estimation is dead**: `InferenceClient` has no `depth_estimation`

  method, and the positional fallback targets `api-inference.huggingface.co`,

  whose DNS no longer resolves. That's why the annotator's depth mode is an

  honest luminance approximation rather than a monocular depth model — the edge

  modes are the genuine article.



Working and fast: FLUX.1-schnell (~3s), FLUX.1-dev (~3.5s), FLUX.1-Kontext-dev

(~6s), Qwen3-4B-Instruct (~1.2s), Qwen2.5-VL-72B (~7s), DETR + ViT (~5s).



### 4. `ImageSlider` outputs are tuples



Both Spaces return a before/after pair, so the cutout / upscaled result is at

`output_index: 1`, verified by calling them rather than reading their docs.



### 5. One image value has to satisfy four different consumers



An `fn` node's image output is consumed four ways, and they disagree:



| consumer | wants |

|---|---|

| REST endpoint (`gr.Image` component) | a real file path — a `data:` URI gets treated as a *filename* and joined to the CWD, raising `OSError: [Errno 22]` |

| canvas | a `url` it can display |

| chained `model` node (`_img_url`) | something a remote provider can fetch |

| chained `fn` node (`_load_image`) | anything, prefers `path` |



So `_emit` returns **both**: `{"path": <temp file>, "url": <data: URI>}`.

`_from_output` takes `path` first (endpoint happy), the frontend and `_img_url`

take `url` first (canvas and providers happy).



### 6. A `json` port silently destroys its value in the canvas



The canvas serializes a `json`-typed port with JavaScript's `String(obj)`

instead of `JSON.stringify`, so the receiving node gets the literal six-word

string `"[object Object]"`. Everything downstream then sees *no data*, with no

error anywhere:



- DETR detections reached `draw_detections` as `"[object Object]"` → zero boxes

  → an annotated image identical to the input, and

  `mask_from_detections` failing with "No detections matched" at **every**
  `min_score`.
- ViT labels reached `top_labels` the same way → "No labels above the score
  threshold".
- `png_info`'s field dict reached its output node as `"[object Object]"`.

Both the executor and the REST API handle `json` ports perfectly, so this is
invisible to `test_pipelines.py` *and* `test_api.py` — only the canvas is
affected. The graph therefore contains **no `json` ports at all**: structured
data travels as JSON *text*, which survives, and `_as_list` parses it back.
`detect_objects` and `classify_image` are `fn` nodes calling `InferenceClient`
for the same reason a `model` node could not be used (their output port type is
fixed by the endpoint schema — gotcha #1).

Reference-node *defaults* are a separate case with the opposite answer: the
canvas strips `path` out of a graph default and keeps only `url`, so the sample
images are referenced by their **public Hub URL** — the one form that renders
*and* that `handle_file` / `InferenceClient` can fetch. That is also why the
Space must stay public.

---

## Tests

```bash

python apps/05_workflow1111/test_nodes.py             # 53 unit tests, offline, ~2s

python apps/05_workflow1111/test_pipelines.py         # all 18 outputs, hits HF

python apps/05_workflow1111/test_pipelines.py local   # only the offline outputs

python apps/05_workflow1111/test_api.py               # the 9 REST endpoints

python apps/05_workflow1111/test_api.py --local       # endpoints needing no token

```

All three layers matter, and they catch different things. `test_pipelines.py`
drives `WorkflowExecutor` directly, so it skips gradio's **output component**
postprocessing — which is precisely where an image port's value shape is
validated. `test_api.py` is the only layer that catches gotcha #5.

`test_pipelines.py` writes every result to `_test_output/` so you can look at
it. Current status: **53/53 unit, 18/18 executor, 9/9 REST**, plus a full run in
the live canvas covering `fn`, `model`, `space`, uploads and image chaining.

---

## Using it as an API

Every subject group is a REST endpoint, so the studio is scriptable:

```python

from gradio_client import Client



client = Client("http://127.0.0.1:7860")

image, params, hires = client.predict(

    "a red fox in a snowy pine forest",   # Prompt

    "",                                    # Negative prompt

    "Cinematic",                           # Style preset

    "enhance fine detail",                 # Hires refine instruction

    api_name="/image",

)

```

Run `python apps/05_workflow1111/test_pipelines.py local` to print the full
endpoint list with parameter names and types.

---

## Extending it

Change a checkpoint by editing the constants at the top of `build_workflow.py`
(`T2I_MODEL`, `EDIT_MODEL`, …) and re-running it. Add a node by writing the
function in `nodes.py`, registering it in `BIND`, and adding an `fn(...)` call
plus `link(...)`s — the verifier will tell you what you got wrong before the
file is written.

> Re-running `build_workflow.py` overwrites node positions, so any layout you

> drag around in the canvas is reset.