README.md

metadata

title: Workflow1111 Diffusion Studio
emoji: 🎨
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.22.0
app_file: app.py
pinned: false
license: mit
hf_oauth: true
hf_oauth_scopes:
  - inference-api
short_description: Automatic1111-style studio on one gr.Workflow canvas

Sign in with Hugging Face (button at the top right) before running anything. This Space carries no token of its own — every model, space and inference-calling fn node runs on your token and your own inference quota.

05 · Workflow1111 — a Diffusion Studio built from gr.Workflow

An Automatic1111-shaped image studio expressed as one canvas graph instead of a tabbed UI: 64 nodes, 10 pipelines, 18 independently runnable outputs, all inside a single gr.Workflow.

pip install -r apps/05_workflow1111/requirements.txt
hf auth login                       # or: set HF_TOKEN=hf_xxx
python apps/05_workflow1111/app.py

---

Why it isn't tabs

gr.Workflow raises if you construct it inside a gr.Blocks context:

if Context.root_block is not None:
    raise ValueError("gr.Workflow cannot be created inside another gr.Blocks context.")

So an A1111 clone genuinely cannot be a gr.Tabs layout here — the graph is the UI. The analogue of a tab is a subject group: a connected cluster of outputs, which gradio also publishes as its own REST endpoint. Ten pipelines sit side by side on one canvas, and you run whichever output you want.

---

What's on the canvas

#	Pipeline	A1111 equivalent	Nodes
1	txt2img	txt2img tab	prompt builder → negative builder → sampler → FLUX.1-schnell → post-processing → image + params
2	Hires fix	Hires. fix	txt2img result → prep → FLUX.1-Kontext re-render
3	img2img	img2img tab	upload → prep → FLUX.1-Kontext edit → post-processing
4	Prompt magic	—	idea → instruction → Qwen3-4B → cleanup
5	Interrogate	CLIP interrogate	image → Qwen2.5-VL → prompt; + ViT classification
6	Detect & mask	inpaint masking	DETR → annotated boxes → feathered inpaint mask
7	Prompt matrix	X/Y/Z plot	4 variants → 4 parallel renders → contact sheet
8	Extras	Extras tab	local Lanczos upscale · AuraSR ×4 · background removal
9	Annotators	ControlNet preprocessors	Canny · line art · sketch · luma-depth · posterize · threshold
10	PNG Info	PNG Info tab	read generation parameters back out of a file

The txt2img node has the real control surface — negative prompt, sampling steps, CFG scale, seed (with -1 = random), width/height with aspect presets, and a model_id box that acts as the checkpoint selector — not just a prompt box. Keeping that surface intact in the browser is exactly what gotcha #1 below is about.

The generation-parameters loop closes. postprocess writes the A1111 parameter block into the PNG's parameters text chunk; the PNG Info pipeline parses it back out. Images this app makes round-trip through a real A1111 install too.

---

Architecture

14 references  →  32 operators  →  18 subjects        73 edges
                  ├─ 28 fn      19 pure-local · 9 calling InferenceClient
                  ├─  2 model   HF Inference Providers
                  └─  2 space   Gradio Spaces on the Hub

19 of the 28 fn nodes are pure local Pillow/numpy — all the prompt logic, post-processing, annotators, masking, grid composition and metadata parsing — so most of the app keeps working with no token, no quota and no network. 13 nodes in total leave the machine.

Files

File	What it is
app.py	Entry point — 12 lines of actual wiring
nodes.py	The 21 bound functions (the fn node library)
build_workflow.py	Generates + verifies workflow.json
workflow.json	The committed graph
test_nodes.py	53 offline unit tests (~2s)
test_pipelines.py	Runs all 18 outputs through the real WorkflowExecutor
test_api.py	Drives the 9 generated REST endpoints against a running app
make_samples.py	Regenerates the shipped sample images
deploy_space.py	Stages + uploads the Space
layout.json	The curated node positions
samples/	Sample images used as reference-node defaults

build_workflow.py derives each fn node's input ports from the bound function's own signature via inspect, so port order can never drift from the Python argument order — the executor passes fn arguments positionally, in port order, and that mismatch is the easiest bug to introduce by hand. It then refuses to write the file unless every edge resolves, every type matches, every required input is wired or defaulted, no input has two incoming edges, and no node is orphaned.

---

Six gotchas this app is built around

All six were found by probing gradio 6.22.0 / huggingface_hub 1.26.0 directly, not from the docs. They are the difference between "renders on the canvas" and "actually runs".

1. The canvas rewrites model node ports — silently

This is the big one, and it is invisible until you open the graph in a browser. The canvas normalizes every model node's input ports to the endpoint's canonical schema in _INFERENCE_ENDPOINT_SCHEMAS, saves the result back over workflow.json, and leaves the now-dangling edges in place.

text_to_image's schema is just ["prompt"]. So a txt2img node carrying negative_prompt, num_inference_steps, guidance_scale, seed, width and height — which works perfectly through the headless executor and the REST API — loses all six the instant a browser loads it. No error, no warning; the image just quietly ignores every setting. chat_completion normalizes to ["image", "text"], which had the same effect on the prompt LLM (it started replying "Hello! It seems like your message might be missing something").

The fix is to stop using model nodes wherever the control surface is richer than the schema: txt2img, chat_llm and interrogate are fn nodes that call InferenceClient themselves. fn ports are never rewritten. The two remaining model nodes (image_to_image) have ports exactly equal to their schema, and build_workflow.py now refuses to build if that ever stops being true.

A useful side effect: an fn node can validate. interrogate requires its image, because the model node version cheerfully described an image it was never given — a fabricated result that looked entirely successful.

2. Image ports do not chain uniformly

model nodes emit {"path", "url": "/gradio_api/file=<abs path>", "is_file": true}, and _img_url() prefers the url key — which means nothing to a remote provider:

chain	works?
model image → another model's image_to_image	❌ File not found at \gradio_api\file=...
data: URI → model task endpoint	✅
data: URI → space	❌ call_space only calls handle_file on dicts
{"path": p} (no url) → model or space	✅
model image → chat_completion VLM	✅ (_chat_image_url strips the prefix)
uploaded reference → space or model	✅

So: fn nodes emit {"path", "url"} carrying a real file and a data: URI (see gotcha #5), prep_image sits between any two model nodes, and space nodes only ever take an uploaded image.

3. Several obvious backends simply don't work

Probed live against this account's enabled providers:

• guidance_scale=0 on FLUX → 422, fal-ai requires >= 1. The sampler node clamps it.
• stable-diffusion-3.5-large-turbo → 504 after ~120s.
• Salesforce/blip-image-captioning-large, Qwen2.5-VL-7B, Llama-3.2-*, Mistral-7B-Instruct-v0.3 → no enabled provider.
• depth-estimation is dead: InferenceClient has no depth_estimation method, and the positional fallback targets api-inference.huggingface.co, whose DNS no longer resolves. That's why the annotator's depth mode is an honest luminance approximation rather than a monocular depth model — the edge modes are the genuine article.

Working and fast: FLUX.1-schnell (3s), FLUX.1-dev (3.5s), FLUX.1-Kontext-dev (6s), Qwen3-4B-Instruct (1.2s), Qwen2.5-VL-72B (7s), DETR + ViT (5s).

4. ImageSlider outputs are tuples

Both Spaces return a before/after pair, so the cutout / upscaled result is at output_index: 1, verified by calling them rather than reading their docs.

5. One image value has to satisfy four different consumers

An fn node's image output is consumed four ways, and they disagree:

consumer	wants
REST endpoint (gr.Image component)	a real file path — a data: URI gets treated as a filename and joined to the CWD, raising OSError: [Errno 22]
canvas	a url it can display
chained model node (_img_url)	something a remote provider can fetch
chained fn node (_load_image)	anything, prefers path

So _emit returns both: {"path": <temp file>, "url": <data: URI>}. _from_output takes path first (endpoint happy), the frontend and _img_url take url first (canvas and providers happy).

6. A json port silently destroys its value in the canvas

The canvas serializes a json-typed port with JavaScript's String(obj) instead of JSON.stringify, so the receiving node gets the literal six-word string "[object Object]". Everything downstream then sees no data, with no error anywhere:

• DETR detections reached draw_detections as "[object Object]" → zero boxes → an annotated image identical to the input, and mask_from_detections failing with "No detections matched" at every min_score.
• ViT labels reached top_labels the same way → "No labels above the score threshold".
• png_info's field dict reached its output node as "[object Object]".

Both the executor and the REST API handle json ports perfectly, so this is invisible to test_pipelines.py and test_api.py — only the canvas is affected. The graph therefore contains no json ports at all: structured data travels as JSON text, which survives, and _as_list parses it back. detect_objects and classify_image are fn nodes calling InferenceClient for the same reason a model node could not be used (their output port type is fixed by the endpoint schema — gotcha #1).

Reference-node defaults are a separate case with the opposite answer: the canvas strips path out of a graph default and keeps only url, so the sample images are referenced by their public Hub URL — the one form that renders and that handle_file / InferenceClient can fetch. That is also why the Space must stay public.

---

Tests

python apps/05_workflow1111/test_nodes.py             # 53 unit tests, offline, ~2s
python apps/05_workflow1111/test_pipelines.py         # all 18 outputs, hits HF
python apps/05_workflow1111/test_pipelines.py local   # only the offline outputs
python apps/05_workflow1111/test_api.py               # the 9 REST endpoints
python apps/05_workflow1111/test_api.py --local       # endpoints needing no token

All three layers matter, and they catch different things. test_pipelines.py drives WorkflowExecutor directly, so it skips gradio's output component postprocessing — which is precisely where an image port's value shape is validated. test_api.py is the only layer that catches gotcha #5.

test_pipelines.py writes every result to _test_output/ so you can look at it. Current status: 53/53 unit, 18/18 executor, 9/9 REST, plus a full run in the live canvas covering fn, model, space, uploads and image chaining.

---

Using it as an API

Every subject group is a REST endpoint, so the studio is scriptable:

from gradio_client import Client

client = Client("http://127.0.0.1:7860")
image, params, hires = client.predict(
    "a red fox in a snowy pine forest",   # Prompt
    "",                                    # Negative prompt
    "Cinematic",                           # Style preset
    "enhance fine detail",                 # Hires refine instruction
    api_name="/image",
)

Run python apps/05_workflow1111/test_pipelines.py local to print the full endpoint list with parameter names and types.

---

Extending it

Change a checkpoint by editing the constants at the top of build_workflow.py (T2I_MODEL, EDIT_MODEL, …) and re-running it. Add a node by writing the function in nodes.py, registering it in BIND, and adding an fn(...) call plus link(...)s — the verifier will tell you what you got wrong before the file is written.

Re-running build_workflow.py overwrites node positions, so any layout you drag around in the canvas is reset.