Spaces:
Running
Running
| title: Workflow1111 Diffusion Studio | |
| emoji: 🎨 | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 6.22.0 | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| hf_oauth: true | |
| hf_oauth_scopes: | |
| - inference-api | |
| short_description: Automatic1111-style studio on one gr.Workflow canvas | |
| > **Sign in with Hugging Face** (button at the top right) before running | |
| > anything. This Space carries no token of its own — every `model`, `space` and | |
| > inference-calling `fn` node runs on *your* token and your own inference quota. | |
| # 05 · Workflow1111 — a Diffusion Studio built from `gr.Workflow` | |
| An **Automatic1111-shaped image studio expressed as one canvas graph** instead | |
| of a tabbed UI: **64 nodes, 10 pipelines, 18 independently runnable outputs**, | |
| all inside a single `gr.Workflow`. | |
| ```bash | |
| pip install -r apps/05_workflow1111/requirements.txt | |
| hf auth login # or: set HF_TOKEN=hf_xxx | |
| python apps/05_workflow1111/app.py | |
| ``` | |
| --- | |
| ## Why it isn't tabs | |
| `gr.Workflow` raises if you construct it inside a `gr.Blocks` context: | |
| ```python | |
| if Context.root_block is not None: | |
| raise ValueError("gr.Workflow cannot be created inside another gr.Blocks context.") | |
| ``` | |
| So an A1111 clone genuinely *cannot* be a `gr.Tabs` layout here — the graph **is** | |
| the UI. The analogue of a tab is a **subject group**: a connected cluster of | |
| outputs, which gradio also publishes as its own REST endpoint. Ten pipelines sit | |
| side by side on one canvas, and you run whichever output you want. | |
| --- | |
| ## What's on the canvas | |
| | # | Pipeline | A1111 equivalent | Nodes | | |
| |---|----------|------------------|-------| | |
| | 1 | **txt2img** | txt2img tab | prompt builder → negative builder → sampler → FLUX.1-schnell → post-processing → image + params | | |
| | 2 | **Hires fix** | Hires. fix | txt2img result → prep → FLUX.1-Kontext re-render | | |
| | 3 | **img2img** | img2img tab | upload → prep → FLUX.1-Kontext edit → post-processing | | |
| | 4 | **Prompt magic** | — | idea → instruction → Qwen3-4B → cleanup | | |
| | 5 | **Interrogate** | CLIP interrogate | image → Qwen2.5-VL → prompt; + ViT classification | | |
| | 6 | **Detect & mask** | inpaint masking | DETR → annotated boxes → feathered inpaint mask | | |
| | 7 | **Prompt matrix** | X/Y/Z plot | 4 variants → 4 parallel renders → contact sheet | | |
| | 8 | **Extras** | Extras tab | local Lanczos upscale · AuraSR ×4 · background removal | | |
| | 9 | **Annotators** | ControlNet preprocessors | Canny · line art · sketch · luma-depth · posterize · threshold | | |
| | 10 | **PNG Info** | PNG Info tab | read generation parameters back out of a file | | |
| **The txt2img node has the real control surface** — negative prompt, sampling | |
| steps, CFG scale, seed (with `-1` = random), width/height with aspect presets, | |
| and a `model_id` box that acts as the checkpoint selector — not just a prompt | |
| box. Keeping that surface intact in the browser is exactly what gotcha #1 below | |
| is about. | |
| **The generation-parameters loop closes.** `postprocess` writes the A1111 | |
| parameter block into the PNG's `parameters` text chunk; the PNG Info pipeline | |
| parses it back out. Images this app makes round-trip through a real A1111 | |
| install too. | |
| --- | |
| ## Architecture | |
| ``` | |
| 14 references → 32 operators → 18 subjects 73 edges | |
| ├─ 28 fn 19 pure-local · 9 calling InferenceClient | |
| ├─ 2 model HF Inference Providers | |
| └─ 2 space Gradio Spaces on the Hub | |
| ``` | |
| 19 of the 28 `fn` nodes are pure local Pillow/numpy — all the prompt logic, | |
| post-processing, annotators, masking, grid composition and metadata parsing — | |
| so most of the app keeps working with no token, no quota and no network. 13 | |
| nodes in total leave the machine. | |
| ### Files | |
| | File | What it is | | |
| |---|---| | |
| | `app.py` | Entry point — 12 lines of actual wiring | | |
| | `nodes.py` | The 21 bound functions (the `fn` node library) | | |
| | `build_workflow.py` | **Generates + verifies** `workflow.json` | | |
| | `workflow.json` | The committed graph | | |
| | `test_nodes.py` | 53 offline unit tests (~2s) | | |
| | `test_pipelines.py` | Runs all 18 outputs through the real `WorkflowExecutor` | | |
| | `test_api.py` | Drives the 9 generated REST endpoints against a running app | | |
| | `make_samples.py` | Regenerates the shipped sample images | | |
| | `deploy_space.py` | Stages + uploads the Space | | |
| | `layout.json` | The curated node positions | | |
| | `samples/` | Sample images used as reference-node defaults | | |
| `build_workflow.py` derives each `fn` node's input ports from the bound | |
| function's own signature via `inspect`, so **port order can never drift from | |
| the Python argument order** — the executor passes `fn` arguments positionally, | |
| in port order, and that mismatch is the easiest bug to introduce by hand. It | |
| then refuses to write the file unless every edge resolves, every type matches, | |
| every required input is wired or defaulted, no input has two incoming edges, | |
| and no node is orphaned. | |
| --- | |
| ## Six gotchas this app is built around | |
| All six were found by probing gradio 6.22.0 / huggingface_hub 1.26.0 | |
| directly, not from the docs. They are the difference between "renders on the | |
| canvas" and "actually runs". | |
| ### 1. The canvas rewrites `model` node ports — silently | |
| This is the big one, and it is invisible until you open the graph in a browser. | |
| The canvas normalizes every `model` node's input ports to the endpoint's | |
| canonical schema in `_INFERENCE_ENDPOINT_SCHEMAS`, **saves the result back over | |
| `workflow.json`**, and leaves the now-dangling edges in place. | |
| `text_to_image`'s schema is just `["prompt"]`. So a txt2img node carrying | |
| `negative_prompt`, `num_inference_steps`, `guidance_scale`, `seed`, `width` and | |
| `height` — which works perfectly through the headless executor and the REST API | |
| — loses all six the instant a browser loads it. No error, no warning; the image | |
| just quietly ignores every setting. `chat_completion` normalizes to | |
| `["image", "text"]`, which had the same effect on the prompt LLM (it started | |
| replying "Hello! It seems like your message might be missing something"). | |
| The fix is to stop using `model` nodes wherever the control surface is richer | |
| than the schema: `txt2img`, `chat_llm` and `interrogate` are `fn` nodes that | |
| call `InferenceClient` themselves. `fn` ports are never rewritten. The two | |
| remaining `model` nodes (`image_to_image`) have ports exactly equal to their | |
| schema, and | |
| `build_workflow.py` now **refuses to build** if that ever stops being true. | |
| A useful side effect: an `fn` node can validate. `interrogate` requires its | |
| image, because the `model` node version cheerfully described an image it was | |
| never given — a fabricated result that looked entirely successful. | |
| ### 2. Image ports do not chain uniformly | |
| `model` nodes emit `{"path", "url": "/gradio_api/file=<abs path>", "is_file": true}`, | |
| and `_img_url()` prefers the `url` key — which means nothing to a remote | |
| provider: | |
| | chain | works? | | |
| |---|---| | |
| | model image → another model's `image_to_image` | ❌ `File not found at \gradio_api\file=...` | | |
| | `data:` URI → model task endpoint | ✅ | | |
| | `data:` URI → **space** | ❌ `call_space` only calls `handle_file` on dicts | | |
| | `{"path": p}` (no `url`) → model or space | ✅ | | |
| | model image → `chat_completion` VLM | ✅ (`_chat_image_url` strips the prefix) | | |
| | uploaded reference → space or model | ✅ | | |
| So: **`fn` nodes emit `{"path", "url"}` carrying a real file *and* a `data:` | |
| URI** (see gotcha #5), **`prep_image` sits between any two model nodes**, and | |
| **`space` nodes only ever take an uploaded image**. | |
| ### 3. Several obvious backends simply don't work | |
| Probed live against this account's enabled providers: | |
| - `guidance_scale=0` on FLUX → **422**, fal-ai requires `>= 1`. The sampler node clamps it. | |
| - `stable-diffusion-3.5-large-turbo` → 504 after ~120s. | |
| - `Salesforce/blip-image-captioning-large`, `Qwen2.5-VL-7B`, `Llama-3.2-*`, | |
| `Mistral-7B-Instruct-v0.3` → no enabled provider. | |
| - **depth-estimation is dead**: `InferenceClient` has no `depth_estimation` | |
| method, and the positional fallback targets `api-inference.huggingface.co`, | |
| whose DNS no longer resolves. That's why the annotator's depth mode is an | |
| honest luminance approximation rather than a monocular depth model — the edge | |
| modes are the genuine article. | |
| Working and fast: FLUX.1-schnell (~3s), FLUX.1-dev (~3.5s), FLUX.1-Kontext-dev | |
| (~6s), Qwen3-4B-Instruct (~1.2s), Qwen2.5-VL-72B (~7s), DETR + ViT (~5s). | |
| ### 4. `ImageSlider` outputs are tuples | |
| Both Spaces return a before/after pair, so the cutout / upscaled result is at | |
| `output_index: 1`, verified by calling them rather than reading their docs. | |
| ### 5. One image value has to satisfy four different consumers | |
| An `fn` node's image output is consumed four ways, and they disagree: | |
| | consumer | wants | | |
| |---|---| | |
| | REST endpoint (`gr.Image` component) | a real file path — a `data:` URI gets treated as a *filename* and joined to the CWD, raising `OSError: [Errno 22]` | | |
| | canvas | a `url` it can display | | |
| | chained `model` node (`_img_url`) | something a remote provider can fetch | | |
| | chained `fn` node (`_load_image`) | anything, prefers `path` | | |
| So `_emit` returns **both**: `{"path": <temp file>, "url": <data: URI>}`. | |
| `_from_output` takes `path` first (endpoint happy), the frontend and `_img_url` | |
| take `url` first (canvas and providers happy). | |
| ### 6. A `json` port silently destroys its value in the canvas | |
| The canvas serializes a `json`-typed port with JavaScript's `String(obj)` | |
| instead of `JSON.stringify`, so the receiving node gets the literal six-word | |
| string `"[object Object]"`. Everything downstream then sees *no data*, with no | |
| error anywhere: | |
| - DETR detections reached `draw_detections` as `"[object Object]"` → zero boxes | |
| → an annotated image identical to the input, and | |
| `mask_from_detections` failing with "No detections matched" at **every** | |
| `min_score`. | |
| - ViT labels reached `top_labels` the same way → "No labels above the score | |
| threshold". | |
| - `png_info`'s field dict reached its output node as `"[object Object]"`. | |
| Both the executor and the REST API handle `json` ports perfectly, so this is | |
| invisible to `test_pipelines.py` *and* `test_api.py` — only the canvas is | |
| affected. The graph therefore contains **no `json` ports at all**: structured | |
| data travels as JSON *text*, which survives, and `_as_list` parses it back. | |
| `detect_objects` and `classify_image` are `fn` nodes calling `InferenceClient` | |
| for the same reason a `model` node could not be used (their output port type is | |
| fixed by the endpoint schema — gotcha #1). | |
| Reference-node *defaults* are a separate case with the opposite answer: the | |
| canvas strips `path` out of a graph default and keeps only `url`, so the sample | |
| images are referenced by their **public Hub URL** — the one form that renders | |
| *and* that `handle_file` / `InferenceClient` can fetch. That is also why the | |
| Space must stay public. | |
| --- | |
| ## Tests | |
| ```bash | |
| python apps/05_workflow1111/test_nodes.py # 53 unit tests, offline, ~2s | |
| python apps/05_workflow1111/test_pipelines.py # all 18 outputs, hits HF | |
| python apps/05_workflow1111/test_pipelines.py local # only the offline outputs | |
| python apps/05_workflow1111/test_api.py # the 9 REST endpoints | |
| python apps/05_workflow1111/test_api.py --local # endpoints needing no token | |
| ``` | |
| All three layers matter, and they catch different things. `test_pipelines.py` | |
| drives `WorkflowExecutor` directly, so it skips gradio's **output component** | |
| postprocessing — which is precisely where an image port's value shape is | |
| validated. `test_api.py` is the only layer that catches gotcha #5. | |
| `test_pipelines.py` writes every result to `_test_output/` so you can look at | |
| it. Current status: **53/53 unit, 18/18 executor, 9/9 REST**, plus a full run in | |
| the live canvas covering `fn`, `model`, `space`, uploads and image chaining. | |
| --- | |
| ## Using it as an API | |
| Every subject group is a REST endpoint, so the studio is scriptable: | |
| ```python | |
| from gradio_client import Client | |
| client = Client("http://127.0.0.1:7860") | |
| image, params, hires = client.predict( | |
| "a red fox in a snowy pine forest", # Prompt | |
| "", # Negative prompt | |
| "Cinematic", # Style preset | |
| "enhance fine detail", # Hires refine instruction | |
| api_name="/image", | |
| ) | |
| ``` | |
| Run `python apps/05_workflow1111/test_pipelines.py local` to print the full | |
| endpoint list with parameter names and types. | |
| --- | |
| ## Extending it | |
| Change a checkpoint by editing the constants at the top of `build_workflow.py` | |
| (`T2I_MODEL`, `EDIT_MODEL`, …) and re-running it. Add a node by writing the | |
| function in `nodes.py`, registering it in `BIND`, and adding an `fn(...)` call | |
| plus `link(...)`s — the verifier will tell you what you got wrong before the | |
| file is written. | |
| > Re-running `build_workflow.py` overwrites node positions, so any layout you | |
| > drag around in the canvas is reset. | |