--- title: Workflow1111 Diffusion Studio emoji: ๐ŸŽจ colorFrom: indigo colorTo: purple sdk: gradio sdk_version: 6.22.0 app_file: app.py pinned: false license: mit hf_oauth: true hf_oauth_scopes: - inference-api short_description: Automatic1111-style studio on one gr.Workflow canvas --- > **Sign in with Hugging Face** (button at the top right) before running > anything. This Space carries no token of its own โ€” every `model`, `space` and > inference-calling `fn` node runs on *your* token and your own inference quota. # 05 ยท Workflow1111 โ€” a Diffusion Studio built from `gr.Workflow` An **Automatic1111-shaped image studio expressed as one canvas graph** instead of a tabbed UI: **64 nodes, 10 pipelines, 18 independently runnable outputs**, all inside a single `gr.Workflow`. ```bash pip install -r apps/05_workflow1111/requirements.txt hf auth login # or: set HF_TOKEN=hf_xxx python apps/05_workflow1111/app.py ``` --- ## Why it isn't tabs `gr.Workflow` raises if you construct it inside a `gr.Blocks` context: ```python if Context.root_block is not None: raise ValueError("gr.Workflow cannot be created inside another gr.Blocks context.") ``` So an A1111 clone genuinely *cannot* be a `gr.Tabs` layout here โ€” the graph **is** the UI. The analogue of a tab is a **subject group**: a connected cluster of outputs, which gradio also publishes as its own REST endpoint. Ten pipelines sit side by side on one canvas, and you run whichever output you want. --- ## What's on the canvas | # | Pipeline | A1111 equivalent | Nodes | |---|----------|------------------|-------| | 1 | **txt2img** | txt2img tab | prompt builder โ†’ negative builder โ†’ sampler โ†’ FLUX.1-schnell โ†’ post-processing โ†’ image + params | | 2 | **Hires fix** | Hires. fix | txt2img result โ†’ prep โ†’ FLUX.1-Kontext re-render | | 3 | **img2img** | img2img tab | upload โ†’ prep โ†’ FLUX.1-Kontext edit โ†’ post-processing | | 4 | **Prompt magic** | โ€” | idea โ†’ instruction โ†’ Qwen3-4B โ†’ cleanup | | 5 | **Interrogate** | CLIP interrogate | image โ†’ Qwen2.5-VL โ†’ prompt; + ViT classification | | 6 | **Detect & mask** | inpaint masking | DETR โ†’ annotated boxes โ†’ feathered inpaint mask | | 7 | **Prompt matrix** | X/Y/Z plot | 4 variants โ†’ 4 parallel renders โ†’ contact sheet | | 8 | **Extras** | Extras tab | local Lanczos upscale ยท AuraSR ร—4 ยท background removal | | 9 | **Annotators** | ControlNet preprocessors | Canny ยท line art ยท sketch ยท luma-depth ยท posterize ยท threshold | | 10 | **PNG Info** | PNG Info tab | read generation parameters back out of a file | **The txt2img node has the real control surface** โ€” negative prompt, sampling steps, CFG scale, seed (with `-1` = random), width/height with aspect presets, and a `model_id` box that acts as the checkpoint selector โ€” not just a prompt box. Keeping that surface intact in the browser is exactly what gotcha #1 below is about. **The generation-parameters loop closes.** `postprocess` writes the A1111 parameter block into the PNG's `parameters` text chunk; the PNG Info pipeline parses it back out. Images this app makes round-trip through a real A1111 install too. --- ## Architecture ``` 14 references โ†’ 32 operators โ†’ 18 subjects 73 edges โ”œโ”€ 28 fn 19 pure-local ยท 9 calling InferenceClient โ”œโ”€ 2 model HF Inference Providers โ””โ”€ 2 space Gradio Spaces on the Hub ``` 19 of the 28 `fn` nodes are pure local Pillow/numpy โ€” all the prompt logic, post-processing, annotators, masking, grid composition and metadata parsing โ€” so most of the app keeps working with no token, no quota and no network. 13 nodes in total leave the machine. ### Files | File | What it is | |---|---| | `app.py` | Entry point โ€” 12 lines of actual wiring | | `nodes.py` | The 21 bound functions (the `fn` node library) | | `build_workflow.py` | **Generates + verifies** `workflow.json` | | `workflow.json` | The committed graph | | `test_nodes.py` | 53 offline unit tests (~2s) | | `test_pipelines.py` | Runs all 18 outputs through the real `WorkflowExecutor` | | `test_api.py` | Drives the 9 generated REST endpoints against a running app | | `make_samples.py` | Regenerates the shipped sample images | | `deploy_space.py` | Stages + uploads the Space | | `layout.json` | The curated node positions | | `samples/` | Sample images used as reference-node defaults | `build_workflow.py` derives each `fn` node's input ports from the bound function's own signature via `inspect`, so **port order can never drift from the Python argument order** โ€” the executor passes `fn` arguments positionally, in port order, and that mismatch is the easiest bug to introduce by hand. It then refuses to write the file unless every edge resolves, every type matches, every required input is wired or defaulted, no input has two incoming edges, and no node is orphaned. --- ## Six gotchas this app is built around All six were found by probing gradio 6.22.0 / huggingface_hub 1.26.0 directly, not from the docs. They are the difference between "renders on the canvas" and "actually runs". ### 1. The canvas rewrites `model` node ports โ€” silently This is the big one, and it is invisible until you open the graph in a browser. The canvas normalizes every `model` node's input ports to the endpoint's canonical schema in `_INFERENCE_ENDPOINT_SCHEMAS`, **saves the result back over `workflow.json`**, and leaves the now-dangling edges in place. `text_to_image`'s schema is just `["prompt"]`. So a txt2img node carrying `negative_prompt`, `num_inference_steps`, `guidance_scale`, `seed`, `width` and `height` โ€” which works perfectly through the headless executor and the REST API โ€” loses all six the instant a browser loads it. No error, no warning; the image just quietly ignores every setting. `chat_completion` normalizes to `["image", "text"]`, which had the same effect on the prompt LLM (it started replying "Hello! It seems like your message might be missing something"). The fix is to stop using `model` nodes wherever the control surface is richer than the schema: `txt2img`, `chat_llm` and `interrogate` are `fn` nodes that call `InferenceClient` themselves. `fn` ports are never rewritten. The two remaining `model` nodes (`image_to_image`) have ports exactly equal to their schema, and `build_workflow.py` now **refuses to build** if that ever stops being true. A useful side effect: an `fn` node can validate. `interrogate` requires its image, because the `model` node version cheerfully described an image it was never given โ€” a fabricated result that looked entirely successful. ### 2. Image ports do not chain uniformly `model` nodes emit `{"path", "url": "/gradio_api/file=", "is_file": true}`, and `_img_url()` prefers the `url` key โ€” which means nothing to a remote provider: | chain | works? | |---|---| | model image โ†’ another model's `image_to_image` | โŒ `File not found at \gradio_api\file=...` | | `data:` URI โ†’ model task endpoint | โœ… | | `data:` URI โ†’ **space** | โŒ `call_space` only calls `handle_file` on dicts | | `{"path": p}` (no `url`) โ†’ model or space | โœ… | | model image โ†’ `chat_completion` VLM | โœ… (`_chat_image_url` strips the prefix) | | uploaded reference โ†’ space or model | โœ… | So: **`fn` nodes emit `{"path", "url"}` carrying a real file *and* a `data:` URI** (see gotcha #5), **`prep_image` sits between any two model nodes**, and **`space` nodes only ever take an uploaded image**. ### 3. Several obvious backends simply don't work Probed live against this account's enabled providers: - `guidance_scale=0` on FLUX โ†’ **422**, fal-ai requires `>= 1`. The sampler node clamps it. - `stable-diffusion-3.5-large-turbo` โ†’ 504 after ~120s. - `Salesforce/blip-image-captioning-large`, `Qwen2.5-VL-7B`, `Llama-3.2-*`, `Mistral-7B-Instruct-v0.3` โ†’ no enabled provider. - **depth-estimation is dead**: `InferenceClient` has no `depth_estimation` method, and the positional fallback targets `api-inference.huggingface.co`, whose DNS no longer resolves. That's why the annotator's depth mode is an honest luminance approximation rather than a monocular depth model โ€” the edge modes are the genuine article. Working and fast: FLUX.1-schnell (~3s), FLUX.1-dev (~3.5s), FLUX.1-Kontext-dev (~6s), Qwen3-4B-Instruct (~1.2s), Qwen2.5-VL-72B (~7s), DETR + ViT (~5s). ### 4. `ImageSlider` outputs are tuples Both Spaces return a before/after pair, so the cutout / upscaled result is at `output_index: 1`, verified by calling them rather than reading their docs. ### 5. One image value has to satisfy four different consumers An `fn` node's image output is consumed four ways, and they disagree: | consumer | wants | |---|---| | REST endpoint (`gr.Image` component) | a real file path โ€” a `data:` URI gets treated as a *filename* and joined to the CWD, raising `OSError: [Errno 22]` | | canvas | a `url` it can display | | chained `model` node (`_img_url`) | something a remote provider can fetch | | chained `fn` node (`_load_image`) | anything, prefers `path` | So `_emit` returns **both**: `{"path": , "url": }`. `_from_output` takes `path` first (endpoint happy), the frontend and `_img_url` take `url` first (canvas and providers happy). ### 6. A `json` port silently destroys its value in the canvas The canvas serializes a `json`-typed port with JavaScript's `String(obj)` instead of `JSON.stringify`, so the receiving node gets the literal six-word string `"[object Object]"`. Everything downstream then sees *no data*, with no error anywhere: - DETR detections reached `draw_detections` as `"[object Object]"` โ†’ zero boxes โ†’ an annotated image identical to the input, and `mask_from_detections` failing with "No detections matched" at **every** `min_score`. - ViT labels reached `top_labels` the same way โ†’ "No labels above the score threshold". - `png_info`'s field dict reached its output node as `"[object Object]"`. Both the executor and the REST API handle `json` ports perfectly, so this is invisible to `test_pipelines.py` *and* `test_api.py` โ€” only the canvas is affected. The graph therefore contains **no `json` ports at all**: structured data travels as JSON *text*, which survives, and `_as_list` parses it back. `detect_objects` and `classify_image` are `fn` nodes calling `InferenceClient` for the same reason a `model` node could not be used (their output port type is fixed by the endpoint schema โ€” gotcha #1). Reference-node *defaults* are a separate case with the opposite answer: the canvas strips `path` out of a graph default and keeps only `url`, so the sample images are referenced by their **public Hub URL** โ€” the one form that renders *and* that `handle_file` / `InferenceClient` can fetch. That is also why the Space must stay public. --- ## Tests ```bash python apps/05_workflow1111/test_nodes.py # 53 unit tests, offline, ~2s python apps/05_workflow1111/test_pipelines.py # all 18 outputs, hits HF python apps/05_workflow1111/test_pipelines.py local # only the offline outputs python apps/05_workflow1111/test_api.py # the 9 REST endpoints python apps/05_workflow1111/test_api.py --local # endpoints needing no token ``` All three layers matter, and they catch different things. `test_pipelines.py` drives `WorkflowExecutor` directly, so it skips gradio's **output component** postprocessing โ€” which is precisely where an image port's value shape is validated. `test_api.py` is the only layer that catches gotcha #5. `test_pipelines.py` writes every result to `_test_output/` so you can look at it. Current status: **53/53 unit, 18/18 executor, 9/9 REST**, plus a full run in the live canvas covering `fn`, `model`, `space`, uploads and image chaining. --- ## Using it as an API Every subject group is a REST endpoint, so the studio is scriptable: ```python from gradio_client import Client client = Client("http://127.0.0.1:7860") image, params, hires = client.predict( "a red fox in a snowy pine forest", # Prompt "", # Negative prompt "Cinematic", # Style preset "enhance fine detail", # Hires refine instruction api_name="/image", ) ``` Run `python apps/05_workflow1111/test_pipelines.py local` to print the full endpoint list with parameter names and types. --- ## Extending it Change a checkpoint by editing the constants at the top of `build_workflow.py` (`T2I_MODEL`, `EDIT_MODEL`, โ€ฆ) and re-running it. Add a node by writing the function in `nodes.py`, registering it in `BIND`, and adding an `fn(...)` call plus `link(...)`s โ€” the verifier will tell you what you got wrong before the file is written. > Re-running `build_workflow.py` overwrites node positions, so any layout you > drag around in the canvas is reset.