# FLUX Redux + ControlNet Style-Composition — Final Report & Replication Guide **Date:** August 2026 · **Pod:** RunPod H100 80GB, ComfyUI 0.27.0 (`plx1029/comfyui-qwen` image) **Goal:** 1 style-reference image + 1 composition-reference image (+ optional text prompt) → style-transferred, composition-locked output on FLUX.1-dev. Maximum quality. Everything here also lives on HF `aleph65/ComfyUI` (workflows in `workflows/`, models in `models/`, this archive in `redux_experiment_final/`). --- ## TL;DR — the final production stack ``` FLUX.1-dev bf16 (UNETLoader, weight_dtype=default) ├─ Style img → sigclip_vision_384 → CLIPVisionEncode(center) → StyleModelApply │ (flux1-redux-dev, strength 0.35, strength_type=MULTIPLY — never attn_bias, see §Findings) ├─ Prompt → handwritten per-run: target's content + style ref's aesthetic → FluxGuidance 3.0 ├─ Comp img → ImageResize+ 896x1152 fill/crop → DepthAnythingV2(vitl) → Union-Pro-2.0 │ via ControlNetApplySD3 (strength 0.7, start 0, end 0.8) — DEPTH ONLY, single CN └─ KSampler euler/simple/32 steps/CFG 1.0/denoise 1.0, seed 777 for all test runs └─ optional HIRES: LatentUpscaleBy bislerp 1.5x → 2nd pass denoise 0.30 (Redux kept, CN released) ``` A/B variant (`bfl-lora`): same graph but structure via `flux1-depth-dev-lora` **0.85** + `InstructPixToPixConditioning`, **FluxGuidance 10.0** (BFL spec — the LoRA is distilled at guidance 10). Competitive with the CN path since the v4 fixes; less tunable (no end_percent, no stacking). ## The 7 workflow files (in `workflows/`) | file | purpose | |---|---| | `flux-v2-redux-style-composition.json` | **The main one.** Style+composition, all fixes baked. Bypassed groups: canny stack, ReduxAdvanced alt, turbo-LoRA fast preview, HIRES | | `flux-v2-redux-style-composition-bfl-lora.json` | BFL depth-LoRA A/B variant, spec settings | | `flux-redux-fal-dev.json` | fal `flux/dev/redux` endpoint replica (768×1024, empty prompt, strength 1.0 multiply, euler/simple/28, guidance 3.5). Doubles as simple 1-image workflow | | `flux-redux-prompt.json` | 1 image + prompt with authority (attn_bias 0.5 — fine here, NO ControlNet in this graph) | | `flux-redux-schnell.json` | Replicate `flux-redux-schnell` replica (4 steps, no guidance, no prompt; 1024² = their 1:1 default, set 896×1152 for portrait) | | `flux-redux-style-composition.json` (v1) | Historical — superseded by flux-v2 | | `flux-redux-style-composition-bfl-lora.json` (v1) | Historical — superseded | `workflows/wip/` = every draft revision; `workflows/original-old-workflows/` = the pre-project XLabs/Union-v1 workflows this started from (retired: XLabs is legacy, Union v1 needs `SetUnionControlNetType` which v2.0 must NOT have). ## Models (all mirrored on HF `aleph65/ComfyUI` under `models/`) | file | dir | source | gated | |---|---|---|---| | `flux1-dev.safetensors` (bf16 23.8GB) | diffusion_models | black-forest-labs/FLUX.1-dev | yes | | `flux1-schnell.safetensors` (bf16 23.8GB) | diffusion_models | black-forest-labs/FLUX.1-schnell | no | | `flux1-redux-dev.safetensors` (129MB) | style_models | black-forest-labs/FLUX.1-Redux-dev | yes | | `FLUX.1-dev-ControlNet-Union-Pro-2.0.safetensors` (4.28GB) | controlnet | Shakker-Labs (renamed from `diffusion_pytorch_model.safetensors`) | no | | `flux1-depth-dev-lora.safetensors` (1.24GB) | loras | black-forest-labs/FLUX.1-Depth-dev-lora | yes | | `t5xxl_fp16.safetensors` + `clip_l.safetensors` | text_encoders | Comfy-Org repack | no | | `ae.safetensors` | vae | FLUX.1-dev repo | (yes) | | `sigclip_vision_patch14_384.safetensors` | clip_vision | Comfy-Org/sigclip_vision_384 | no | | `flux1-turbo-alpha.safetensors` | loras | alimama-creative (fast-preview group) | no | | `depth_anything_v2_vitl.pth` | auto-downloads into `custom_nodes/comfyui_controlnet_aux/ckpts/` | | no | **Custom node packs (3):** `comfyui_controlnet_aux` (Fannovel16 — DepthAnythingV2), `ComfyUI_essentials` (cubiq — ImageResize+), `ComfyUI_AdvancedRefluxControl` (kaibioinfo — ReduxAdvanced alt branch). Everything else is ComfyUI core. ## Replication on a fresh pod 1. Pod from `plx1029/comfyui-qwen` image (or any ComfyUI ≥0.3.8 — that's when native StyleModelApply strength landed). 2. `HUGGING_FACE_ACCESS_TOKEN` env var; accept licenses for FLUX.1-dev, FLUX.1-Redux-dev, FLUX.1-Depth-dev-lora on that account. 3. `./download_missing_models.sh flux-v2-redux-style-composition.json ...` — resolves the node packs from each node's `cnr_id`/`aux_id` and pulls all models from the mirror. (Or run `scripts/download_models.py` which pulls from original sources.) 4. Restart ComfyUI, load the workflow, drop images, go. Defaults are the calibrated ones. 5. Batch runs: `scripts/run_matrix_v4.py` queues via the API (`POST /prompt` with API-format graphs, poll `/history/`). `scripts/prompts_v4.py` holds the handwritten prompt fragments. ## Findings (chronological, what actually mattered) 1. **Baseline worked but bodies deformed** (v1–v3): aesthetics great, frequent limb/torso deformities in both two-image paths. Single-image workflows always clean. 2. **ROOT CAUSE (code-verified, undocumented upstream): `attn_bias` + ControlNet is broken.** `StyleModelApply(attn_bias)` appends all 729 Redux tokens at FULL strength and stores the attenuation as an attention-bias mask. The base flux transformer honors the mask; the ControlNet branch never receives it (`ControlNetFlux.forward_orig` has no attn_mask param; not in its `extra_conds`). Base attends Redux @50%, CN attends @100% → two conditioning streams steer different images every step → ghosting/deformities. Also: any attention mask kicks flux off the flash-attention fast path. **Fix: `strength_type=multiply` at 0.25–0.35 whenever a ControlNet is present.** attn_bias remains fine in CN-free graphs. (8-run isolation diagnostic + two deep-research passes; see `comparison_sheets/_diag_sheet.jpg` — column d is the smoking gun.) 3. **BFL depth-LoRA wants its spec: guidance 10.0 + LoRA 0.85.** It is distilled at guidance 10; running 3–4 gives mush. Earlier sweeps that said "guidance 4 looks better" were contaminated by the attn_bias bug. The CN path conversely wants guidance ~3.0. **The two paths need different FluxGuidance values.** 4. **Prompts are the anatomy lever.** Flux resolves limb ambiguity from text. Empty prompt = worst; generic anatomy boilerplate = marginal; **handwritten per-run prompts describing the composition target's actual content fused with the style ref's actual aesthetic = best**. Depth maps are semantically ambiguous at limb crossings; the prompt disambiguates. Negative prompts are INERT at CFG 1.0 (comfy skips uncond entirely) — anatomy must be handled positively. 5. **Calibration verdicts** (30-run, `comparison_sheets/_C*.jpg`): multiply 0.35 > 0.25/0.30 (style strength, all clean) > ReduxAdvanced ds3 (clean but weakens style); 896×1152 fine despite Union-Pro-2.0's 512 training res (no penalty once the conditioning fight was fixed); CN end 0.4 loses composition, 0.6–0.8 both hold → keep 0.8, use 0.6 for freedom; depth-only single CN (canny stack changes scenes — keep off unless contour lock wanted). 6. **Input preprocessing matters**: fill/crop everything to 896×1152 in-graph (portrait 3:4 bucket), depth map == latent to the pixel, zero padding (padding becomes literal image content in the latent-concat/bfl path). Save crops + depth maps for auditing. 7. **Input spec** (what to feed it): 3:4 portrait, 1152×1536, sRGB 8-bit, no watermarks/text/ borders, EXIF baked. Style refs: style-defining content in the vertical middle — sigclip center-crops to a square, top/bottom ~12% discarded. Composition refs: clear fg/bg depth separation. 8. **Hires two-pass** (1.5× latent bislerp + denoise 0.30, Redux carried, CN released): validated win — real detail gain, aesthetic preserved. Ships bypassed. 9. **Known edge case, unsolved**: person-style × empty-room-target (our r×r4 combos) is a contradictory ask; results inconsistent even with interior prompts. Needs its own treatment. 10. **Redux mode compatibility notes**: scaled-fp8 flux checkpoints black-screen with Redux (ComfyUI #5849). XLabs CN stack is legacy/dormant. Union-Pro-**2.0** must NOT have `SetUnionControlNetType` (that's v1-only). ## Prompt-writing recipe (for new image pairs) Write one sentence of **content** from the composition target (subject, pose, clothing, setting — what the depth map encodes), one sentence of **style** from the style ref (lighting type, palette, finish), then `"Sharp focus, natural proportions, coherent anatomy."` If the target has no person, say so explicitly and describe the space instead. Fragments for the 10 experiment inputs are in `scripts/prompts_v4.py`. ## What's in this archive ``` README.md ← this file workflows/ ← all 7 finals + wip/ history + original-old-workflows/ inputs/ ← inputs_v1 (first test set) + inputs_v2 (curated 1152×1536 3:4 set) scripts/ ← graph builders, all matrix/sweep/calibration runners, prompts, model downloader research/ ← FLUX-REDUX-CONTROLNET-RESEARCH.md (full research log w/ sources) outputs_v4_final/ ← the final 84-run matrix: 36 style-comp (+hires/ +inputs/ audits), 36 bfl-lora, 12 singles, calib/ (30 calibration runs) comparison_sheets/ ← every verdict sheet: _diag_sheet (the attn_bias smoking gun), _A* (v2 sweeps), _C* (v4 calibration), _v2_vs_v3, _v3_vs_v4 claude_memory/ ← Claude Code memory files from this project ``` Run provenance: every test image used **seed 777**; matrices covered all 24 r×t + 12 ordered r×r combos per two-image workflow. v1→v4 output sets remain in `/workspace/outputs*` on the pod (not archived here except v4). — Built with Claude Code (Fable 5), Aug 2026. It was fun, sir. 🫡