redux_experiment_final/README.md

FLUX Redux + ControlNet Style-Composition — Final Report & Replication Guide

Date: August 2026 · Pod: RunPod H100 80GB, ComfyUI 0.27.0 (plx1029/comfyui-qwen image) Goal: 1 style-reference image + 1 composition-reference image (+ optional text prompt) → style-transferred, composition-locked output on FLUX.1-dev. Maximum quality.

Everything here also lives on HF aleph65/ComfyUI (workflows in workflows/, models in models/, this archive in redux_experiment_final/).

---

TL;DR — the final production stack

FLUX.1-dev bf16 (UNETLoader, weight_dtype=default)
├─ Style img  → sigclip_vision_384 → CLIPVisionEncode(center) → StyleModelApply
│               (flux1-redux-dev, strength 0.35, strength_type=MULTIPLY — never attn_bias, see §Findings)
├─ Prompt     → handwritten per-run: target's content + style ref's aesthetic → FluxGuidance 3.0
├─ Comp img   → ImageResize+ 896x1152 fill/crop → DepthAnythingV2(vitl) → Union-Pro-2.0
│               via ControlNetApplySD3 (strength 0.7, start 0, end 0.8) — DEPTH ONLY, single CN
└─ KSampler euler/simple/32 steps/CFG 1.0/denoise 1.0, seed 777 for all test runs
   └─ optional HIRES: LatentUpscaleBy bislerp 1.5x → 2nd pass denoise 0.30 (Redux kept, CN released)

A/B variant (bfl-lora): same graph but structure via flux1-depth-dev-lora 0.85 + InstructPixToPixConditioning, FluxGuidance 10.0 (BFL spec — the LoRA is distilled at guidance 10). Competitive with the CN path since the v4 fixes; less tunable (no end_percent, no stacking).

The 7 workflow files (in workflows/)

file	purpose
flux-v2-redux-style-composition.json	The main one. Style+composition, all fixes baked. Bypassed groups: canny stack, ReduxAdvanced alt, turbo-LoRA fast preview, HIRES
flux-v2-redux-style-composition-bfl-lora.json	BFL depth-LoRA A/B variant, spec settings
flux-redux-fal-dev.json	fal flux/dev/redux endpoint replica (768×1024, empty prompt, strength 1.0 multiply, euler/simple/28, guidance 3.5). Doubles as simple 1-image workflow
flux-redux-prompt.json	1 image + prompt with authority (attn_bias 0.5 — fine here, NO ControlNet in this graph)
flux-redux-schnell.json	Replicate flux-redux-schnell replica (4 steps, no guidance, no prompt; 1024² = their 1:1 default, set 896×1152 for portrait)
flux-redux-style-composition.json (v1)	Historical — superseded by flux-v2
flux-redux-style-composition-bfl-lora.json (v1)	Historical — superseded

workflows/wip/ = every draft revision; workflows/original-old-workflows/ = the pre-project XLabs/Union-v1 workflows this started from (retired: XLabs is legacy, Union v1 needs SetUnionControlNetType which v2.0 must NOT have).

Models (all mirrored on HF aleph65/ComfyUI under models/)

file	dir	source	gated
flux1-dev.safetensors (bf16 23.8GB)	diffusion_models	black-forest-labs/FLUX.1-dev	yes
flux1-schnell.safetensors (bf16 23.8GB)	diffusion_models	black-forest-labs/FLUX.1-schnell	no
flux1-redux-dev.safetensors (129MB)	style_models	black-forest-labs/FLUX.1-Redux-dev	yes
FLUX.1-dev-ControlNet-Union-Pro-2.0.safetensors (4.28GB)	controlnet	Shakker-Labs (renamed from diffusion_pytorch_model.safetensors)	no
flux1-depth-dev-lora.safetensors (1.24GB)	loras	black-forest-labs/FLUX.1-Depth-dev-lora	yes
t5xxl_fp16.safetensors + clip_l.safetensors	text_encoders	Comfy-Org repack	no
ae.safetensors	vae	FLUX.1-dev repo	(yes)
sigclip_vision_patch14_384.safetensors	clip_vision	Comfy-Org/sigclip_vision_384	no
flux1-turbo-alpha.safetensors	loras	alimama-creative (fast-preview group)	no
depth_anything_v2_vitl.pth	auto-downloads into custom_nodes/comfyui_controlnet_aux/ckpts/		no

Custom node packs (3): comfyui_controlnet_aux (Fannovel16 — DepthAnythingV2), ComfyUI_essentials (cubiq — ImageResize+), ComfyUI_AdvancedRefluxControl (kaibioinfo — ReduxAdvanced alt branch). Everything else is ComfyUI core.

Replication on a fresh pod

1. Pod from plx1029/comfyui-qwen image (or any ComfyUI ≥0.3.8 — that's when native StyleModelApply strength landed).
2. HUGGING_FACE_ACCESS_TOKEN env var; accept licenses for FLUX.1-dev, FLUX.1-Redux-dev, FLUX.1-Depth-dev-lora on that account.
3. ./download_missing_models.sh flux-v2-redux-style-composition.json ... — resolves the node packs from each node's cnr_id/aux_id and pulls all models from the mirror. (Or run scripts/download_models.py which pulls from original sources.)
4. Restart ComfyUI, load the workflow, drop images, go. Defaults are the calibrated ones.
5. Batch runs: scripts/run_matrix_v4.py queues via the API (POST /prompt with API-format graphs, poll /history/<id>). scripts/prompts_v4.py holds the handwritten prompt fragments.

Findings (chronological, what actually mattered)

1. Baseline worked but bodies deformed (v1–v3): aesthetics great, frequent limb/torso deformities in both two-image paths. Single-image workflows always clean.
2. ROOT CAUSE (code-verified, undocumented upstream): attn_bias + ControlNet is broken. StyleModelApply(attn_bias) appends all 729 Redux tokens at FULL strength and stores the attenuation as an attention-bias mask. The base flux transformer honors the mask; the ControlNet branch never receives it (ControlNetFlux.forward_orig has no attn_mask param; not in its extra_conds). Base attends Redux @50%, CN attends @100% → two conditioning streams steer different images every step → ghosting/deformities. Also: any attention mask kicks flux off the flash-attention fast path. Fix: strength_type=multiply at 0.25–0.35 whenever a ControlNet is present. attn_bias remains fine in CN-free graphs. (8-run isolation diagnostic + two deep-research passes; see comparison_sheets/_diag_sheet.jpg — column d is the smoking gun.)
3. BFL depth-LoRA wants its spec: guidance 10.0 + LoRA 0.85. It is distilled at guidance 10; running 3–4 gives mush. Earlier sweeps that said "guidance 4 looks better" were contaminated by the attn_bias bug. The CN path conversely wants guidance ~3.0. The two paths need different FluxGuidance values.
4. Prompts are the anatomy lever. Flux resolves limb ambiguity from text. Empty prompt = worst; generic anatomy boilerplate = marginal; handwritten per-run prompts describing the composition target's actual content fused with the style ref's actual aesthetic = best. Depth maps are semantically ambiguous at limb crossings; the prompt disambiguates. Negative prompts are INERT at CFG 1.0 (comfy skips uncond entirely) — anatomy must be handled positively.
5. Calibration verdicts (30-run, comparison_sheets/_C*.jpg): multiply 0.35 > 0.25/0.30 (style strength, all clean) > ReduxAdvanced ds3 (clean but weakens style); 896×1152 fine despite Union-Pro-2.0's 512 training res (no penalty once the conditioning fight was fixed); CN end 0.4 loses composition, 0.6–0.8 both hold → keep 0.8, use 0.6 for freedom; depth-only single CN (canny stack changes scenes — keep off unless contour lock wanted).
6. Input preprocessing matters: fill/crop everything to 896×1152 in-graph (portrait 3:4 bucket), depth map == latent to the pixel, zero padding (padding becomes literal image content in the latent-concat/bfl path). Save crops + depth maps for auditing.
7. Input spec (what to feed it): 3:4 portrait, 1152×1536, sRGB 8-bit, no watermarks/text/ borders, EXIF baked. Style refs: style-defining content in the vertical middle — sigclip center-crops to a square, top/bottom ~12% discarded. Composition refs: clear fg/bg depth separation.
8. Hires two-pass (1.5× latent bislerp + denoise 0.30, Redux carried, CN released): validated win — real detail gain, aesthetic preserved. Ships bypassed.
9. Known edge case, unsolved: person-style × empty-room-target (our r×r4 combos) is a contradictory ask; results inconsistent even with interior prompts. Needs its own treatment.
10. Redux mode compatibility notes: scaled-fp8 flux checkpoints black-screen with Redux (ComfyUI #5849). XLabs CN stack is legacy/dormant. Union-Pro-2.0 must NOT have SetUnionControlNetType (that's v1-only).

Prompt-writing recipe (for new image pairs)

Write one sentence of content from the composition target (subject, pose, clothing, setting — what the depth map encodes), one sentence of style from the style ref (lighting type, palette, finish), then "Sharp focus, natural proportions, coherent anatomy." If the target has no person, say so explicitly and describe the space instead. Fragments for the 10 experiment inputs are in scripts/prompts_v4.py.

What's in this archive

README.md                  ← this file
workflows/                 ← all 7 finals + wip/ history + original-old-workflows/
inputs/                    ← inputs_v1 (first test set) + inputs_v2 (curated 1152×1536 3:4 set)
scripts/                   ← graph builders, all matrix/sweep/calibration runners, prompts, model downloader
research/                  ← FLUX-REDUX-CONTROLNET-RESEARCH.md (full research log w/ sources)
outputs_v4_final/          ← the final 84-run matrix: 36 style-comp (+hires/ +inputs/ audits),
                             36 bfl-lora, 12 singles, calib/ (30 calibration runs)
comparison_sheets/         ← every verdict sheet: _diag_sheet (the attn_bias smoking gun),
                             _A* (v2 sweeps), _C* (v4 calibration), _v2_vs_v3, _v3_vs_v4
claude_memory/             ← Claude Code memory files from this project

Run provenance: every test image used seed 777; matrices covered all 24 r×t + 12 ordered r×r combos per two-image workflow. v1→v4 output sets remain in /workspace/outputs* on the pod (not archived here except v4).

— Built with Claude Code (Fable 5), Aug 2026. It was fun, sir. 🫡