Head-Swap (head-swap-flux-klein.json) — Improvement Research, 2026-07-17
Research notes for improving the Lissie / FLUX.2 Klein head-swap workflow. No changes applied yet.
Current state of the workflow
- FLUX.2 Klein 9B distilled bf16 (
flux-2-klein-9b.safetensors), CFG 1.0, samplerlcm, Flux2Scheduler @ 8 steps - Text encoder
qwen_3_8b_fp8mixed.safetensors(fp8) - LoRAs:
bfs_head_v1_flux-klein_9b_step3500_rank128@ 1.0 +lissie_flux2_klein@ 1.0 - Body img → 1.5 MP → ReferenceLatent 1; face → RMBG-2.0 → 1.5 MP → ReferenceLatent 2
- Whole frame regenerated (EmptyFlux2LatentImage, denoise 1.0); no inpaint/compositing
- Bugs in graph: QwenVL expression caption generated but NOT wired into prompt (dead Reroute);
ImageCropByMaskorphaned; negative prompt is a no-op at CFG 1.0; unused BasicScheduler(bong_tangent, 4).
Answer: better CLIP/text encoder
Yes — full-precision bf16 encoder exists and is a drop-in via the same CLIPLoader type=flux2:
qwen_3_8b.safetensors— 16.38 GB — official Comfy-Org repo: https://huggingface.co/Comfy-Org/flux2-klein-9B/resolve/main/split_files/text_encoders/qwen_3_8b.safetensors- No published A/B vs fp8mixed; expect subtle gains ("different, not clearly worse") but it removes one quantization variable at zero cost on the H100 (80 GB).
- Klein 9B officially uses Qwen3-8B (BFL model card); FLUX.2-dev uses Mistral-Small-24B — not interchangeable.
- Other variants (fp4mixed, GGUF quants, ponpoke "uncensored" abliterated, SearchingMan 5.1B prune) exist; none demonstrated better than bf16; the prune is not a drop-in for the flux2 CLIPLoader.
Highest-impact findings (ranked)
- LoRA-on-distilled mismatch (likely the biggest quality bug). Community LoRAs (incl. BFS) are trained
on klein-base; documented that base-trained LoRAs "do not work well" on the distilled checkpoint
(nomadoor note; Comfy-Org/ComfyUI issue #11975). Fix options:
- Switch to flux-2-klein-base-9b (bf16, ~29 GB VRAM): euler + Flux2Scheduler, 20–24 steps, CFG 4–5 (official base template: 20 steps CFG 5). Unlocks real negative prompts + higher step counts. https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B
- Or base + "Klein 4B/9B Base to Turbo" difference LoRA @ ~0.5 → 10–20 steps at CFG 2–2.5 (https://civitai.com/models/2324315).
- Crop-and-stitch instead of whole-frame regen. Regenerate ONLY a ~1 MP crop around the head (ComfyUI-Inpaint-CropAndStitch: Inpaint Crop → edit → Inpaint Stitch); body/background pixels stay pixel-identical, face gets full latent resolution. This is the 2026 best-practice pattern (MyAIForce 8K head-swap; aistudynow; official BFS author recommends crop+inpaint for small faces). Alternative lighter fix: SetLatentNoiseMask / InpaintModelConditioning to restrict edits.
- Steps & sampler. On the distilled model: official recipe is euler + Flux2Scheduler @ 4 steps,
CFG 1.0; sweet spot 4–6; MORE steps on distilled → overcooked/waxy ("deep-fried") — current 8 with
lcmis off-recipe on both counts. Increasing steps meaningfully = switch to base (20–50 steps). Max-quality combo on base:res_2s+bong_tangent(RES4LYF) @ ~20 steps. Avoid euler_ancestral. - LoRA strengths. Identity LoRA (lissie) @ 1.0 fights the "copy head from Picture 2" objective + reference latents. Try lissie 0.5–0.7 (or drop it — Picture 2 already carries identity). BFS can go 1.2–1.3 if similarity is weak (author guidance). Also A/B without BFS entirely — MyAIForce found bare Klein sometimes swaps better (hair/eye-direction drift with the LoRA).
- Reference sizing. Official templates scale every reference to 1 MP (ImageScaleToTotalPixels, lanczos); workflow uses 1.5 MP refs (untested regime). Face ref: head+shoulders crop helps scale inference; keep full hair for head-swap; RMBG on ref is fine/helpful.
- Wire the QwenVL expression caption into the prompt (StringConcatenate output currently dead-ends). Author-sanctioned to append expression description below the template.
- Finisher pass. SeedVR2 (7B, tiled 2x, wavelet color fix, latent_noise_scale ~0.025) is the community default post-swap finisher; FaceDetailer (Impact Pack, supports FLUX) as low-denoise face refinement. GAN restorers (CodeFormer w≈0.5) only as a careful second pass — they can shift identity.
- Negative prompting on distilled: only via NAG (native NAGuidance node — but open bug #12707 reports no effect on Klein + 2x time; ComfyUI-NAG custom node is the alternative). Real negatives → use base.
Alternative / hybrid routes
- Qwen-Image-Edit 2511 (already installed, bf16 40 GB + Lightning 4/8-step LoRAs). 2511 specifically improved character consistency & image drift. Max quality: bf16, 40 steps, CFG 4, euler+simple, no Lightning (Lightning = main "plastic skin" source; Skin_Fix LoRA counteracts). BFS Head V5 is the 2511-specific head-swap LoRA (Picture1=body, Picture2=face — same convention): https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap BFS author himself found Flux2 head-swap "not as good as with QWEN". Use TextEncodeQwenImageEditAdvanced (QwenEditUtils, installed) for pixel-exact reference latents instead of Plus's forced 1 MP rescale.
- FLUX.2-dev 32B fp8 (~32 GB + Mistral 24B TE) fits the H100; best FLUX.2 quality but no Klein head-swap LoRA; 20 steps, FluxGuidance 4.
- PuLID-Flux2 (iFayens) — first PuLID for FLUX.2, supports Klein 9B, strength 1.0–1.4 — extra identity signal option. https://github.com/iFayens/ComfyUI-PuLID-Flux2
- flux-2-klein-9b-kv variant: 2.5x faster multi-reference editing, same 4-step behavior.
- Other LoRAs: BFS rank64/step3750 variant; SmartCharacterSwap (
jhuangswap, handles occlusion); "Flux2-Klein-9B-Enhanced-Details" (fixes plastic skin); Klein Consistency LoRA.
Candidate downloads (not yet downloaded)
| What | Size | URL |
|---|---|---|
| bf16 text encoder | 16.4 GB | https://huggingface.co/Comfy-Org/flux2-klein-9B/resolve/main/split_files/text_encoders/qwen_3_8b.safetensors |
| klein-base-9b (bf16) | ~18 GB | https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B (gated — accept license) |
| Base→Turbo LoRA | small | https://civitai.com/models/2324315 |
| BFS Head V5 (Qwen 2511) | ~600 MB | https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap |
| Inpaint-CropAndStitch nodes | — | https://github.com/lquesada/ComfyUI-Inpaint-CropAndStitch |
| RES4LYF (res_2s/bong_tangent) | — | https://github.com/ClownsharkBatwing/RES4LYF |
| SeedVR2 nodes + 7B model | ~16 GB | https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler |
Suggested experiment order
- Quick wins on current graph: euler @ 4–6 steps, refs → 1 MP, lissie → 0.6, wire expression caption, bf16 text encoder. (A/B with fixed seed.)
- Structural: crop-and-stitch head inpaint version of the workflow.
- Model: klein-base-9b @ 20 steps CFG 4–5 with both LoRAs (they're base-trained — should behave better), optional res_2s+bong_tangent.
- Parallel track: Qwen-Edit 2511 + BFS Head V5, 40 steps CFG 4 no Lightning, vs the Klein pipeline.
- Finisher: SeedVR2 pass on the winner.