workflows/HEADSWAP-IMPROVEMENT-RESEARCH.md

# Head-Swap (head-swap-flux-klein.json) — Improvement Research, 2026-07-17

Research notes for improving the Lissie / FLUX.2 Klein head-swap workflow. No changes applied yet.

## Current state of the workflow

- FLUX.2 Klein 9B **distilled** bf16 (`flux-2-klein-9b.safetensors`), CFG 1.0, sampler `lcm`, Flux2Scheduler @ 8 steps
- Text encoder `qwen_3_8b_fp8mixed.safetensors` (fp8)
- LoRAs: `bfs_head_v1_flux-klein_9b_step3500_rank128` @ 1.0 + `lissie_flux2_klein` @ 1.0
- Body img → 1.5 MP → ReferenceLatent 1; face → RMBG-2.0 → 1.5 MP → ReferenceLatent 2
- Whole frame regenerated (EmptyFlux2LatentImage, denoise 1.0); no inpaint/compositing
- Bugs in graph: QwenVL expression caption generated but NOT wired into prompt (dead Reroute);
  `ImageCropByMask` orphaned; negative prompt is a no-op at CFG 1.0; unused BasicScheduler(bong_tangent, 4).

## Answer: better CLIP/text encoder

Yes — full-precision **bf16** encoder exists and is a drop-in via the same `CLIPLoader type=flux2`:

- `qwen_3_8b.safetensors` — 16.38 GB — official Comfy-Org repo:
  https://huggingface.co/Comfy-Org/flux2-klein-9B/resolve/main/split_files/text_encoders/qwen_3_8b.safetensors
- No published A/B vs fp8mixed; expect subtle gains ("different, not clearly worse") but it removes one
  quantization variable at zero cost on the H100 (80 GB).
- Klein 9B officially uses Qwen3-8B (BFL model card); FLUX.2-dev uses Mistral-Small-24B — not interchangeable.
- Other variants (fp4mixed, GGUF quants, ponpoke "uncensored" abliterated, SearchingMan 5.1B prune) exist;
  none demonstrated better than bf16; the prune is not a drop-in for the flux2 CLIPLoader.

## Highest-impact findings (ranked)

1. **LoRA-on-distilled mismatch (likely the biggest quality bug).** Community LoRAs (incl. BFS) are trained
   on **klein-base**; documented that base-trained LoRAs "do not work well" on the distilled checkpoint
   (nomadoor note; Comfy-Org/ComfyUI issue #11975). Fix options:
   - Switch to **flux-2-klein-base-9b** (bf16, ~29 GB VRAM): euler + Flux2Scheduler, **20–24 steps, CFG 4–5**
     (official base template: 20 steps CFG 5). Unlocks real negative prompts + higher step counts.
     https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B
   - Or base + **"Klein 4B/9B Base to Turbo" difference LoRA** @ ~0.5 → 10–20 steps at CFG 2–2.5
     (https://civitai.com/models/2324315).
2. **Crop-and-stitch instead of whole-frame regen.** Regenerate ONLY a ~1 MP crop around the head
   (ComfyUI-Inpaint-CropAndStitch: Inpaint Crop → edit → Inpaint Stitch); body/background pixels stay
   pixel-identical, face gets full latent resolution. This is the 2026 best-practice pattern
   (MyAIForce 8K head-swap; aistudynow; official BFS author recommends crop+inpaint for small faces).
   Alternative lighter fix: SetLatentNoiseMask / InpaintModelConditioning to restrict edits.
3. **Steps & sampler.** On the distilled model: official recipe is **euler + Flux2Scheduler @ 4 steps,
   CFG 1.0**; sweet spot 4–6; MORE steps on distilled → overcooked/waxy ("deep-fried") — current 8 with
   `lcm` is off-recipe on both counts. Increasing steps meaningfully = switch to base (20–50 steps).
   Max-quality combo on base: `res_2s` + `bong_tangent` (RES4LYF) @ ~20 steps. Avoid euler_ancestral.
4. **LoRA strengths.** Identity LoRA (lissie) @ 1.0 fights the "copy head from Picture 2" objective +
   reference latents. Try lissie 0.5–0.7 (or drop it — Picture 2 already carries identity). BFS can go
   1.2–1.3 if similarity is weak (author guidance). Also A/B **without BFS entirely** — MyAIForce found bare
   Klein sometimes swaps better (hair/eye-direction drift with the LoRA).
5. **Reference sizing.** Official templates scale every reference to **1 MP** (ImageScaleToTotalPixels,
   lanczos); workflow uses 1.5 MP refs (untested regime). Face ref: head+shoulders crop helps scale
   inference; keep full hair for head-swap; RMBG on ref is fine/helpful.
6. **Wire the QwenVL expression caption into the prompt** (StringConcatenate output currently dead-ends).
   Author-sanctioned to append expression description below the template.
7. **Finisher pass.** SeedVR2 (7B, tiled 2x, wavelet color fix, latent_noise_scale ~0.025) is the community
   default post-swap finisher; FaceDetailer (Impact Pack, supports FLUX) as low-denoise face refinement.
   GAN restorers (CodeFormer w≈0.5) only as a careful second pass — they can shift identity.
8. **Negative prompting on distilled**: only via NAG (native NAGuidance node — but open bug #12707 reports
   no effect on Klein + 2x time; ComfyUI-NAG custom node is the alternative). Real negatives → use base.

## Alternative / hybrid routes

- **Qwen-Image-Edit 2511 (already installed, bf16 40 GB + Lightning 4/8-step LoRAs).** 2511 specifically
  improved character consistency & image drift. Max quality: bf16, **40 steps, CFG 4, euler+simple, no
  Lightning** (Lightning = main "plastic skin" source; Skin_Fix LoRA counteracts). BFS **Head V5** is the
  2511-specific head-swap LoRA (Picture1=body, Picture2=face — same convention):
  https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap
  BFS author himself found Flux2 head-swap "not as good as with QWEN". Use TextEncodeQwenImageEditAdvanced
  (QwenEditUtils, installed) for pixel-exact reference latents instead of Plus's forced 1 MP rescale.
- **FLUX.2-dev 32B** fp8 (~32 GB + Mistral 24B TE) fits the H100; best FLUX.2 quality but no Klein
  head-swap LoRA; 20 steps, FluxGuidance 4.
- **PuLID-Flux2** (iFayens) — first PuLID for FLUX.2, supports Klein 9B, strength 1.0–1.4 — extra identity
  signal option. https://github.com/iFayens/ComfyUI-PuLID-Flux2
- **flux-2-klein-9b-kv** variant: 2.5x faster multi-reference editing, same 4-step behavior.
- Other LoRAs: BFS rank64/step3750 variant; SmartCharacterSwap (`jhuangswap`, handles occlusion);
  "Flux2-Klein-9B-Enhanced-Details" (fixes plastic skin); Klein Consistency LoRA.

## Candidate downloads (not yet downloaded)

| What | Size | URL |
|---|---|---|
| bf16 text encoder | 16.4 GB | https://huggingface.co/Comfy-Org/flux2-klein-9B/resolve/main/split_files/text_encoders/qwen_3_8b.safetensors |
| klein-base-9b (bf16) | ~18 GB | https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B (gated — accept license) |
| Base→Turbo LoRA | small | https://civitai.com/models/2324315 |
| BFS Head V5 (Qwen 2511) | ~600 MB | https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap |
| Inpaint-CropAndStitch nodes | — | https://github.com/lquesada/ComfyUI-Inpaint-CropAndStitch |
| RES4LYF (res_2s/bong_tangent) | — | https://github.com/ClownsharkBatwing/RES4LYF |
| SeedVR2 nodes + 7B model | ~16 GB | https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler |

## Suggested experiment order

1. Quick wins on current graph: euler @ 4–6 steps, refs → 1 MP, lissie → 0.6, wire expression caption,
   bf16 text encoder. (A/B with fixed seed.)
2. Structural: crop-and-stitch head inpaint version of the workflow.
3. Model: klein-base-9b @ 20 steps CFG 4–5 with both LoRAs (they're base-trained — should behave better),
   optional res_2s+bong_tangent.
4. Parallel track: Qwen-Edit 2511 + BFS Head V5, 40 steps CFG 4 no Lightning, vs the Klein pipeline.
5. Finisher: SeedVR2 pass on the winner.