README.md

metadata

license: apache-2.0
base_model:
  - black-forest-labs/FLUX.2-klein-base-9B
pipeline_tag: image-to-image
tags:
  - flux
  - flux2
  - klein
  - depth
  - controlnet
  - refcontrol
  - img2img
  - image
  - editing
  - lora
library_name: diffusers

RefControl FLUX.2 Klein 9B – Reference Depth LoRA

📝 Short description

A LoRA for FLUX.2 Klein 9B Base that fuses a reference image with a depth map.
It preserves identity and style from the reference while following the pose and structure from the depth map.

Trigger word: refcontrol

---

📊 Examples

Reference	Depth	Result

---

📖 Extended description

This LoRA was primarily trained on humans, but it also works with objects and stylized characters.
Its main purpose is to preserve identity — facial features, clothing, style, or object characteristics — from the reference image, while adapting the subject to the pose and composition defined by the depth map.

Part of the RefControl family: reference + control fusion for consistent, controllable generation on FLUX.2 Klein 9B Base.

---

⚙️ How to use

1. Use first image as depth reference, second image input
2. Add the trigger word refcontrol in your prompt.
3. Adjust LoRA weight (recommended 0.8–1.0) depending on how strongly you want to preserve identity.

ComfyUI requirenments

For extracting depth you need to install nodes with depth estimation in workflow used:

https://github.com/Fannovel16/comfyui_controlnet_aux

Base model

Use with black-forest-labs/FLUX.2-klein-base-9B.
The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, guidance_scale ~4.0).

✅ Example prompt

refcontrol

---

🎯 What it does

• Preserves character or object identity across generations.
• Embeds the subject into a new pose or scene defined by the depth map.
• Works best when the depth map has similar proportions and scale to the reference.

---

⚡ Tips

• Best results when the depth map is not drastically different in object scale from the reference.
• Combine with text prompts to refine background, lighting, or mood.
• Can be combined with other RefControl LoRAs (pose, lineart, canny) on compatible pipelines for multi-constraint generation.

---

📌 Use cases

• Human portraits in different poses.
• Consistent character design across multiple scenes.
• Object transformations (cars, furniture, props) with depth-guided placement.
• Storyboarding, comics, or animation frame generation.

---

📦 Files

• Weights: flux2_klein_9b_refcontrol_depth.safetensors
• ComfyUI workflow: image_flux2_klein_image_edit_9b_refcontrol_depth.json
• Repo: thedeoxen/refcontrol-FLUX.2-klein-9B-reference-depth-lora