---
license: apache-2.0
base_model:
- black-forest-labs/FLUX.2-klein-base-9B
pipeline_tag: image-to-image
tags:
- flux
- flux2
- klein
- normal
- normal-map
- controlnet
- refcontrol
- img2img
- image
- editing
- lora
library_name: diffusers
---

# RefControl FLUX.2 Klein 9B – Reference Normal LoRA

## 📝 Short description
A LoRA for **FLUX.2 Klein 9B Base** that fuses a **reference image** (identity) with a **normal map** (structure / surface orientation).  
It preserves **identity and style** from the reference while following the **3D shape and composition** from the normal control map.

**Trigger word:** `refcontrol`

<Gallery />

---

## 📊 Examples

Each preview is a **single combined image** from ComfyUI: **Normal → Reference → Result** (left to right).

<table>
  <thead>
    <tr>
      <th>Normal → Reference → Result</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/resolve/main/images/a (1).png" width="960" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/resolve/main/images/a (2).png" width="960" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/resolve/main/images/a (3).png" width="960" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/resolve/main/images/a (4).png" width="960" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/resolve/main/images/a (5).png" width="960" /></td>
    </tr>
  </tbody>
</table>
---

## 📖 Extended description
This LoRA was primarily trained on **humans**, but it also works with **stylized characters** and some objects.  
Its main purpose is to **preserve identity** — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the **structure and surface orientation** defined by the normal map.

**FLUX.2 Klein 9B Base** already handles reference + normal-guided transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and **better preserves character identity and normal-map fidelity** than the base model without LoRA.

Part of the **RefControl** family: reference + control fusion for consistent, controllable generation on **FLUX.2 Klein 9B Base**.

---

## ⚙️ How to use
1. Use the **first image** as the **normal map** (structure / surface orientation).
2. Use the **second image** as the **reference** (character, person, or object).
3. Add the trigger word **`refcontrol`** in your prompt.
4. Adjust LoRA weight (recommended **0.8–1.0**) depending on how strongly you want to preserve identity.

## ComfyUI requirements
Normal map extraction in the included workflow uses **DSINE-NormalMapPreprocessor** (via [comfyui_controlnet_aux](https://github.com/Fannovel16/comfyui_controlnet_aux)):

https://github.com/Fannovel16/comfyui_controlnet_aux

You can disable the built-in normal preprocessor if your input is already a normal map image.

### Base model
Trained on and recommended with [`black-forest-labs/FLUX.2-klein-base-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B).  
The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, `guidance_scale` ~4.0).

The LoRA also works with the **4-step distilled** [`black-forest-labs/FLUX.2-klein-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-9B) for faster inference (~4 steps, `guidance_scale` ~1.0), but **quality may be slightly lower** — especially for identity and normal-map fidelity — compared to the Base model.

### ✅ Example prompt

```
refcontrol
```

---

## 🎯 What it does
- Preserves **character or object identity** across generations.
- Adapts the subject to a **new structure or composition** defined by the normal map.
- Works best when the normal map has **similar proportions and scale** to the reference.

---

## ⚡ Tips
- Best results when the normal map is not drastically different in body scale or framing from the reference.
- Combine with **text prompts** to refine background, lighting, or mood.
- **Normal maps** with clear surface orientation and consistent lighting work well as control input.

---

## 📌 Use cases
- Character restyling while keeping identity and normal-defined structure.
- Consistent **character design** across different compositions.
- Illustration and storyboard generation with normal-guided layout.
- Object transformations with surface-orientation-guided placement.

---

## 📦 Files
- **Weights:** [`flux2_klein_9b_refcontrol_normal.safetensors`](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/blob/main/flux2_klein_9b_refcontrol_normal.safetensors)
- **ComfyUI workflow:** [`refcontrol_normal_flux_klein_9b.json`](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora/blob/main/refcontrol_normal_flux_klein_9b.json)
- **Repo:** [thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora)