README.md

---
license: other
base_model:
- Lightricks/LTX-2.3
tags:
- ltx-video
- identity-preservation
- ipt2v
- reference-to-video
- lora
- comfyui
- ltx2.3
pipeline_tag: text-to-video
---

# LTX-Best-Face-ID — LTX-2.3 Identity LoRA (Reference-to-Video / IPT2V)

An identity-preserving **reference-to-video** LoRA for **LTX-2.3 (22B)**. Give it a reference
photo of a person + a text prompt, and it generates a video that keeps that person's identity.

Built with **overlap reference conditioning + TASS-RoPE (source-phase RoPE)** and a differentiable
**ArcFace identity loss**. Runs in ComfyUI via the companion **BFS Nodes**.

> **Status:** two reference modes are now available from the same recipe: the original
> **close-up face** reference, and a newer **character-sheet** reference (face + full-body
> views) that also carries clothing/appearance consistency — see
> [Character-Sheet Reference](#character-sheet-reference-new) below. Other experimental
> variants (native Gemma-vision conditioning, timestep-split texture injection) may be
> released later if they prove out.

---

## 🎬 Examples


### 🎬 Face ID Base
<!--
Drop result videos/gifs in an `assets/` folder in this repo and reference them below.
On the HF model page, .mp4 in a Markdown <video> tag renders inline; .gif works too.
Suggested layout: reference image (left) → generated video (right), with the prompt underneath.
-->

<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_1.mp4" controls autoplay loop muted></video> 
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_2.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_3.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_4.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_5.mp4" controls autoplay loop muted></video>

### 🎬 Character Sheet

<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_6.mp4" controls autoplay loop muted></video> 
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_7.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_8.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_9.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_10.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_11.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_12.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_13.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_14.mp4" controls autoplay loop muted></video>
---

## What it does

- **Reference-to-video (ref_t2v):** one reference image → video of that identity performing the
  prompt's action.
- Identity is injected by placing the **reference latent** in the target's frame-0 RoPE grid
  (overlap) and tagging it with a distinct **source phase** so the model knows it is a *reference*,
  not the first frame to generate.
- An auxiliary **ArcFace face-similarity loss** on the decoded prediction sharpens the identity.

---

## How it works (technique)

### Overlap reference + TASS-RoPE (source-phase)
The reference latent is concatenated to the video sequence sharing the frame-0 grid (classic
IC-LoRA "overlap"). To stop the reference from leaking into / being confused with the generated
first frame, each source gets a distinct **multiplicative RoPE phase**:

```
phase[d] = source_id · phase_scale · θ^(−d/L)      (θ = 10000)
target tokens: source_id = 0  (phase 0, exact no-op)
reference:     source_id = 2  (distinct rotary "tag")
```

This "source tag" lets the model separate *who is who* in the sequence and strongly improves
identity transfer. Because the tag is positional, the same mechanism generalizes to **multiple
references** (source_id = 2, 3, 4, …) for multi-subject conditioning.

> **TASS-RoPE** (*Temporal-Adjacent Spatial-Shifted RoPE*) is from **ST-DRC**, Chen et al.,
> *Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video
> Generation* — [arXiv:2606.02441](https://arxiv.org/abs/2606.02441). This LoRA uses an
> overlap + per-source RoPE-phase variant of that idea.

### ArcFace identity loss
During training the flow-matching prediction is decoded to pixels, the face is aligned and passed
through ArcFace (buffalo_l `w600k_r50`), and a cosine identity loss pulls the generated face toward
the reference embedding — the same recognition space used to evaluate identity.

---

## Character-Sheet Reference (NEW)

The base model above (and the guidance further down) was trained on **close-up face crops** —
great for identity, but the model never saw the person's actual clothing or body, so it has to
*invent* an outfit rather than reproduce a specific one.

We've since trained a **continuation checkpoint** on a **character-sheet reference** instead: a
single image with **4 panels side by side — a face close-up, then full-body front / side / back
views** of the same person, all on a white background. It uses the exact same mechanism
(`source_id=2`, `layout=overlap`) — it's just a *different, richer reference image*, so the model
learns to reproduce the reference's clothing and build consistently, not only the face.

**This checkpoint continues training from the close-up FaceID LoRA above rather than replacing
it** — in our tests the combination (close-up identity foundation + character-sheet continuation)
gave noticeably better and more consistent results than either recipe trained alone. If you only
have a plain headshot, the original close-up model still works well; the character-sheet
checkpoint is for when you want the generated video to also match a specific outfit/build.

**File:** [`Best_FaceID_CharacterSheet_v1.0_LoRA.safetensors`](Best_FaceID_CharacterSheet_v1.0_LoRA.safetensors)
— use it exactly like the base LoRA above (same node, same `source_id=2`/`layout=overlap`), just
swap the reference image for a 4-panel character sheet instead of a headshot, and set
`ref_resize_mode = native_resolution` (see below). This checkpoint has **no ArcFace projector** —
its identity/appearance comes entirely from the overlap reference latent, so leave that dropdown
on `None`.

> 💡 **Identity coming out weak?** Load the base [`Best_FaceID_v1.0_LoRA.safetensors`](Best_FaceID_v1.0_LoRA.safetensors)
> alongside this one at a low strength (**~0.2 or more**) — mixing in some of the base close-up
> LoRA strengthens identity while the character-sheet LoRA keeps handling clothing/body.

**How to build a character sheet (for now):** take (or generate) a normal reference photo, then
use ChatGPT (or another image-editing model, e.g. Nano-Banana) with this prompt:

> *"Convert the person in the photo into a character reference sheet with four panels arranged
> left to right: a close-up of the face, then full-body front, side, and back views, on a white
> background."*

Full write-up (longer prompt variant, gotchas) in [`character_sheet_prompt.md`](character_sheet_prompt.md).

> 🛠️ We're training a dedicated **Flux Klein 9B LoRA** to generate these character sheets
> directly (more consistent, no API needed) — **still in training, not released yet.** Until it's
> out, use the prompt above with ChatGPT/gpt-image for the most faithful character sheet.

**Usage difference vs. the close-up model:** the character-sheet reference must be exactly
**`1536×1024`** — that's the fixed resolution it was trained at, independent of the output video's
resolution. When using the **LTX Identity Transfer** node with this checkpoint, set
**`ref_resize_mode = native_resolution`** (not the default `match_target`) — otherwise the node
resizes the sheet down to the video's resolution and destroys the facial/clothing detail the model
relies on. This setting doesn't matter for the plain close-up reference, where ref and video share
the same resolution bucket.

### Character-Sheet Examples

<!--
Same pattern as above — drop the character-sheet result videos in `examples/` and reference them,
paired with the sheet image that produced them if you want to show the reference too.
-->

<!--
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/charsheet_sample_1.mp4" controls autoplay loop muted></video>
<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/charsheet_sample_2.mp4" controls autoplay loop muted></video>
-->

---

## Training details

| | |
|---|---|
| Base model | LTX-2.3 (22B) |
| Method | LoRA (rank 128, alpha 128) |
| Conditioning | overlap reference latent + TASS-RoPE source-phase (`source_id=2`, `phase_scale=1.0`) |
| Aux loss | ArcFace identity loss (+ temporal identity consistency) |
| Data | reference-image ↔ video pairs (OpenS2V subset + HuMoSet), close-up / frontal identities |
| Character-sheet continuation | same recipe, continued from the close-up checkpoint on 4-panel character-sheet references (face + front/side/back) |

---


## Reference image — what works best

This model was trained on face-focused reference images. Looking at the training data, every reference follows the same pattern:

- Close-up / bust crop — roughly chest-up, with the face large and clearly visible.
- A single subject, centered in the frame.
- Frontal or near-frontal (slight 3/4 angles are fine, but the face is always well exposed).
- Tightly cropped on the face/upper body — the training crops are about 460×406 (nearly square).

Recommendation: for the best identity transfer, give the model a reference that looks like this — crop your image to the face / upper body. Full-body shots, wide shots, or images where the face is small and far away work noticeably worse, because the model never saw references like that during training.

In short: a clean, frontal, well-lit close-up of the face is the ideal reference.

> **Using a full-body reference instead?** See [Character-Sheet Reference](#character-sheet-reference-new)
> above — the close-up guidance on this page is specifically for the base checkpoint.

## Captions (IMPORTANT)

This model was trained on captions in the **`ref_t2v:`** format that **describe the scene and the
person's action** (not just "a person"). Identity in LTX is strongly prompt-driven — the more the
prompt describes, the better the identity holds. For best results:

- Prefix prompts with **`ref_t2v:`**.
- Describe the **action, setting, framing, camera** in present-progressive.
- Describing **identity attributes** of the reference person (skin tone, hair, eyes, facial hair,
  glasses, face shape) **noticeably improves** the result.

The shared ComfyUI workflow includes a **Prompt Enhancer** that looks at the reference image (via
the model's own multimodal Gemma-3 text encoder) and automatically enriches your prompt with the
person's identity attributes — so you don't have to describe them by hand.

Example (enhanced):
```
ref_t2v: A light-skinned man with long dark-brown hair past his shoulders, narrow rectangular
metal-frame glasses, light blue-gray eyes and light stubble is folding clothes in a laundry room,
medium-wide shot. He places the folded clothes into a basket, then carries it across the room.
Neutral indoor lighting, simple domestic environment.
```

---

## Usage (ComfyUI)

**Requires the BFS Nodes:** https://github.com/alisson-anjos/ComfyUI-BFSNodes

1. Install ComfyUI-BFSNodes (ComfyUI Manager → Install via Git URL, or clone into `custom_nodes/`).
   Dependencies (`insightface`, `transformers`, …) install automatically.
2. Load LTX-2.3 (checkpoint + the Gemma-3 text encoder / CLIP) as usual.
3. Add the **LTX Identity Transfer** (overlap + source-phase) node; feed it the reference image.
   Using the character-sheet checkpoint? Set **`ref_resize_mode = native_resolution`** (see
   [Character-Sheet Reference](#character-sheet-reference-new)).
4. Load **this LoRA** on the MODEL path.
5. Use the shared workflow (it wires the **Prompt Enhancer** → identity node → sampler).

A ready-made **workflow JSON** is included alongside the model.

### About the ArcFace projector (optional — you don't need it)

The identity node has an optional **ArcFace projector** dropdown (it projects a face embedding into
extra conditioning tokens). **You can leave it on `None`.** In our experiments the projector's
effect was **marginal** — identity is carried almost entirely by the overlap reference latent (and
the prompt), so the projector adds very little on top and is not required for this model. It is kept
only as an optional extra; skipping it keeps the graph simpler with no meaningful quality loss.

---

## Tips for best results

- Use a **clean, frontal, well-lit close-up** as the reference — identity quality tracks reference
  quality closely. Need clothing/body consistency too? Use the
  [character-sheet checkpoint](#character-sheet-reference-new) instead of a plain close-up.
- Let the **Prompt Enhancer** add the identity attributes (or write them yourself).
- Keep the reference framing simple; busy/occluded reference faces degrade identity.

---

## Limitations

- **First-frame / "mask" tendency:** because identity comes from the raw reference *latent*, the
  reference appearance (and framing / accessories like glasses) can partially copy into the video —
  it can look like the reference is "pasted"/masked rather than freshly generated. Using a clean
  frontal reference and the trim-first-frame node mitigates this.
- **Prompt dependence:** identity is much stronger when the prompt describes the person. A bare,
  generic prompt gives weaker identity — use the Prompt Enhancer.
- **Close-up bias (base checkpoint):** the base training data skews to close-up / frontal faces, so
  full-body or large-angle shots hold identity less well *with the base checkpoint* — use the
  character-sheet checkpoint for those cases instead.
- **Metric caveat:** ArcFace similarity is unreliable on small / turned / occluded faces — judge
  visually, not only by score.

---

## Citation

If you use this model, please cite the base model and the technique:

```bibtex
@misc{ltx_best_face_id,
  title  = {LTX-Best-Face-ID: LTX-2.3 Identity LoRA (Reference-to-Video)},
  author = {Alissonerdx},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Alissonerdx/LTX-Best-Face-ID}}
}
```

- **Base model:** LTX-2.3 (Lightricks).
- **TASS-RoPE / reference conditioning — ST-DRC:**

```bibtex
@article{chen2026stdrc,
  title  = {Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation},
  author = {Chen, Yuheng and Hu, Teng and Wang, Yuji and He, Qingdong and Ma, Lizhuang and Zhang, Jiangning},
  journal = {arXiv preprint arXiv:2606.02441},
  year   = {2026}
}
```

- **Identity loss:** ArcFace (Deng et al., 2019) via InsightFace `buffalo_l`.
- **Tooling:** [ComfyUI-BFSNodes](https://github.com/alisson-anjos/ComfyUI-BFSNodes).

---

## Acknowledgements

Trained on top of LTX-2.3. Identity supervision via InsightFace ArcFace. ComfyUI integration via
the BFS Nodes.