Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,96 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
base_model: circlestone-labs/Anima
|
| 4 |
+
pipeline_tag: image-to-image
|
| 5 |
+
tags:
|
| 6 |
+
- anime
|
| 7 |
+
- anima
|
| 8 |
+
- character-consistency
|
| 9 |
+
- reference-based-generation
|
| 10 |
+
- identity-transfer
|
| 11 |
+
- lora
|
| 12 |
+
- lokr
|
| 13 |
+
- comfyui
|
| 14 |
---
|
| 15 |
+
|
| 16 |
+
# AnimaRefLora
|
| 17 |
+
|
| 18 |
+
Reference-based **character identity transfer** for [Anima](https://huggingface.co/circlestone-labs/Anima)
|
| 19 |
+
(the 2B anime DiT by CircleStone Labs / Comfy Org): give it **one reference image**
|
| 20 |
+
of a character, and it generates new images of that character following your prompt —
|
| 21 |
+
new poses, outfits, and scenes — without per-character LoRA training.
|
| 22 |
+
|
| 23 |
+
- **Code, training scripts, and ComfyUI plugin:**
|
| 24 |
+
[github.com/crazysheep924/AnimaRefLora](https://github.com/crazysheep924/AnimaRefLora)
|
| 25 |
+
- **Technical write-up:** see the `blog/` directory in the GitHub repo.
|
| 26 |
+
|
| 27 |
+
All example characters used in the project are AI-generated original designs.
|
| 28 |
+
|
| 29 |
+
## Files
|
| 30 |
+
|
| 31 |
+
```text
|
| 32 |
+
anima-reflora/idinject_500k.animaref.safetensors ← the release checkpoint (single-file bundle)
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
The `.animaref.safetensors` bundle is **adapter-only** (it is not a merged model).
|
| 36 |
+
One file packs everything inference needs:
|
| 37 |
+
|
| 38 |
+
| Component | Contents |
|
| 39 |
+
| --- | --- |
|
| 40 |
+
| `lora.*` | LoKr adapter for the Anima DiT (dim 512 / alpha 512) |
|
| 41 |
+
| `ref_conditioner.*` | reference/identity conditioning modules |
|
| 42 |
+
| `crepa_projector.*` | CREPA feature projector |
|
| 43 |
+
| metadata | feature config + RoPE reference-position layout (read automatically by the loader) |
|
| 44 |
+
|
| 45 |
+
## Requirements
|
| 46 |
+
|
| 47 |
+
You also need the base models, from
|
| 48 |
+
[circlestone-labs/Anima](https://huggingface.co/circlestone-labs/Anima):
|
| 49 |
+
|
| 50 |
+
- `anima-base-v1.0.safetensors` (DiT)
|
| 51 |
+
- the Qwen3-0.6B text encoder (file must be named `model.safetensors` — see plugin README)
|
| 52 |
+
- `qwen_image_vae.safetensors` (VAE)
|
| 53 |
+
|
| 54 |
+
> Verified on **Anima Base v1.0** only. Support for the community layer-expanded
|
| 55 |
+
> Anima variants (2.9B/3B-class) has not been confirmed.
|
| 56 |
+
|
| 57 |
+
## Usage (ComfyUI)
|
| 58 |
+
|
| 59 |
+
Install the plugin from the GitHub repo
|
| 60 |
+
([`comfyui/ComfyUI-AnimaRefLora`](https://github.com/crazysheep924/AnimaRefLora/tree/main/comfyui/ComfyUI-AnimaRefLora)),
|
| 61 |
+
then place the models as:
|
| 62 |
+
|
| 63 |
+
```text
|
| 64 |
+
ComfyUI/models/diffusion_models/anima-base-v1.0.safetensors
|
| 65 |
+
ComfyUI/models/text_encoders/model.safetensors
|
| 66 |
+
ComfyUI/models/vae/qwen_image_vae.safetensors
|
| 67 |
+
ComfyUI/models/anima_reflora/idinject_500k.animaref.safetensors
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
Node graph: `Anima RefLora Loader → Anima Ref Encode (reference image) → Anima RefLora Sampler`.
|
| 71 |
+
An optional `Anima Extra LoRA` node accepts regular Anima style LoRAs
|
| 72 |
+
(sd-scripts native and diffusers/PEFT keys, DoRA included).
|
| 73 |
+
|
| 74 |
+
## Usage (CLI)
|
| 75 |
+
|
| 76 |
+
The GitHub repo's `anima_reflora.local_ref_ab_infer` module provides the same
|
| 77 |
+
inference path used for the project's REF evaluations; see the repo README for
|
| 78 |
+
setup and commands.
|
| 79 |
+
|
| 80 |
+
## Training summary
|
| 81 |
+
|
| 82 |
+
- Trained on image pairs derived from Danbooru, with a T=3 frame contract
|
| 83 |
+
(head-crop reference / full reference / noised target) and a disjoint RoPE
|
| 84 |
+
layout separating reference and target positions.
|
| 85 |
+
- The release checkpoint corresponds to ~500K training steps, including
|
| 86 |
+
anti-copy objectives (diff-weighted loss, tag-level caption dropout) and
|
| 87 |
+
identity-accessory injection; full recipes and provenance are documented in
|
| 88 |
+
the GitHub repo's README.
|
| 89 |
+
|
| 90 |
+
## License
|
| 91 |
+
|
| 92 |
+
The adapter weights in this repository are released under **Apache-2.0**.
|
| 93 |
+
Note that the Anima base model itself is distributed under the CircleStone Labs
|
| 94 |
+
Non-Commercial License (and the NVIDIA Open Model License as a
|
| 95 |
+
Cosmos-Predict2 derivative) — using these adapters together with the base model
|
| 96 |
+
is subject to those base-model terms.
|