Model card redesign: banner, gallery widget, showcase images
Browse files- .gitattributes +15 -0
- README.md +114 -44
- showcase/banner.jpg +3 -0
- showcase/girl_beach.jpg +3 -0
- showcase/girl_cake.jpg +3 -0
- showcase/girl_classroom.jpg +3 -0
- showcase/girl_neon.jpg +3 -0
- showcase/girl_sakura.jpg +3 -0
- showcase/girl_umbrella.jpg +3 -0
- showcase/man_beach.jpg +3 -0
- showcase/man_cafe.jpg +3 -0
- showcase/man_library.jpg +3 -0
- showcase/man_night_rain.jpg +3 -0
- showcase/man_sakura.jpg +3 -0
- showcase/man_snow.jpg +3 -0
- showcase/ref_girl.jpg +3 -0
- showcase/ref_man.jpg +3 -0
.gitattributes
CHANGED
|
@@ -38,3 +38,18 @@ figures/fig3_unseen_artist_oc.jpg filter=lfs diff=lfs merge=lfs -text
|
|
| 38 |
figures/fig4_ref_guidance.jpg filter=lfs diff=lfs merge=lfs -text
|
| 39 |
figures/fig5_corrections.jpg filter=lfs diff=lfs merge=lfs -text
|
| 40 |
figures/fig6_failures.jpg filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
figures/fig4_ref_guidance.jpg filter=lfs diff=lfs merge=lfs -text
|
| 39 |
figures/fig5_corrections.jpg filter=lfs diff=lfs merge=lfs -text
|
| 40 |
figures/fig6_failures.jpg filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
showcase/banner.jpg filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
showcase/girl_beach.jpg filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
showcase/girl_cake.jpg filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
showcase/girl_classroom.jpg filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
showcase/girl_neon.jpg filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
showcase/girl_sakura.jpg filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
showcase/girl_umbrella.jpg filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
showcase/man_beach.jpg filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
showcase/man_cafe.jpg filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
showcase/man_library.jpg filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
showcase/man_night_rain.jpg filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
showcase/man_sakura.jpg filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
showcase/man_snow.jpg filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
showcase/ref_girl.jpg filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
showcase/ref_man.jpg filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -13,75 +13,145 @@ tags:
|
|
| 13 |
- character-consistency
|
| 14 |
- style-transfer
|
| 15 |
- anime
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
---
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
# Anima Reference Adapter (pilot, r32)
|
| 19 |
|
| 20 |
-
|
| 21 |
-
**character and its drawing style from one reference image**, while **pose, framing and background
|
| 22 |
-
follow the prompt**. The prompt does not need the character's name, artist tags or appearance tags.
|
| 23 |
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
> independent project, not an official product of CircleStone Labs and not endorsed by it. See
|
| 30 |
-
> [NOTICE.txt](NOTICE.txt) and [LICENSE.md](LICENSE.md). Non-commercial use only.
|
| 31 |
|
| 32 |
-
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
|
|
|
| 36 |
|
| 37 |
-
|
|
|
|
| 38 |
|
| 39 |
-
|
| 40 |
-
Use the nodes and workflows from **[ElectronGirl/ComfyUI-Anima-Ref-Adapter](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter)**:
|
| 41 |
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
-
|
| 48 |
-
`ref_guidance` 4.5–6 with `apg` correction.
|
| 49 |
|
| 50 |
-
|
| 51 |
|
| 52 |
-
|
| 53 |
-
the DiT's time axis. It gets timestep 0 (clean) and its RoPE h coordinates are shifted by one target
|
| 54 |
-
height, so the video prior cannot copy it pixel-aligned (no pose copying, unlike an aligned temporal
|
| 55 |
-
reference). A rank-32 LoRA (45.9M parameters, all attention and MLP projections of the 28 blocks)
|
| 56 |
-
learns to read the frame; it is applied only to model calls that carry the reference. Guidance
|
| 57 |
-
separates text and reference scales, with adaptive projected guidance against over-saturation.
|
| 58 |
-
Full details, data and failure modes: [report.md](report.md).
|
| 59 |
|
| 60 |
-
##
|
| 61 |
|
| 62 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|---|---|
|
| 64 |
-
|
|
| 65 |
-
|
|
| 66 |
-
|
|
| 67 |
-
| optimization | 2,000 steps, batch 4, AdamW 1e-4 cosine, bf16, 2.4 h on one RTX PRO 6000 Max-Q |
|
| 68 |
|
| 69 |
## Limitations
|
| 70 |
|
| 71 |
-
- Pilot
|
| 72 |
-
-
|
| 73 |
-
-
|
| 74 |
-
-
|
| 75 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
|
| 77 |
-
|
| 78 |
-
imitations as their work, and follow the license's restrictions on use (§4 of LICENSE.md).
|
| 79 |
|
| 80 |
## Files
|
| 81 |
|
| 82 |
| file | |
|
| 83 |
|---|---|
|
| 84 |
-
| `anima_ref_adapter_pilot_r32.safetensors` | LoRA
|
| 85 |
| `report.md`, `figures/` | technical report |
|
| 86 |
| `workflows/` | ComfyUI workflows |
|
|
|
|
| 87 |
| `LICENSE.md`, `NOTICE.txt` | CircleStone Non-Commercial License and attribution notice |
|
|
|
|
| 13 |
- character-consistency
|
| 14 |
- style-transfer
|
| 15 |
- anime
|
| 16 |
+
widget:
|
| 17 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, full body, walking, hands in pockets, city, street, night, rain"
|
| 18 |
+
output:
|
| 19 |
+
url: showcase/man_night_rain.jpg
|
| 20 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, upper body, sitting, holding cup, drinking, cafe, indoors, window, sunlight"
|
| 21 |
+
output:
|
| 22 |
+
url: showcase/man_cafe.jpg
|
| 23 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, cowboy shot, looking at viewer, smile, beach, ocean, sunset"
|
| 24 |
+
output:
|
| 25 |
+
url: showcase/man_beach.jpg
|
| 26 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, portrait, from side, looking afar, cherry blossoms, petals"
|
| 27 |
+
output:
|
| 28 |
+
url: showcase/man_sakura.jpg
|
| 29 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, upper body, reading, holding book, library, bookshelf"
|
| 30 |
+
output:
|
| 31 |
+
url: showcase/man_library.jpg
|
| 32 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, full body, standing, snow, forest, winter"
|
| 33 |
+
output:
|
| 34 |
+
url: showcase/man_snow.jpg
|
| 35 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, full body, walking, smile, city, street, night, neon lights"
|
| 36 |
+
output:
|
| 37 |
+
url: showcase/girl_neon.jpg
|
| 38 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, upper body, sitting, eating, cake, cafe, indoors"
|
| 39 |
+
output:
|
| 40 |
+
url: showcase/girl_cake.jpg
|
| 41 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, cowboy shot, looking back, smile, beach, ocean, sunset"
|
| 42 |
+
output:
|
| 43 |
+
url: showcase/girl_beach.jpg
|
| 44 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, portrait, closed eyes, smile, cherry blossoms, petals, wind"
|
| 45 |
+
output:
|
| 46 |
+
url: showcase/girl_sakura.jpg
|
| 47 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, full body, sitting, on chair, crossed legs, classroom, sunlight"
|
| 48 |
+
output:
|
| 49 |
+
url: showcase/girl_classroom.jpg
|
| 50 |
+
- text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, upper body, holding umbrella, rain, street, looking at viewer"
|
| 51 |
+
output:
|
| 52 |
+
url: showcase/girl_umbrella.jpg
|
| 53 |
---
|
| 54 |
|
| 55 |
+
<div align="center">
|
| 56 |
+
|
| 57 |
+
<img src="showcase/banner.jpg" width="100%" alt="Anima Reference Adapter">
|
| 58 |
+
|
| 59 |
# Anima Reference Adapter (pilot, r32)
|
| 60 |
|
| 61 |
+
**Character and style from one image. Pose, place and framing from your prompt.**
|
|
|
|
|
|
|
| 62 |
|
| 63 |
+
[](https://huggingface.co/ElectronGirl/anima-ref-adapter)
|
| 64 |
+
[](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter)
|
| 65 |
+
[](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter/blob/main/docs/report.md)
|
| 66 |
+
[](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter/blob/main/LICENSE)
|
| 67 |
+
[](https://huggingface.co/ElectronGirl/anima-ref-adapter/blob/main/LICENSE.md)
|
|
|
|
|
|
|
| 68 |
|
| 69 |
+
</div>
|
| 70 |
|
| 71 |
+
A LoRA + ComfyUI recipe for [Anima](https://huggingface.co/circlestone-labs/Anima): give one reference
|
| 72 |
+
image and a prompt that only describes the scene. Hair, eyes, outfit and accessories come from the image;
|
| 73 |
+
pose, background and composition follow the prompt. No character name, artist or appearance tags needed.
|
| 74 |
|
| 75 |
+
> **This is not a plain LoRA.** The reference must be attached as an extra latent frame with shifted RoPE.
|
| 76 |
+
> Use the nodes and workflows from **[ElectronGirl/ComfyUI-Anima-Ref-Adapter](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter)**.
|
| 77 |
|
| 78 |
+
## Gallery
|
|
|
|
| 79 |
|
| 80 |
+
Prompt: `masterpiece, best quality, score_7, newest, highres, absurdres, safe, <gender>, solo, <scene>`. Euler 28 steps, cfg 4.5, reference guidance 4.5 + APG.
|
| 81 |
+
Not cherry-picked.
|
| 82 |
+
|
| 83 |
+
<table>
|
| 84 |
+
<tr><td rowspan="2" align="center" valign="middle" width="25%"><img src="showcase/ref_man.jpg" width="100%"><br><b>Reference A</b><br><sub>input image</sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_night_rain.jpg" width="100%"><br><sub><code>full body, walking, hands in pockets, city, street, night, rain</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_cafe.jpg" width="100%"><br><sub><code>upper body, sitting, holding cup, drinking, cafe, indoors, window, sunlight</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_beach.jpg" width="100%"><br><sub><code>cowboy shot, looking at viewer, smile, beach, ocean, sunset</code></sub></td></tr>
|
| 85 |
+
<tr><td align="center" valign="top" width="25%"><img src="showcase/man_sakura.jpg" width="100%"><br><sub><code>portrait, from side, looking afar, cherry blossoms, petals</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_library.jpg" width="100%"><br><sub><code>upper body, reading, holding book, library, bookshelf</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_snow.jpg" width="100%"><br><sub><code>full body, standing, snow, forest, winter</code></sub></td></tr>
|
| 86 |
+
</table>
|
| 87 |
+
|
| 88 |
+
<table>
|
| 89 |
+
<tr><td rowspan="2" align="center" valign="middle" width="25%"><img src="showcase/ref_girl.jpg" width="100%"><br><b>Reference B</b><br><sub>input image</sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_neon.jpg" width="100%"><br><sub><code>full body, walking, smile, city, street, night, neon lights</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_cake.jpg" width="100%"><br><sub><code>upper body, sitting, eating, cake, cafe, indoors</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_beach.jpg" width="100%"><br><sub><code>cowboy shot, looking back, smile, beach, ocean, sunset</code></sub></td></tr>
|
| 90 |
+
<tr><td align="center" valign="top" width="25%"><img src="showcase/girl_sakura.jpg" width="100%"><br><sub><code>portrait, closed eyes, smile, cherry blossoms, petals, wind</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_classroom.jpg" width="100%"><br><sub><code>full body, sitting, on chair, crossed legs, classroom, sunlight</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_umbrella.jpg" width="100%"><br><sub><code>upper body, holding umbrella, rain, street, looking at viewer</code></sub></td></tr>
|
| 91 |
+
</table>
|
| 92 |
|
| 93 |
+
### Held-out characters
|
|
|
|
| 94 |
|
| 95 |
+
<img src="figures/fig2_heldout.jpg" width="100%">
|
| 96 |
|
| 97 |
+
<sub>reference · adapter with a scene-only prompt · same prompt without the adapter · image generated with the full character tags</sub>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
+
## Quick start (ComfyUI)
|
| 100 |
|
| 101 |
+
1. Put `anima_ref_adapter_pilot_r32.safetensors` in `ComfyUI/models/loras/`.
|
| 102 |
+
2. `git clone https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter` into `ComfyUI/custom_nodes/`, restart.
|
| 103 |
+
3. Open `workflows/anima_ref_adapter.json`, load a reference image, write a scene-only prompt.
|
| 104 |
+
|
| 105 |
+
| | recommended | notes |
|
| 106 |
+
|---|---|---|
|
| 107 |
+
| base model | Anima base v1.0 | no turbo LoRA (the adapter was trained on the base model) |
|
| 108 |
+
| compute | bf16 (`ModelComputeDtype`) | Anima overflows in fp16 |
|
| 109 |
+
| sampler | Euler, 28-30 steps, `simple` | single-resolution samplers; SamplerSPEED's half-resolution stage is unreliable |
|
| 110 |
+
| `cfg` | 4.5 | text guidance |
|
| 111 |
+
| `ref_guidance` | 4.5 - 6 | 1 = weak and cheaper; higher = more character detail and style, more saturation |
|
| 112 |
+
| `correction` | `apg` | or `rescale` (phi 0.5) |
|
| 113 |
+
| prompt | quality + gender + `solo` + scene tags | no character name, artist tag or appearance tags |
|
| 114 |
+
|
| 115 |
+
## How it works
|
| 116 |
+
|
| 117 |
+
The reference image is letterboxed to the output size, VAE-encoded and appended as a **second frame** on
|
| 118 |
+
the DiT's time axis. It is fed **clean** (timestep 0) and its **RoPE h coordinates are shifted by one
|
| 119 |
+
target height**, so the video prior cannot copy it pixel-aligned (no pose copying). A **rank-32 LoRA**
|
| 120 |
+
(45.9M parameters, all attention and MLP projections of the 28 blocks), applied only to calls that carry
|
| 121 |
+
the reference, learns to read it. Guidance separates text and reference scales, with adaptive projected
|
| 122 |
+
guidance against over-saturation. Full details: [report.md](report.md).
|
| 123 |
+
|
| 124 |
+
| training | |
|
| 125 |
|---|---|
|
| 126 |
+
| data | 3,600 images generated by Anima: 600 groups of one character in one artist style x 6 scenes; 300 characters, 427 artists; 12 characters held out |
|
| 127 |
+
| pairs | 17,280 (reference, target) pairs; target prompt keeps scene tags only (70%), + some appearance tags (20%), or nothing (10%) |
|
| 128 |
+
| optimization | flow matching, 2,000 steps, batch 4, AdamW 1e-4 cosine, bf16; 2.4 h on one RTX PRO 6000 Max-Q |
|
|
|
|
| 129 |
|
| 130 |
## Limitations
|
| 131 |
|
| 132 |
+
- **Pilot release**: 300 training characters, 2,000 steps, rank 32.
|
| 133 |
+
- Identity transfers much better than style; strongly painterly styles barely transfer.
|
| 134 |
+
- Small patterns, scars, earrings and unusual skin tones are often lost; large garments can drift in color.
|
| 135 |
+
- The reference character occasionally reappears as an object in the scene.
|
| 136 |
+
- Trained only on images generated by Anima; it inherits the base model's biases.
|
| 137 |
+
|
| 138 |
+
Artist names were used only as generation tags. Please respect artists: do not pass off imitations as
|
| 139 |
+
their work, and follow the use restrictions in §4 of [LICENSE.md](LICENSE.md).
|
| 140 |
+
|
| 141 |
+
## License and attribution
|
| 142 |
+
|
| 143 |
+
**Non-commercial.** These weights are a modified derivative of the CircleStone Model (Anima base v1.0),
|
| 144 |
+
distributed under the CircleStone Non-Commercial License ([LICENSE.md](LICENSE.md), [NOTICE.txt](NOTICE.txt)).
|
| 145 |
+
This is an independent project, not an official product of CircleStone Labs and not endorsed by it.
|
| 146 |
|
| 147 |
+
> The CircleStone Model is licensed by CircleStone Labs LLC under the CircleStone Non-Commercial License. Copyright CircleStone Labs LLC. IN NO EVENT SHALL CIRCLESTONE LABS LLC BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.
|
|
|
|
| 148 |
|
| 149 |
## Files
|
| 150 |
|
| 151 |
| file | |
|
| 152 |
|---|---|
|
| 153 |
+
| `anima_ref_adapter_pilot_r32.safetensors` | LoRA, bf16, ComfyUI key format (`diffusion_model.*.lora_down / lora_up / alpha`, 840 tensors) |
|
| 154 |
| `report.md`, `figures/` | technical report |
|
| 155 |
| `workflows/` | ComfyUI workflows |
|
| 156 |
+
| `showcase/` | gallery images |
|
| 157 |
| `LICENSE.md`, `NOTICE.txt` | CircleStone Non-Commercial License and attribution notice |
|
showcase/banner.jpg
ADDED
|
Git LFS Details
|
showcase/girl_beach.jpg
ADDED
|
Git LFS Details
|
showcase/girl_cake.jpg
ADDED
|
Git LFS Details
|
showcase/girl_classroom.jpg
ADDED
|
Git LFS Details
|
showcase/girl_neon.jpg
ADDED
|
Git LFS Details
|
showcase/girl_sakura.jpg
ADDED
|
Git LFS Details
|
showcase/girl_umbrella.jpg
ADDED
|
Git LFS Details
|
showcase/man_beach.jpg
ADDED
|
Git LFS Details
|
showcase/man_cafe.jpg
ADDED
|
Git LFS Details
|
showcase/man_library.jpg
ADDED
|
Git LFS Details
|
showcase/man_night_rain.jpg
ADDED
|
Git LFS Details
|
showcase/man_sakura.jpg
ADDED
|
Git LFS Details
|
showcase/man_snow.jpg
ADDED
|
Git LFS Details
|
showcase/ref_girl.jpg
ADDED
|
Git LFS Details
|
showcase/ref_man.jpg
ADDED
|
Git LFS Details
|