ElectronGirl commited on
Commit
fe8a45a
·
verified ·
1 Parent(s): 07c4c4f

Model card redesign: banner, gallery widget, showcase images

Browse files
.gitattributes CHANGED
@@ -38,3 +38,18 @@ figures/fig3_unseen_artist_oc.jpg filter=lfs diff=lfs merge=lfs -text
38
  figures/fig4_ref_guidance.jpg filter=lfs diff=lfs merge=lfs -text
39
  figures/fig5_corrections.jpg filter=lfs diff=lfs merge=lfs -text
40
  figures/fig6_failures.jpg filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  figures/fig4_ref_guidance.jpg filter=lfs diff=lfs merge=lfs -text
39
  figures/fig5_corrections.jpg filter=lfs diff=lfs merge=lfs -text
40
  figures/fig6_failures.jpg filter=lfs diff=lfs merge=lfs -text
41
+ showcase/banner.jpg filter=lfs diff=lfs merge=lfs -text
42
+ showcase/girl_beach.jpg filter=lfs diff=lfs merge=lfs -text
43
+ showcase/girl_cake.jpg filter=lfs diff=lfs merge=lfs -text
44
+ showcase/girl_classroom.jpg filter=lfs diff=lfs merge=lfs -text
45
+ showcase/girl_neon.jpg filter=lfs diff=lfs merge=lfs -text
46
+ showcase/girl_sakura.jpg filter=lfs diff=lfs merge=lfs -text
47
+ showcase/girl_umbrella.jpg filter=lfs diff=lfs merge=lfs -text
48
+ showcase/man_beach.jpg filter=lfs diff=lfs merge=lfs -text
49
+ showcase/man_cafe.jpg filter=lfs diff=lfs merge=lfs -text
50
+ showcase/man_library.jpg filter=lfs diff=lfs merge=lfs -text
51
+ showcase/man_night_rain.jpg filter=lfs diff=lfs merge=lfs -text
52
+ showcase/man_sakura.jpg filter=lfs diff=lfs merge=lfs -text
53
+ showcase/man_snow.jpg filter=lfs diff=lfs merge=lfs -text
54
+ showcase/ref_girl.jpg filter=lfs diff=lfs merge=lfs -text
55
+ showcase/ref_man.jpg filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -13,75 +13,145 @@ tags:
13
  - character-consistency
14
  - style-transfer
15
  - anime
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ---
17
 
 
 
 
 
18
  # Anima Reference Adapter (pilot, r32)
19
 
20
- A LoRA + inference recipe that lets [Anima](https://huggingface.co/circlestone-labs/Anima) take a
21
- **character and its drawing style from one reference image**, while **pose, framing and background
22
- follow the prompt**. The prompt does not need the character's name, artist tags or appearance tags.
23
 
24
- > **Attribution notice.** The CircleStone Model is licensed by CircleStone Labs LLC under the
25
- > CircleStone Non-Commercial License. Copyright CircleStone Labs LLC. IN NO EVENT SHALL CIRCLESTONE LABS
26
- > LLC BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
27
- > OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.
28
- > **These weights are a modified derivative of the CircleStone Model (Anima base v1.0).** This is an
29
- > independent project, not an official product of CircleStone Labs and not endorsed by it. See
30
- > [NOTICE.txt](NOTICE.txt) and [LICENSE.md](LICENSE.md). Non-commercial use only.
31
 
32
- ![held-out characters](figures/fig2_heldout.jpg)
33
 
34
- *Reference · adapter with a scene-only prompt · same prompt without adapter · image generated with full
35
- character tags. These characters were held out of training.*
 
36
 
37
- ## Use it (ComfyUI)
 
38
 
39
- This is **not a plain LoRA**: the reference must be attached as an extra latent frame with shifted RoPE.
40
- Use the nodes and workflows from **[ElectronGirl/ComfyUI-Anima-Ref-Adapter](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter)**:
41
 
42
- 1. Put `anima_ref_adapter_pilot_r32.safetensors` in `ComfyUI/models/loras/`.
43
- 2. Install the node pack into `ComfyUI/custom_nodes/` and restart.
44
- 3. Open `workflows/anima_ref_adapter.json` (or `_pag.json`), load a reference image, write a
45
- scene-only prompt (quality tags, gender, `solo`, framing, pose, expression, background).
 
 
 
 
 
 
 
 
46
 
47
- Recommended: Anima base v1.0 without the turbo LoRA, bf16 compute, Euler 30 steps, `cfg` 4.5,
48
- `ref_guidance` 4.5–6 with `apg` correction.
49
 
50
- ## Method in one paragraph
51
 
52
- The reference image is letterboxed to the output size, VAE-encoded and appended as a second frame on
53
- the DiT's time axis. It gets timestep 0 (clean) and its RoPE h coordinates are shifted by one target
54
- height, so the video prior cannot copy it pixel-aligned (no pose copying, unlike an aligned temporal
55
- reference). A rank-32 LoRA (45.9M parameters, all attention and MLP projections of the 28 blocks)
56
- learns to read the frame; it is applied only to model calls that carry the reference. Guidance
57
- separates text and reference scales, with adaptive projected guidance against over-saturation.
58
- Full details, data and failure modes: [report.md](report.md).
59
 
60
- ## Training
61
 
62
- | | |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
  |---|---|
64
- | base | Anima base v1.0 (frozen), flow matching, shift 3 |
65
- | data | 3,600 images generated by Anima: 600 groups of one character in one artist style × 6 scenes; 300 characters, 427 artists; exact Danbooru appearance/outfit tags per character; 12 characters held out |
66
- | pairs | 17,280 (reference, target) pairs within a group; target prompt keeps scene tags only (70%), + some appearance tags (20%), or nothing (10%) |
67
- | optimization | 2,000 steps, batch 4, AdamW 1e-4 cosine, bf16, 2.4 h on one RTX PRO 6000 Max-Q |
68
 
69
  ## Limitations
70
 
71
- - Pilot scale. Identity transfer is much stronger than style transfer; painterly styles barely transfer.
72
- - Small patterns, scars, earrings and unusual skin tones are often lost; large garments may drift in color.
73
- - The character occasionally reappears as an object in the scene.
74
- - Samplers with a half-resolution stage (SamplerSPEED) are unreliable.
75
- - Inherits Anima's biases; trained only on synthetic images.
 
 
 
 
 
 
 
 
 
76
 
77
- Artist names were used only as generation tags. Please respect artists: do not use this to pass off
78
- imitations as their work, and follow the license's restrictions on use (§4 of LICENSE.md).
79
 
80
  ## Files
81
 
82
  | file | |
83
  |---|---|
84
- | `anima_ref_adapter_pilot_r32.safetensors` | LoRA (bf16, ComfyUI key format `diffusion_model.*.lora_down/lora_up/alpha`, 840 tensors) |
85
  | `report.md`, `figures/` | technical report |
86
  | `workflows/` | ComfyUI workflows |
 
87
  | `LICENSE.md`, `NOTICE.txt` | CircleStone Non-Commercial License and attribution notice |
 
13
  - character-consistency
14
  - style-transfer
15
  - anime
16
+ widget:
17
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, full body, walking, hands in pockets, city, street, night, rain"
18
+ output:
19
+ url: showcase/man_night_rain.jpg
20
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, upper body, sitting, holding cup, drinking, cafe, indoors, window, sunlight"
21
+ output:
22
+ url: showcase/man_cafe.jpg
23
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, cowboy shot, looking at viewer, smile, beach, ocean, sunset"
24
+ output:
25
+ url: showcase/man_beach.jpg
26
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, portrait, from side, looking afar, cherry blossoms, petals"
27
+ output:
28
+ url: showcase/man_sakura.jpg
29
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, upper body, reading, holding book, library, bookshelf"
30
+ output:
31
+ url: showcase/man_library.jpg
32
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1boy, solo, full body, standing, snow, forest, winter"
33
+ output:
34
+ url: showcase/man_snow.jpg
35
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, full body, walking, smile, city, street, night, neon lights"
36
+ output:
37
+ url: showcase/girl_neon.jpg
38
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, upper body, sitting, eating, cake, cafe, indoors"
39
+ output:
40
+ url: showcase/girl_cake.jpg
41
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, cowboy shot, looking back, smile, beach, ocean, sunset"
42
+ output:
43
+ url: showcase/girl_beach.jpg
44
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, portrait, closed eyes, smile, cherry blossoms, petals, wind"
45
+ output:
46
+ url: showcase/girl_sakura.jpg
47
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, full body, sitting, on chair, crossed legs, classroom, sunlight"
48
+ output:
49
+ url: showcase/girl_classroom.jpg
50
+ - text: "masterpiece, best quality, score_7, newest, highres, absurdres, safe, 1girl, solo, upper body, holding umbrella, rain, street, looking at viewer"
51
+ output:
52
+ url: showcase/girl_umbrella.jpg
53
  ---
54
 
55
+ <div align="center">
56
+
57
+ <img src="showcase/banner.jpg" width="100%" alt="Anima Reference Adapter">
58
+
59
  # Anima Reference Adapter (pilot, r32)
60
 
61
+ **Character and style from one image. Pose, place and framing from your prompt.**
 
 
62
 
63
+ [![Weights](https://img.shields.io/badge/%F0%9F%A4%97%20weights-anima--ref--adapter-ffcc4d)](https://huggingface.co/ElectronGirl/anima-ref-adapter)
64
+ [![Code](https://img.shields.io/badge/%F0%9F%A7%A9%20ComfyUI-nodes-1f6feb)](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter)
65
+ [![Report](https://img.shields.io/badge/%F0%9F%93%84%20report-method%20%26%20results-8a63d2)](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter/blob/main/docs/report.md)
66
+ [![Code license](https://img.shields.io/badge/code-Apache--2.0-2ea44f)](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter/blob/main/LICENSE)
67
+ [![Weights license](https://img.shields.io/badge/weights-CircleStone%20Non--Commercial-555)](https://huggingface.co/ElectronGirl/anima-ref-adapter/blob/main/LICENSE.md)
 
 
68
 
69
+ </div>
70
 
71
+ A LoRA + ComfyUI recipe for [Anima](https://huggingface.co/circlestone-labs/Anima): give one reference
72
+ image and a prompt that only describes the scene. Hair, eyes, outfit and accessories come from the image;
73
+ pose, background and composition follow the prompt. No character name, artist or appearance tags needed.
74
 
75
+ > **This is not a plain LoRA.** The reference must be attached as an extra latent frame with shifted RoPE.
76
+ > Use the nodes and workflows from **[ElectronGirl/ComfyUI-Anima-Ref-Adapter](https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter)**.
77
 
78
+ ## Gallery
 
79
 
80
+ Prompt: `masterpiece, best quality, score_7, newest, highres, absurdres, safe, <gender>, solo, <scene>`. Euler 28 steps, cfg 4.5, reference guidance 4.5 + APG.
81
+ Not cherry-picked.
82
+
83
+ <table>
84
+ <tr><td rowspan="2" align="center" valign="middle" width="25%"><img src="showcase/ref_man.jpg" width="100%"><br><b>Reference A</b><br><sub>input image</sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_night_rain.jpg" width="100%"><br><sub><code>full body, walking, hands in pockets, city, street, night, rain</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_cafe.jpg" width="100%"><br><sub><code>upper body, sitting, holding cup, drinking, cafe, indoors, window, sunlight</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_beach.jpg" width="100%"><br><sub><code>cowboy shot, looking at viewer, smile, beach, ocean, sunset</code></sub></td></tr>
85
+ <tr><td align="center" valign="top" width="25%"><img src="showcase/man_sakura.jpg" width="100%"><br><sub><code>portrait, from side, looking afar, cherry blossoms, petals</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_library.jpg" width="100%"><br><sub><code>upper body, reading, holding book, library, bookshelf</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/man_snow.jpg" width="100%"><br><sub><code>full body, standing, snow, forest, winter</code></sub></td></tr>
86
+ </table>
87
+
88
+ <table>
89
+ <tr><td rowspan="2" align="center" valign="middle" width="25%"><img src="showcase/ref_girl.jpg" width="100%"><br><b>Reference B</b><br><sub>input image</sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_neon.jpg" width="100%"><br><sub><code>full body, walking, smile, city, street, night, neon lights</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_cake.jpg" width="100%"><br><sub><code>upper body, sitting, eating, cake, cafe, indoors</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_beach.jpg" width="100%"><br><sub><code>cowboy shot, looking back, smile, beach, ocean, sunset</code></sub></td></tr>
90
+ <tr><td align="center" valign="top" width="25%"><img src="showcase/girl_sakura.jpg" width="100%"><br><sub><code>portrait, closed eyes, smile, cherry blossoms, petals, wind</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_classroom.jpg" width="100%"><br><sub><code>full body, sitting, on chair, crossed legs, classroom, sunlight</code></sub></td><td align="center" valign="top" width="25%"><img src="showcase/girl_umbrella.jpg" width="100%"><br><sub><code>upper body, holding umbrella, rain, street, looking at viewer</code></sub></td></tr>
91
+ </table>
92
 
93
+ ### Held-out characters
 
94
 
95
+ <img src="figures/fig2_heldout.jpg" width="100%">
96
 
97
+ <sub>reference · adapter with a scene-only prompt · same prompt without the adapter · image generated with the full character tags</sub>
 
 
 
 
 
 
98
 
99
+ ## Quick start (ComfyUI)
100
 
101
+ 1. Put `anima_ref_adapter_pilot_r32.safetensors` in `ComfyUI/models/loras/`.
102
+ 2. `git clone https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter` into `ComfyUI/custom_nodes/`, restart.
103
+ 3. Open `workflows/anima_ref_adapter.json`, load a reference image, write a scene-only prompt.
104
+
105
+ | | recommended | notes |
106
+ |---|---|---|
107
+ | base model | Anima base v1.0 | no turbo LoRA (the adapter was trained on the base model) |
108
+ | compute | bf16 (`ModelComputeDtype`) | Anima overflows in fp16 |
109
+ | sampler | Euler, 28-30 steps, `simple` | single-resolution samplers; SamplerSPEED's half-resolution stage is unreliable |
110
+ | `cfg` | 4.5 | text guidance |
111
+ | `ref_guidance` | 4.5 - 6 | 1 = weak and cheaper; higher = more character detail and style, more saturation |
112
+ | `correction` | `apg` | or `rescale` (phi 0.5) |
113
+ | prompt | quality + gender + `solo` + scene tags | no character name, artist tag or appearance tags |
114
+
115
+ ## How it works
116
+
117
+ The reference image is letterboxed to the output size, VAE-encoded and appended as a **second frame** on
118
+ the DiT's time axis. It is fed **clean** (timestep 0) and its **RoPE h coordinates are shifted by one
119
+ target height**, so the video prior cannot copy it pixel-aligned (no pose copying). A **rank-32 LoRA**
120
+ (45.9M parameters, all attention and MLP projections of the 28 blocks), applied only to calls that carry
121
+ the reference, learns to read it. Guidance separates text and reference scales, with adaptive projected
122
+ guidance against over-saturation. Full details: [report.md](report.md).
123
+
124
+ | training | |
125
  |---|---|
126
+ | data | 3,600 images generated by Anima: 600 groups of one character in one artist style x 6 scenes; 300 characters, 427 artists; 12 characters held out |
127
+ | pairs | 17,280 (reference, target) pairs; target prompt keeps scene tags only (70%), + some appearance tags (20%), or nothing (10%) |
128
+ | optimization | flow matching, 2,000 steps, batch 4, AdamW 1e-4 cosine, bf16; 2.4 h on one RTX PRO 6000 Max-Q |
 
129
 
130
  ## Limitations
131
 
132
+ - **Pilot release**: 300 training characters, 2,000 steps, rank 32.
133
+ - Identity transfers much better than style; strongly painterly styles barely transfer.
134
+ - Small patterns, scars, earrings and unusual skin tones are often lost; large garments can drift in color.
135
+ - The reference character occasionally reappears as an object in the scene.
136
+ - Trained only on images generated by Anima; it inherits the base model's biases.
137
+
138
+ Artist names were used only as generation tags. Please respect artists: do not pass off imitations as
139
+ their work, and follow the use restrictions in §4 of [LICENSE.md](LICENSE.md).
140
+
141
+ ## License and attribution
142
+
143
+ **Non-commercial.** These weights are a modified derivative of the CircleStone Model (Anima base v1.0),
144
+ distributed under the CircleStone Non-Commercial License ([LICENSE.md](LICENSE.md), [NOTICE.txt](NOTICE.txt)).
145
+ This is an independent project, not an official product of CircleStone Labs and not endorsed by it.
146
 
147
+ > The CircleStone Model is licensed by CircleStone Labs LLC under the CircleStone Non-Commercial License. Copyright CircleStone Labs LLC. IN NO EVENT SHALL CIRCLESTONE LABS LLC BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.
 
148
 
149
  ## Files
150
 
151
  | file | |
152
  |---|---|
153
+ | `anima_ref_adapter_pilot_r32.safetensors` | LoRA, bf16, ComfyUI key format (`diffusion_model.*.lora_down / lora_up / alpha`, 840 tensors) |
154
  | `report.md`, `figures/` | technical report |
155
  | `workflows/` | ComfyUI workflows |
156
+ | `showcase/` | gallery images |
157
  | `LICENSE.md`, `NOTICE.txt` | CircleStone Non-Commercial License and attribution notice |
showcase/banner.jpg ADDED

Git LFS Details

  • SHA256: 0c0d34cd5034aca3797c5234bce1f8a2eb6311780170d68c9fa07b64f1722849
  • Pointer size: 131 Bytes
  • Size of remote file: 520 kB
showcase/girl_beach.jpg ADDED

Git LFS Details

  • SHA256: bf3f2e6475a6136e578fa0f965aaddbf4646cc588064d1766feb718775f338b7
  • Pointer size: 131 Bytes
  • Size of remote file: 174 kB
showcase/girl_cake.jpg ADDED

Git LFS Details

  • SHA256: b9f1c17cb0836624a5cba6c886a3aa42b0771be1a3eecb99322e0f3bc01e1938
  • Pointer size: 131 Bytes
  • Size of remote file: 187 kB
showcase/girl_classroom.jpg ADDED

Git LFS Details

  • SHA256: dd4bbc9bab00fee2a13a773f895d3094a476719f2a3981a4d0070225d2b73577
  • Pointer size: 131 Bytes
  • Size of remote file: 110 kB
showcase/girl_neon.jpg ADDED

Git LFS Details

  • SHA256: 9c40fb449ec7a6bed922f24c06884e160d1c5cdbdf60a2c734a6141b72105445
  • Pointer size: 131 Bytes
  • Size of remote file: 125 kB
showcase/girl_sakura.jpg ADDED

Git LFS Details

  • SHA256: ff10f5caf2904195f8b8e42b6bd3aa66ad737c74fba809669e7e3899296bc0be
  • Pointer size: 131 Bytes
  • Size of remote file: 152 kB
showcase/girl_umbrella.jpg ADDED

Git LFS Details

  • SHA256: 099cb75ae83c22072a5ea02f78b838f431f00bd1a1e2df525c97a5dd18159206
  • Pointer size: 131 Bytes
  • Size of remote file: 149 kB
showcase/man_beach.jpg ADDED

Git LFS Details

  • SHA256: 3906084fa6b53fc1bfa4473250832c479347fdd1a67b87fa9c1960611a74b9be
  • Pointer size: 131 Bytes
  • Size of remote file: 175 kB
showcase/man_cafe.jpg ADDED

Git LFS Details

  • SHA256: ab4de573e7d81110219762e6fe65b49bbfd4939826d2c09aa490dca54b756d86
  • Pointer size: 131 Bytes
  • Size of remote file: 143 kB
showcase/man_library.jpg ADDED

Git LFS Details

  • SHA256: 97bdf34b18bb8fde360943a4ab17a3709eee6dc1ee1badfa99078fe674735115
  • Pointer size: 131 Bytes
  • Size of remote file: 188 kB
showcase/man_night_rain.jpg ADDED

Git LFS Details

  • SHA256: 5b57133a86746d7a2a8c3daab935a64c8197622af8c0e39904beec1898b4e3d9
  • Pointer size: 131 Bytes
  • Size of remote file: 143 kB
showcase/man_sakura.jpg ADDED

Git LFS Details

  • SHA256: 6c920658ba2efa8f9f55783467845853b59eda7586a983d7691bb3d899cbccb8
  • Pointer size: 131 Bytes
  • Size of remote file: 136 kB
showcase/man_snow.jpg ADDED

Git LFS Details

  • SHA256: bc2485aacebeaf148cb0a2f0094950ec9b39aad3b6404592b1f647b0a8cdae2e
  • Pointer size: 131 Bytes
  • Size of remote file: 110 kB
showcase/ref_girl.jpg ADDED

Git LFS Details

  • SHA256: ec1960b3a4336de2d51b3522d298c97aef172f50a3d683844ec743264a0d11db
  • Pointer size: 131 Bytes
  • Size of remote file: 164 kB
showcase/ref_man.jpg ADDED

Git LFS Details

  • SHA256: 37146c517be1b78575dadf69e6cf0291167881137600ad65a54607672bd71581
  • Pointer size: 131 Bytes
  • Size of remote file: 143 kB