joelthomas77 commited on
Commit
75dd39f
Β·
verified Β·
1 Parent(s): c38762d

Add AnimeLoom organization card

Browse files
Files changed (1) hide show
  1. README.md +129 -0
README.md ADDED
@@ -0,0 +1,129 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: AnimeLoom
3
+ short_description: Open anime character LoRAs and tools for the AnimeLoom text-to-anime-video pipeline.
4
+ ---
5
+
6
+ # AnimeLoom
7
+
8
+ **Open-source character LoRAs and tooling for face-consistent anime video generation.**
9
+
10
+ AnimeLoom is a text-to-anime-video pipeline focused on the hardest part of long-form
11
+ anime generation: **keeping the same character's face the same across every shot.**
12
+
13
+ This organization hosts the character LoRAs used by the
14
+ [AnimeLoom pipeline](https://github.com/JoelJohnsonThomas/AnimeLoom) on GitHub.
15
+
16
+ ---
17
+
18
+ ## Character LoRAs
19
+
20
+ | Character | Series | Repo |
21
+ |-----------|--------|------|
22
+ | Sakura Haruno | Naruto | [AnimeLoom/sakura-haruno](https://huggingface.co/AnimeLoom/sakura-haruno) |
23
+ | Denji | Chainsaw Man | [AnimeLoom/denji](https://huggingface.co/AnimeLoom/denji) |
24
+ | Yuki Nagato | The Melancholy of Haruhi Suzumiya | [AnimeLoom/yuki-nagato](https://huggingface.co/AnimeLoom/yuki-nagato) |
25
+
26
+ Every LoRA ships **two adapters** in one repo:
27
+
28
+ - `sdxl/` β€” for [Animagine XL 3.1](https://huggingface.co/cagliostrolab/animagine-xl-3.1), used by AnimeLoom's identity-keyframe stage
29
+ - `sd15/` β€” for [DreamShaper 8](https://huggingface.co/Lykon/dreamshaper-8), for lightweight AnimateDiff workflows
30
+
31
+ All adapters are LoRA rank 32, trained with [PEFT](https://huggingface.co/docs/peft).
32
+
33
+ ---
34
+
35
+ ## The pipeline
36
+
37
+ AnimeLoom is a Director-Orchestrated pipeline that turns a text story into a
38
+ multi-shot anime video with consistent character identity.
39
+
40
+ ```
41
+ text story
42
+ ↓
43
+ Story Decomposer (Gemini β†’ Claude) β†’ shot script
44
+ ↓
45
+ SDXL + character LoRA + IP-Adapter β†’ identity keyframe per shot (Phase 2)
46
+ ↓
47
+ Wan2.2 I2V β†’ motion driving clip (Phase 3a)
48
+ ↓
49
+ Wan2.2-Animate face-lock β†’ face from keyframe pasted
50
+ onto motion clip (Phase 3b)
51
+ ↓
52
+ RIFE + Real-ESRGAN + GFPGAN β†’ temporal/spatial upscale,
53
+ anime face restore (Phase 4)
54
+ ↓
55
+ Cross-dissolve assembly β†’ final 24fps video
56
+ ```
57
+
58
+ ### Why face-lock matters
59
+
60
+ The "Animate" stage at Phase 3b is the moat. Per
61
+ [community testing](https://apatero.com/blog/wan22-animate-14b-character-animation-guide-2025),
62
+ Wan2.2-Animate-14B achieves an **89% usable rate** versus AnimateDiff's 71% and
63
+ commercial APIs' ~78%. Combined with character-specific LoRAs trained for the
64
+ SDXL identity stage, AnimeLoom can keep a character recognizably "themselves"
65
+ across every shot of a multi-shot scene β€” the part most open-source pipelines
66
+ struggle with.
67
+
68
+ ---
69
+
70
+ ## Quick start
71
+
72
+ Train your own character LoRA, or use one from this org:
73
+
74
+ ```python
75
+ import torch
76
+ from diffusers import StableDiffusionXLPipeline
77
+ from peft import PeftModel
78
+
79
+ pipe = StableDiffusionXLPipeline.from_pretrained(
80
+ "cagliostrolab/animagine-xl-3.1",
81
+ torch_dtype=torch.float16,
82
+ ).to("cuda")
83
+
84
+ # Load any AnimeLoom character LoRA
85
+ pipe.unet = PeftModel.from_pretrained(
86
+ pipe.unet,
87
+ "AnimeLoom/sakura-haruno", # or AnimeLoom/denji, AnimeLoom/yuki-nagato
88
+ subfolder="sdxl",
89
+ )
90
+
91
+ img = pipe(
92
+ "1girl, sakura haruno, pink hair, cherry blossom forest, anime, masterpiece",
93
+ num_inference_steps=28,
94
+ guidance_scale=6.5,
95
+ height=1024, width=1024,
96
+ ).images[0]
97
+ img.save("out.png")
98
+ ```
99
+
100
+ For the full text-to-video pipeline, see the
101
+ [AnimeLoom RunPod notebook](https://github.com/JoelJohnsonThomas/AnimeLoom/blob/main/notebooks/AnimeLoom_RunPod.ipynb).
102
+
103
+ ---
104
+
105
+ ## License
106
+
107
+ All character LoRAs in this org are released under
108
+ [OpenRAIL++](https://huggingface.co/spaces/CompVis/stable-diffusion-license),
109
+ inheriting from their base models (Animagine XL 3.1 / DreamShaper 8).
110
+ The AnimeLoom pipeline code on GitHub is MIT licensed.
111
+
112
+ ---
113
+
114
+ ## Contributing
115
+
116
+ Want to add a character? The training script is in
117
+ [`agents/character/trainer.py`](https://github.com/JoelJohnsonThomas/AnimeLoom/tree/main/agents/character).
118
+ Recommended dataset: 15-30 clean anime reference images of the character at
119
+ 512 or 1024 resolution.
120
+
121
+ Issues, PRs, and new character requests: [GitHub Issues](https://github.com/JoelJohnsonThomas/AnimeLoom/issues).
122
+
123
+ ---
124
+
125
+ ## Links
126
+
127
+ - πŸ™ **GitHub**: [JoelJohnsonThomas/AnimeLoom](https://github.com/JoelJohnsonThomas/AnimeLoom)
128
+ - πŸ“¦ **Pipeline notebook**: [AnimeLoom_RunPod.ipynb](https://github.com/JoelJohnsonThomas/AnimeLoom/blob/main/notebooks/AnimeLoom_RunPod.ipynb)
129
+ - πŸ€— **All models**: [huggingface.co/AnimeLoom](https://huggingface.co/AnimeLoom)