Update README.md
Browse files
README.md
CHANGED
|
@@ -11,4 +11,62 @@ tags:
|
|
| 11 |
base_model:
|
| 12 |
- circlestone-labs/Anima
|
| 13 |
base_model_relation: quantized
|
| 14 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
base_model:
|
| 12 |
- circlestone-labs/Anima
|
| 13 |
base_model_relation: quantized
|
| 14 |
+
---
|
| 15 |
+
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11
|
| 16 |
+
|
| 17 |
+
Feel free to request for other models for compression as well, although models whose architecture I am unfamiliar with might be slightly tricky for me.
|
| 18 |
+
|
| 19 |
+
### How to Use
|
| 20 |
+
|
| 21 |
+
#### ComfyUI
|
| 22 |
+
Install the ComfyUI DFloat11 Extended node via the ComfyUI manager. After installing, use the provided workflow [json](anima-preview-DF11-workflow.json), or simply replace the "Load Diffusion Model" node of an existing workflow with the "Load Diffusion Model" node. If you run into any issues, feel free to leave a comment. The workflow is also embedded in the below [png](anima-preview-DF11-workflow.png) image.
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
#### `diffusers`
|
| 27 |
+
As far as I know, this model is not implemented in `diffusers` yet.
|
| 28 |
+
|
| 29 |
+
### Compression Details
|
| 30 |
+
|
| 31 |
+
This is the `pattern_dict` for compressing Anima-based models in ComfyUI:
|
| 32 |
+
|
| 33 |
+
```python
|
| 34 |
+
pattern_dict_comfyui = {
|
| 35 |
+
r"t_embedder\.1" : (
|
| 36 |
+
"linear_1",
|
| 37 |
+
"linear_2",
|
| 38 |
+
),
|
| 39 |
+
r"blocks\.\d+" : (
|
| 40 |
+
"self_attn.q_proj",
|
| 41 |
+
"self_attn.k_proj",
|
| 42 |
+
"self_attn.v_proj",
|
| 43 |
+
"self_attn.output_proj",
|
| 44 |
+
"cross_attn.q_proj",
|
| 45 |
+
"cross_attn.k_proj",
|
| 46 |
+
"cross_attn.v_proj",
|
| 47 |
+
"cross_attn.output_proj",
|
| 48 |
+
"mlp.layer1",
|
| 49 |
+
"mlp.layer2",
|
| 50 |
+
"adaln_modulation_self_attn.1",
|
| 51 |
+
"adaln_modulation_self_attn.2",
|
| 52 |
+
"adaln_modulation_cross_attn.1",
|
| 53 |
+
"adaln_modulation_cross_attn.2",
|
| 54 |
+
"adaln_modulation_mlp.1",
|
| 55 |
+
"adaln_modulation_mlp.2",
|
| 56 |
+
),
|
| 57 |
+
r"llm_adapter\.embed": [],
|
| 58 |
+
|
| 59 |
+
r"llm_adapter\.blocks\.\d+" : (
|
| 60 |
+
"self_attn.q_proj",
|
| 61 |
+
"self_attn.k_proj",
|
| 62 |
+
"self_attn.v_proj",
|
| 63 |
+
"self_attn.o_proj",
|
| 64 |
+
"cross_attn.q_proj",
|
| 65 |
+
"cross_attn.k_proj",
|
| 66 |
+
"cross_attn.v_proj",
|
| 67 |
+
"cross_attn.o_proj",
|
| 68 |
+
"mlp.0",
|
| 69 |
+
"mlp.2",
|
| 70 |
+
),
|
| 71 |
+
},
|
| 72 |
+
```
|