documentation

README.md

@@ -1 +1,62 @@
-This repository saves all AnimateDiff models in fp16 & safetensors format, including motion modules, motion LoRA, domain adapters and potentially more, for A1111 AnimateDiff users.
+# AnimateDiff Model Checkpoints for A1111 SD WebUI
+This repository saves all AnimateDiff models in fp16 & safetensors format for A1111 AnimateDiff users, including
+- motion module (v1-v3)
+- [motion LoRA](#motion-lora) (v2 only, use like any other LoRAs)
+- domain adapter (v3 only, use like any other LoRAs)
+- [sparse ControlNet](#sparse-controlnet) (v3 only, use like any other ControlNets)
+
+Unless specified below, you are fine to use models from the [official model repository](https://huggingface.co/guoyww/animatediff/tree/main). I will only convert state dict keys if absolutely necessary.
+
+
+## Motion LoRA
+Put motion LoRAs to `stable-diffusion-webui/models/Lora` and use motion LoRAs like any other LoRAs you use.
+
+`lora_v2` contains motion LoRAs for AnimateDiff-A1111 v2.0.0. Old motion LoRAs won't work for v2.0.0 and later due to maintenance reason. `lora` will be removed after AnimateDiff-A1111 v2.0.0 is released to master branch.
+
+I converted the original state dict via the following code. You may do so if you want to use a motion LoRA from community. Run the following script to make your own motion LoRA checkpoint compatible with AnimateDiff-A1111 v2.0.0 and later.
+```python
+import os, re, torch
+import safetensors.torch
+
+def convert_mm_name_to_compvis(key):
+    sd_module_key, _, network_part = re.split(r'(_lora\.)', key)
+    sd_module_key = sd_module_key.replace("processor.", "").replace("to_out", "to_out.0")
+    sd_module_key = sd_module_key.replace(".", "_")
+    return f'{sd_module_key}.lora_{network_part}'
+
+file_path = # replace with path to your own old motion LoRA checkpoint
+save_path = # replace with path to your own new motion LoRA checkpoint
+state_dict = safetensors.torch.load_file(file_path) if file_path.endswith(".safetensors") else torch.load(file_path)
+modified_dict = {convert_mm_name_to_compvis(k): v for k, v in state_dict.items()}
+safetensors.torch.save_file(modified_dict, save_path)
+```
+
+
+## Sparse ControlNet
+Put Sparse ControlNets to `stable-diffusion-webui/models/ControlNet` and use Sparse ControlNets like any other ControlNets you use.
+
+Like Motion LoRA, I also converted state dict keys inside sparse ControlNet. Run the following script to make your own sparse ControlNet checkpoint compatible with AnimateDiff-A1111.
+```python
+import torch
+import safetensors.torch
+ad_cn_old = "v3_sd15_sparsectrl_scribble.ckpt" # replace with path to your own old sparse ControlNet checkpoint
+ad_cn_new = "mm_sd15_v3_sparsectrl_scribble.safetensors" # replace with path to your own new sparse ControlNet checkpoint
+normal_cn_path = "diffusion_pytorch_model.fp16.safetensors" # download https://huggingface.co/lllyasviel/control_v11p_sd15_openpose/resolve/main/diffusion_pytorch_model.fp16.safetensors?download=true and replace with the path to this model
+ad_cn = safetensors.torch.load_file(file_path) if file_path.endswith(".safetensors") else torch.load(ad_cn_old)
+normal_cn = safetensors.torch.load_file(normal_cn_path)
+ad_cn_l, ad_cn_m = {}, {}
+for k in ad_cn.keys():
+    if k.startswith("controlnet_cond_embedding"):
+        new_key = k.replace("controlnet_cond_embedding.", "input_hint_block.0.")
+        ad_cn_m[new_key] = ad_cn[k].to(torch.float16)
+    elif not k in normal_cn:
+        if "motion_modules" in k:
+            ad_cn_m[k] = ad_cn[k].to(torch.float16)
+        else:
+            raise Exception(f"{k} not in normal_cn")
+    else:
+        ad_cn_l[k] = ad_cn[k].to(torch.float16)
+ad_cn_l = convert_from_diffuser_state_dict(ad_cn_l)
+ad_cn_l.update(ad_cn_m)
+safetensors.torch.save_file(ad_cn_l, ad_cn_new)
+```

lora_v2/mm_sd15_v3_adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78b7fd991066c10ffb24d6e7c89e01438be2c702568a2b5c5bf3a0371b1c854b
+size 51059544