Upload folder using huggingface_hub

README.md

@@ -0,0 +1,37 @@
+# AnimateDiff Model Checkpoints for A1111 SD WebUI
+This repository saves all AnimateDiff models in fp16 & safetensors format for A1111 AnimateDiff users, including
+- motion module (v1-v3)
+- [motion LoRA](#motion-lora) (v2 only, use like any other LoRA)
+- domain adapter (v3 only, use like any other LoRA)
+- [sparse ControlNet](#sparse-controlnet) (v3 only, use like any other ControlNet)
+
+Unless specified below, you are fine to use models from the [official model repository](https://huggingface.co/guoyww/animatediff/tree/main). I will only convert state dict keys if absolutely necessary.
+
+
+## Motion LoRA
+Put Motion LoRAs to `stable-diffusion-webui/models/Lora` and use Motion LoRAs like any other LoRA you use.
+
+[lora_v2](https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main/lora_v2) contains motion LoRAs for AnimateDiff-A1111 v2.0.0 and later. I converted state dict keys inside motion LoRAs. Originlal motion LoRAs won't work for AnimateDiff-A1111 v2.0.0 and later due to maintenance reason.
+
+Use [convert.py](convert.py) in the following way if you want to convert a third-party motion LoRA to be compatible with A1111:
+- Activate your A1111 Python environment first.
+- Command: `python script.py lora [file_path] [save_path]`
+- Replace `[file_path]` with the path to your old LoRA checkpoint.
+- Replace `[save_path]` with the path where you want to save the new LoRA checkpoint.
+
+
+## Sparse ControlNet
+Put Sparse ControlNets to `stable-diffusion-webui/models/ControlNet` and use Sparse ControlNets like any other ControlNet you use.
+
+Like Motion LoRA, I converted state dict keys inside sparse ControlNet. Original sparse ControlNets won't work for A1111 due to maintenance reason.
+
+Use [convert.py](convert.py) in the following way if you want to convert a third-party sparse ControlNet to be compatible with A1111:
+- Activate your A1111 Python environment first.
+- Command: `python script.py controlnet [ad_cn_old] [ad_cn_new] [normal_cn_path]`
+- Replace `[ad_cn_old]` with the path to your old sparse ControlNet checkpoint.
+- Replace `[ad_cn_new]` with the path where you want to save the new sparse ControlNet checkpoint.
+- Replace `[normal_cn_path]` with the path to the normal ControlNet model. Download normal ControlNet from [here](https://huggingface.co/lllyasviel/control_v11p_sd15_openpose/resolve/main/diffusion_pytorch_model.fp16.safetensors?download=true).
+
+
+## HotShotXL
+You need to use HotShotXL in this repository if you want to use it for AnimateDiff-A1111 v2.0.0 and later. Like Motion LoRA, I converted state dict keys inside HotShotXL. Original HotShotXL won't work for AnimateDiff-A1111 v2.0.0 and later due to maintenance reason.

control/mm_sd15_v3_sparsectrl_rgb.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d07d75bd0395fd419477395305b34d4cd23b5d3e7ad5896d73b7fdfe49594d2
+size 993974272

control/mm_sd15_v3_sparsectrl_scribble.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e6451a0f0d1430684e6b895ba13adf147946e553ad568038a4621dfaca1d52b
+size 996119576

convert.py

@@ -0,0 +1,158 @@
+import argparse
+import re
+import torch
+import safetensors.torch
+
+
+def convert_mm_name_to_compvis(key):
+    sd_module_key, _, network_part = re.split(r'(_lora\.)', key)
+    sd_module_key = sd_module_key.replace("processor.", "").replace("to_out", "to_out.0")
+    sd_module_key = sd_module_key.replace(".", "_")
+    return f'{sd_module_key}.lora_{network_part}'
+
+def convert_from_diffuser_state_dict(ad_cn_l):
+    unet_conversion_map = [
+        # (stable-diffusion, HF Diffusers)
+        ("time_embed.0.weight", "time_embedding.linear_1.weight"),
+        ("time_embed.0.bias", "time_embedding.linear_1.bias"),
+        ("time_embed.2.weight", "time_embedding.linear_2.weight"),
+        ("time_embed.2.bias", "time_embedding.linear_2.bias"),
+        ("label_emb.0.0.weight", "add_embedding.linear_1.weight"),
+        ("label_emb.0.0.bias", "add_embedding.linear_1.bias"),
+        ("label_emb.0.2.weight", "add_embedding.linear_2.weight"),
+        ("label_emb.0.2.bias", "add_embedding.linear_2.bias"),
+        ("input_blocks.0.0.weight", "conv_in.weight"),
+        ("input_blocks.0.0.bias", "conv_in.bias"),
+        ("middle_block_out.0.weight", "controlnet_mid_block.weight"),
+        ("middle_block_out.0.bias", "controlnet_mid_block.bias"),
+    ]
+
+    unet_conversion_map_resnet = [
+        # (stable-diffusion, HF Diffusers)
+        ("in_layers.0", "norm1"),
+        ("in_layers.2", "conv1"),
+        ("out_layers.0", "norm2"),
+        ("out_layers.3", "conv2"),
+        ("emb_layers.1", "time_emb_proj"),
+        ("skip_connection", "conv_shortcut"),
+    ]
+
+    unet_conversion_map_layer = []
+    # hardcoded number of downblocks and resnets/attentions...
+    # would need smarter logic for other networks.
+    for i in range(4):
+        # loop over downblocks/upblocks
+
+        for j in range(10):
+            # loop over resnets/attentions for downblocks
+            hf_down_res_prefix = f"down_blocks.{i}.resnets.{j}."
+            sd_down_res_prefix = f"input_blocks.{3*i + j + 1}.0."
+            unet_conversion_map_layer.append((sd_down_res_prefix, hf_down_res_prefix))
+
+            hf_down_atn_prefix = f"down_blocks.{i}.attentions.{j}."
+            sd_down_atn_prefix = f"input_blocks.{3 * i + j + 1}.1."
+            unet_conversion_map_layer.append((sd_down_atn_prefix, hf_down_atn_prefix))
+
+        hf_downsample_prefix = f"down_blocks.{i}.downsamplers.0.conv."
+        sd_downsample_prefix = f"input_blocks.{3 * (i + 1)}.0.op."
+        unet_conversion_map_layer.append((sd_downsample_prefix, hf_downsample_prefix))
+
+
+    hf_mid_atn_prefix = "mid_block.attentions.0."
+    sd_mid_atn_prefix = "middle_block.1."
+    unet_conversion_map_layer.append((sd_mid_atn_prefix, hf_mid_atn_prefix))
+
+    for j in range(2):
+        hf_mid_res_prefix = f"mid_block.resnets.{j}."
+        sd_mid_res_prefix = f"middle_block.{2*j}."
+        unet_conversion_map_layer.append((sd_mid_res_prefix, hf_mid_res_prefix))
+
+    # controlnet specific
+
+    controlnet_cond_embedding_names = ['conv_in'] + [f'blocks.{i}' for i in range(6)] + ['conv_out']
+    for i, hf_prefix in enumerate(controlnet_cond_embedding_names):
+        hf_prefix = f"controlnet_cond_embedding.{hf_prefix}."
+        sd_prefix = f"input_hint_block.{i*2}."
+        unet_conversion_map_layer.append((sd_prefix, hf_prefix))
+
+    for i in range(12):
+        hf_prefix = f"controlnet_down_blocks.{i}."
+        sd_prefix = f"zero_convs.{i}.0."
+        unet_conversion_map_layer.append((sd_prefix, hf_prefix))
+
+
+    def _convert_from_diffuser_state_dict(unet_state_dict):
+        mapping = {k: k for k in unet_state_dict.keys()}
+        for sd_name, hf_name in unet_conversion_map:
+            mapping[hf_name] = sd_name
+        for k, v in mapping.items():
+            if "resnets" in k:
+                for sd_part, hf_part in unet_conversion_map_resnet:
+                    v = v.replace(hf_part, sd_part)
+                mapping[k] = v
+        for k, v in mapping.items():
+            for sd_part, hf_part in unet_conversion_map_layer:
+                v = v.replace(hf_part, sd_part)
+            mapping[k] = v
+        new_state_dict = {v: unet_state_dict[k] for k, v in mapping.items() if k in unet_state_dict}
+        return new_state_dict
+
+    return _convert_from_diffuser_state_dict(ad_cn_l)
+
+
+def lora_conversion(file_path, save_path):
+    state_dict = safetensors.torch.load_file(file_path) if file_path.endswith(".safetensors") else torch.load(file_path)
+    modified_dict = {convert_mm_name_to_compvis(k): v for k, v in state_dict.items()}
+    safetensors.torch.save_file(modified_dict, save_path)
+    print(f"LoRA conversion completed: {save_path}")
+
+
+def controlnet_conversion(ad_cn_old, ad_cn_new, normal_cn_path):
+    ad_cn = safetensors.torch.load_file(ad_cn_old) if ad_cn_old.endswith(".safetensors") else torch.load(ad_cn_old)
+    normal_cn = safetensors.torch.load_file(normal_cn_path)
+    ad_cn_l, ad_cn_m = {}, {}
+    
+    for k in ad_cn.keys():
+        if k.startswith("controlnet_cond_embedding"):
+            new_key = k.replace("controlnet_cond_embedding.", "input_hint_block.0.")
+            ad_cn_m[new_key] = ad_cn[k].to(torch.float16)
+        elif not k in normal_cn:
+            if "motion_modules" in k:
+                ad_cn_m[k] = ad_cn[k].to(torch.float16)
+            else:
+                raise Exception(f"{k} not in normal_cn")
+        else:
+            ad_cn_l[k] = ad_cn[k].to(torch.float16)
+    
+    ad_cn_l = convert_from_diffuser_state_dict(ad_cn_l)
+    ad_cn_l.update(ad_cn_m)
+    safetensors.torch.save_file(ad_cn_l, ad_cn_new)
+    print(f"ControlNet conversion completed: {ad_cn_new}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Script to convert LoRA and ControlNet models.")
+    subparsers = parser.add_subparsers(dest='command')
+
+    # LoRA conversion parser
+    lora_parser = subparsers.add_parser('lora', help='LoRA conversion')
+    lora_parser.add_argument('file_path', type=str, help='Path to the old LoRA checkpoint')
+    lora_parser.add_argument('save_path', type=str, help='Path to save the new LoRA checkpoint')
+
+    # ControlNet conversion parser
+    cn_parser = subparsers.add_parser('controlnet', help='ControlNet conversion')
+    cn_parser.add_argument('ad_cn_old', type=str, help='Path to the old sparse ControlNet checkpoint')
+    cn_parser.add_argument('ad_cn_new', type=str, help='Path to save the new sparse ControlNet checkpoint')
+    cn_parser.add_argument('normal_cn_path', type=str, help='Path to the normal ControlNet model')
+
+    args = parser.parse_args()
+
+    if args.command == 'lora':
+        lora_conversion(args.file_path, args.save_path)
+    elif args.command == 'controlnet':
+        controlnet_conversion(args.ad_cn_old, args.ad_cn_new, args.normal_cn_path)
+    else:
+        parser.print_help()
+
+if __name__ == "__main__":
+    main()

lora/mm_sd15_v2_lora_PanLeft.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6276d00d0398241f5d03862a95fb4002fa251c7238e9c8a284399ee48445d65
+size 38730736

lora/mm_sd15_v2_lora_PanRight.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86e04ae653fbbe2e8287cca41961301c343941c60a2e24ef39ea71da765891cb
+size 38730736

lora/mm_sd15_v2_lora_RollingAnticlockwise.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edf299118571342807d57e4e4c1840c86be26c57b8682b3a7a3a35f7d2a134ee
+size 38730736

lora/mm_sd15_v2_lora_RollingClockwise.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfb12b055b650b859f95dea0cd66b104aed19ddaf65d119346b529d2ab162a02
+size 38730736

lora/mm_sd15_v2_lora_TiltDown.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb0d712627d48906b012f9d9ee41d5963fdb82929f7ac842b8ea29a3fb3b6183
+size 38730736

lora/mm_sd15_v2_lora_TiltUp.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d474baad5434ea5f5eb2125d7b031c642cfab46c2d8643921bbf4e22dcb8d056
+size 38730736

lora/mm_sd15_v2_lora_ZoomIn.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b62ba0b02f7904772002e736c808894026295ed6087a20a2410b340cbb9b444
+size 38730736

lora/mm_sd15_v2_lora_ZoomOut.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a2726d7e464e8bc17e64618f3b5a90845b0202c00fdc4346be1582ba08fd8a0
+size 38730736

lora/mm_sd15_v3_adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78b7fd991066c10ffb24d6e7c89e01438be2c702568a2b5c5bf3a0371b1c854b
+size 51059544

lora_v2/mm_sd15_v2_lora_PanLeft.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c21abede9d32778826d1f26976baed1acf79ae90a3cbfcf35c987ef7c862e77
+size 38727544

lora_v2/mm_sd15_v2_lora_PanRight.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f70336a72ac316f8667cd88d986fd01081ca8fc4b940fd6155df700f9389f897
+size 38727544

lora_v2/mm_sd15_v2_lora_RollingAnticlockwise.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2148bd92f4102dcc05dfeb4b3d4c6e2c9e099d2f7583481cea17041167589f1
+size 38727544

lora_v2/mm_sd15_v2_lora_RollingClockwise.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e855537b7e33438362f156b59693030db437960e12d269e6f89f33d23a05ed45
+size 38727544

lora_v2/mm_sd15_v2_lora_TiltDown.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38f2a7f96ffa612f8a397dda465a267b994379e91f67b0584875200608c6bc13
+size 38727544

lora_v2/mm_sd15_v2_lora_TiltUp.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54f0cb3900af167360b1af934e101a88e00541dee297a5fa629415e834a4bf0f
+size 38727544

lora_v2/mm_sd15_v2_lora_ZoomIn.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cc1467ab86de38073c877b1b1798c96de0dc945dc91df04031313bb20670ef5
+size 38727544

lora_v2/mm_sd15_v2_lora_ZoomOut.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce4ebb8f003ac988bf69db17fe5ef655d72b8512dabb18f07e4355d3b6b8f1ec
+size 38727544

lora_v2/mm_sd15_v3_adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78b7fd991066c10ffb24d6e7c89e01438be2c702568a2b5c5bf3a0371b1c854b
+size 51059544

motion_module/mm_sd14_v1.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc16214feab8b0d8b9e7b05f5a4d9c5e2648601e9c807c51b1b0dc7478879535
+size 836054808

motion_module/mm_sd15_AnimateLCM.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3c5eb97c65d35362e713416e402d54174ccaa0f531f45ad80006e48199bc77d
+size 906505488

motion_module/mm_sd15_v1.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb1858cebf3b0d83e056c7413fc9c43702f24f78c295a4cde1a8ba97f299f7df
+size 836054808

motion_module/mm_sd15_v2.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3976a7dcd11d4f5202d5360c6f1996fc669d6114c530afb3dca4769cc6dd2b2
+size 908929664

motion_module/mm_sd15_v3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8ca0eadb18c6ad6652bca344a458f31780901a87f765bf95a81ef9c5be55f29
+size 836618008

motion_module/mm_sdxl_hs.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8345a99396e5be93e0f90ace4a78ec0bd85d2d7cf63e2aae747fe64288aab7b
+size 474700904

motion_module/mm_sdxl_v10_beta.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:deda6b381d44c3e60d410432fb5be4d7147460948e0edb55a7d5997804e5f6c3
+size 475059304