| | --- |
| | license: mit |
| | tags: |
| | - stable diffusion |
| | - animatediff |
| | - motion lora |
| | language: |
| | - en |
| | --- |
| | - [Basic camera motion LoRAs for Animatediff v2 motion models - version 1](#basic-camera-motion-loras-for-animatediff-v2-motion-models---version-1) |
| | - [Model Description](#model-description) |
| | - [File summary](#file-summary) |
| | - [How to use](#how-to-use) |
| | - [General recommendation](#general-recommendation) |
| | - [Details and examples](#details-and-examples) |
| | - [Zooming in (16 frame)](#zooming-in-16-frame) |
| | - [Zooming in (32 frame)](#zooming-in-32-frame) |
| | - [Zooming out](#zooming-out) |
| | - [Lateral (horizontal) left](#lateral-horizontal-left) |
| | - [Lateral (horizontal) right](#lateral-horizontal-right) |
| | - [Crane (vertical) up](#crane-vertical-up) |
| | - [Crane (vertical) down](#crane-vertical-down) |
| |
|
| | # Basic camera motion LoRAs for Animatediff v2 motion models - version 1 |
| |
|
| | ## Model Description |
| | This repository contains a collection of basic camera motion LoRAs finetuned on Animatediff v2 models. The training process utilized Kijai's [AnimateDiff Motion Director ComfyUI nodes](https://github.com/kijai/ComfyUI-ADMotionDirector). These LoRAs have been tested exclusively in ComfyUI with Kosinkadink's [AnimateDiff-Evolved nodes](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved), however they may be compatible with other AnimateDiff implementations. |
| |
|
| | ## File summary |
| |
|
| | | Name | Compatible Motion Model | Base AR (w/h) | Frames | input videos | |
| | | ------------------------------------------------------------- | ----------------------- | ------------- | ------ | ------------ | |
| | | 1600_cseti_9192119_camera-zoomout-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
| | | 1600_cseti_1077723_camera-zoomin-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
| | | 2800_cseti_1093371_camera-zoomin-32f-10vid_mv2.safetensors | v2 | (3:2) | 32 | 10 | |
| | | 1600_cseti_6664682_camera-lateral_right-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
| | | 1600_cseti_9213319_camera-lateral_left-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
| | | 1600_cseti_9406077_camera-crane_down-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
| | | 1600_cseti_1720230_camera-crane_up-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
| |
|
| | ## How to use |
| | - copy the motion lora files into your "ComfyUI\models\animatediff_motion_lora" folder |
| | - Use the "Load Animatediff LoRA" node |
| |
|
| | ## General recommendation |
| | - These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models |
| | - They work with SD 1.5 checkpoints |
| | - Animatediff v3 adapter LoRA is recommended regardless they are v2 models |
| | - If you want more motion try increasing the scale multival |
| | - I recommend using the above Base AR (Aspect ratio) for inference however they might work with other ARs too |
| | - Try playing with the strength of |
| | - motion LoRA |
| | - scale multival |
| | - v3_adapter lora |
| | |
| | like increasing the scale multival and lowering the lora strength. |
| | - I recommend using IP-Adapter, however it reduces the effect of the mLORAs, so play with the strength. "Ease in out" weight type is a good starting point. |
| | - If you chain multiple motion LoRAs, I recommend lowering their strength, otherwise, the quality of the result will be worse |
| | |
| | ## Details and examples |
| | |
| | ### Zooming in (16 frame) |
| | - recommended strength: 0.7-1.0 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-zoomin16.mp4" title="zoomin 16 frames"></video> |
| | |
| | ### Zooming in (32 frame) |
| | - this is an experimental motion LoRA trained on 32 frames so it can be used with context length 32. |
| | - recommended strength: 0.7-0.85 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-zoomin32.mp4" title="zoom in 32 frames"></video> |
| | |
| | ### Zooming out |
| | - recommended strength: 0.7-1.0 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-zoomout16.mp4" title="zoom out"></video> |
| | |
| | ### Lateral (horizontal) left |
| | - recommended strength: 0.7-1.0 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-lateral_left.mp4" title="right view"></video> |
| | |
| | ### Lateral (horizontal) right |
| | - recommended strength: 0.7-1.0 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-lateral_right.mp4" title="right view"></video> |
| | |
| | ### Crane (vertical) up |
| | - recommended strength: 0.7-1.0 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-crane_up.mp4" title="right view"></video> |
| | |
| | ### Crane (vertical) down |
| | - recommended strength: 0.7-1.0 |
| | |
| | <video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-crane_down.mp4" title="right view"></video> |
| | |
| | |
