ReMoVe: Refining a Video Model's Own Motion for Coherent Human Video Generation

ReMoVe improves human-video motion coherence by refining the motion of a Wan2.2-TI2V-5B video model with rendered mesh and depth conditioning.

Model composition

This release contains the ReMoVe weights only:

Wan2.2-TI2V-5B base model
+ ReMoVe LoRA adapter
+ ReMoVe expanded patch embedding

The Wan2.2 base model is not included. Download it from the official Wan2.2-TI2V-5B Diffusers model.

Usage

Clone the official ReMoVe code and install the required dependencies:

git clone https://github.com/Lizruletheworld/ReMoVe.git
cd ReMoVe
bash setup/clone_tools.sh
bash setup/install_envs.sh

Download this checkpoint and the official Wan2.2 base model:

bash scripts/download_model_weights.sh --source huggingface

Generate a coarse Wan2.2 video:

python scripts/coarse/generate_video.py \
  --image examples/lunges/head.jpg \
  --prompt "A person stepping one foot forward into a deep lunge, bending both knees, then pushing back to a standing position and repeating." \
  --output examples/lunges/coarse.mp4 \
  --height 704 --width 1280 --num_frames 81 --fps 24

Run ReMoVe:

bash run_pipeline.sh \
  --video examples/lunges/coarse.mp4 \
  --prompt "A person stepping one foot forward into a deep lunge, bending both knees, then pushing back to a standing position and repeating." \
  --head examples/lunges/head.jpg \
  --work outputs/lunges

An existing coarse Wan2.2 video can also be used directly as the input video.