byungjun-kim/DWM-CogVideoX-Fun-5b-LoRA

DWM CogVideoX Static-Hand-Concat LoRA

This repository contains a checkpoint-8000 release of the DWM CogVideoX static-scene + hand-video concat model.

It was trained on top of:

• alibaba-pai/CogVideoX-Fun-V1.1-5b-InP

This checkpoint is a derivative of CogVideoX-Fun-V1.1-5b-InP, and its use must comply with the upstream CogVideoX license.

The checkpoint includes:

• pytorch_lora_weights.safetensors: LoRA weights
• non_lora_weights.safetensors: non-LoRA projection weights for patch_embed.proj
• dwm_cogvideox_5b_lora.yaml: training config used for this checkpoint

Usage

This checkpoint is intended to be used with the dwm repository inference script:

python training/cogvideox/inference.py \
  --checkpoint_path /path/to/this_repo \
  --experiment_config /path/to/this_repo/dwm_cogvideox_5b_lora.yaml \
  --data_root /path/to/data \
  --video physics/videos/00001.mp4 \
  --output_dir outputs_infer/dwm_cogvideox_hf

The expected sibling inputs are:

• videos/<stem>.mp4
• videos_static/<stem>.mp4
• videos_hands/<stem>.mp4
• prompts_rewrite/<stem>.txt

Notes

• non_lora_weights.safetensors was converted from the original non_lora_weights.pt checkpoint artifact and stores:
  • patch_embed.proj.weight
  • patch_embed.proj.bias
• Use of this checkpoint should follow both the DWM repository terms and the upstream license terms of CogVideoX-Fun-V1.1-5b-InP.