greenery8848's picture
Upload README.md with huggingface_hub
a2cbe99 verified
|
Raw
History Blame Contribute Delete
1.89 kB
metadata
license: apache-2.0
base_model: Wan-AI/Wan2.2-TI2V-5B-Diffusers
tags:
  - video-generation
  - world-model
  - uav
  - lora
  - pose-conditioned

SkyWander — UAV World Model Checkpoints

Checkpoints for SkyWander, a pose-conditioned UAV video world model built on Wan2.2-TI2V-5B (Apache-2.0). These weights are only meaningful together with the SkyWander inference code (private research repository; to be released with the paper).

Contents

Path Size What
teachers/full_r80_ws72_ddp4/step=20000.ckpt 2.5G teacher LoRA r80, sim-only, GemDepth depth condition (baseline)
teachers/mixdepth_r80_ws72_ddp4/step=20000.ckpt 2.5G teacher LoRA, per-sample mixed depth sources (GemDepth + VGGT conf30/50/70)
teachers/mixreal_r80_ws72_ddp4/step=4000.ckpt 2.5G teacher LoRA, sim+real continuation of the mixdepth teacher
merged/uav_teacher_merged_step20000.pt 9.4G mixdepth teacher LoRA folded into the Wan2.2 base + pose modules (full transformer state dict)
student_stage1/stage1_student_step10000_weights_bf16.pt 10.7G causal-AR student, full-finetune SFT step 10000, weights-only bf16 export (no optimizer state — inference/eval only)
prompt_embeds/prompt_embeds_umt5xxl_len512_*.pt 4×8M cached UMT5-XXL prompt embeddings (skip loading the text encoder at inference)
pva_planner/ckpt_final.pt + norm_stats.json 233M PVA→pose flow-matching planner (15M DiT-1D, EMA included) + required normalization stats

Teacher LoRA checkpoints are PyTorch-Lightning monoliths and require the Wan2.2 base model at load time. The student export contains state_dict with transformer.* and pose-module keys and loads into the SkyWander causal transformer with strict prefix matching.

Trained on the NUS Hopper cluster (H100/H200), 2026.