SAWMAN: Action-conditioned six-view world models

Selected Stage3 EMA checkpoints for Wan2.1 Fun-InP 1.3B and Wan2.2 TI2V 5B. Code and installation instructions.

Directory	Backbone	Checkpoint	Training rollout	Inference
wan1p3b/	Wan2.1 Fun-InP 1.3B	Stage3 iteration 100, EMA	2 steps	2 or 4 steps
wan5b/	Wan2.2 TI2V 5B	Stage3 iteration 100, EMA	4 steps	2 or 4 steps

Two steps are the fast inference preset. The 5B two-step setting is an additional sampling evaluation; its DMD training used four steps. Each selected checkpoint represents 20 generator updates. Selection used the highest mean PSNR among the evaluated Stage3 EMA checkpoints on six representative training examples (one per action). This is not a claim of optimal performance on an independent test set.

Download and use

From the code repository root, after installation:

hf download EmbodiedCity/SAWMAN --local-dir weights/sawman

CUDA_VISIBLE_DEVICES=0 bash scripts/infer_wan1p3b.sh \
  --checkpoint weights/sawman/wan1p3b/diffusion_pytorch_model_ema.safetensors \
  --model-root weights/wan1p3b/base --metadata data/metadata.jsonl \
  --output outputs/eval/wan1p3b_2step --steps 2 --limit 2

CUDA_VISIBLE_DEVICES=0 bash scripts/infer_wan5b.sh \
  --checkpoint weights/sawman/wan5b/diffusion_pytorch_model_ema.safetensors \
  --model-root weights/wan5b/base --metadata data/metadata.jsonl \
  --output outputs/eval/wan5b_2step --steps 2 --limit 2

For one model only, add --include 'wan1p3b/*' or --include 'wan5b/*' to the download command. Supply the corresponding base model files and your own 21-frame six-view videos as described in the code README. The inference helper uses metadata videos for offline evaluation. Future ground-truth frames are not student conditioning. The public EMA serves directly as the inference DiT; separate SAWMAN SFT or Stage2 weights are not required for this inference path.

Format and input contract

Each directory contains a complete BF16 DiT state dictionary in Safetensors format and a portable inference config.json. These are neither LoRA adapters nor standalone Hugging Face Diffusers pipelines. Use the native code entries; DiffusionPipeline.from_pretrained is not the loading interface. Text encoders, tokenizers, VAEs, the 1.3B CLIP encoder and upstream base files are supplied separately. No LightTAE weights or optimizer/resume states are included. manifest.json records file sizes, SHA-256 checksums and tensor counts.

• 21 RGB frames at height 640 × width 960, including the initial observation.
• Fixed 2×3 canvas: [left, front, right] above [up, back, down].
• Actions: move up, move down, move left, move right, move forth, move back.
• Native bidirectional whole-video prediction, not causal blockwise CFPP inference.
• 1.3B uses CLIP plus masked-video conditioning; 5B clamps the observed first latent.
• Student guidance scale 1; sigma shift 5. Playback FPS in config is not generation throughput.

Scope and limitations

These checkpoints support research on six-view aerial prediction. They can show blur, motion underestimation and geometric drift. Higher PSNR does not establish better motion or navigation quality. The release does not include a closed-loop VLM navigation agent or certify navigation benchmark scores for these checkpoints. The checkpoint iteration is retained for provenance; training to a larger iteration did not give the highest score in the available small selection evaluation.

The repository declares Apache-2.0 for these released weights. Refer also to the respective upstream model licenses. Code licensing and third-party attribution are documented in the code repository.