FlowPilot Checkpoints

Model checkpoints for FlowPilot, a Flow Matching based world-action model for real-time onboard UAV navigation.

The model uses a dual-stream MoT to jointly model video latents and action tokens. Actions are parameterized by a 7th-order Bernstein polynomial with 8 control points (3 fixed by the initial state and 5 predicted by the network), producing 48 waypoints at 30 Hz.

Checkpoint roles

Phase 1 is a training intermediate and is not used for evaluation. Evaluation uses Phase 2 for both Flightmare trajectory collection and future-video prediction. The Phase-1 checkpoint is optional and is not loaded by any evaluation script.

File Size Role
ckpt/phase2/ckpt_final.pt 651 MB Evaluation checkpoint. Joint video/action model used for control and future-video prediction. The inference state dict is stored under model; step, optimizer, and lr_scheduler are also included.
ckpt/phase1.pt 539 MB Optional video-pretraining intermediate. Not required or used for evaluation.
ckpt/action_stats.pt 3.1 KB State/goal and action normalization statistics used by Phase 2.
ckpt/coeff_stats.pt 2.7 KB Bernstein coefficient normalization statistics used by Phase 2.
ckpt/Wan2.2_VAE.pth about 2.8 GB Depth encoder/decoder used by both phases. Required for evaluation.

Only the Phase-2 checkpoint, Wan2.2 VAE, and normalization statistics are needed for evaluation.

Download

Clone the code repository first, then download the checkpoints into its root:

git clone https://github.com/flyingbitac/FlowPilot.git
cd FlowPilot

hf download YuDinging/FlowPilot --include "ckpt/*" --local-dir .
hf download Wan-AI/Wan2.2-TI2V-5B Wan2.2_VAE.pth --local-dir ckpt

If hf is not available, install it with pip install -U huggingface_hub.

The --include "ckpt/*" filter matters: without it, hf download would also write this model card as README.md and overwrite the README of the code repository.

The evaluation files land under ckpt/, the default location expected by configs/runtime/rviz.py:

ckpt/
β”œβ”€β”€ phase2/ckpt_final.pt
β”œβ”€β”€ Wan2.2_VAE.pth
β”œβ”€β”€ action_stats.pt
└── coeff_stats.pt

ckpt/phase1.pt may also be present for training analysis, but can be omitted for evaluation.

hf download also creates a .cache/huggingface/ directory in the current directory. It only holds download metadata and can be removed afterwards. Weights can instead be kept outside the repository and symlinked at the paths above. For example:

ln -s /path/to/Wan2.2_VAE.pth ckpt/Wan2.2_VAE.pth

python docker/docker.py start resolves model symlinks and mounts external targets read-only at the same absolute path inside the container.

Phase-2 evaluation

Build and start the simulation container:

python docker/docker.py build
export DISPLAY=:1
python docker/docker.py start

The standard container exposes all GPUs, allowing Flightmare to use NVIDIA rendering. Inside the container, run the offline collection and future-video comparison without ROS or RViz:

python scripts/collect_phase2_trajectory.py
python scripts/replay_phase2_comparison.py --start-time 0.0

Outputs are written to runs/phase2_replay/:

  • trajectory.npz: 30 Hz states, goals, waypoints, and grayscale depth frames; no RGB data.
  • comparison.png: two rows and nine columns, with ground-truth depth above Phase-2 predictions.

The comparison covers 0 to 1.6 seconds at 0.2-second intervals. The first frame is the observed condition, followed by eight generated future frames. Phase 2 uses the recorded state/goal at the condition frame. Select another condition time with --start-time.

Intended Use

These weights are released for research on learning-based UAV navigation, mainly for simulation and onboard inference experiments. They are not certified for safety-critical flight.

License

The checkpoints are released under the Apache License 2.0.

ckpt/Wan2.2_VAE.pth is downloaded from the official Wan2.2 model repository (Alibaba Wan Team) and is also licensed under Apache-2.0.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading