YuDinging commited on
Commit
c16b901
·
verified ·
1 Parent(s): 0be59bf

Update checkpoint documentation

Browse files
Files changed (1) hide show
  1. README.md +37 -10
README.md CHANGED
@@ -18,17 +18,21 @@ The model uses a dual-stream MoT to jointly model video latents and action token
18
 
19
  - Code: [flyingbitac/FlowPilot](https://github.com/flyingbitac/FlowPilot)
20
 
21
- ## Files
22
 
23
- | File | Size | Description |
 
 
24
  | --- | --- | --- |
25
- | `ckpt/phase2/ckpt_final.pt` | 651 MB | Phase-2 checkpoint. The inference state dict is stored under the `model` key; `step`, `optimizer`, and `lr_scheduler` are also included. |
26
- | `ckpt/action_stats.pt` | 3.1 KB | Action and state normalization statistics. |
27
- | `ckpt/coeff_stats.pt` | 2.7 KB | Bernstein coefficient normalization statistics. |
 
 
28
 
29
- The Wan2.2 VAE weights are not stored in this repository; download the official file from [Wan-AI/Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B) as shown below.
30
 
31
- ## Usage
32
 
33
  Clone the code repository first, then download the checkpoints into its root:
34
 
@@ -44,7 +48,7 @@ If `hf` is not available, install it with `pip install -U huggingface_hub`.
44
 
45
  The `--include "ckpt/*"` filter matters: without it, `hf download` would also write this model card as `README.md` and overwrite the README of the code repository.
46
 
47
- The files land under `ckpt/`, the default location expected by `configs/runtime/rviz.py`:
48
 
49
  ```
50
  ckpt/
@@ -54,17 +58,40 @@ ckpt/
54
  └── coeff_stats.pt
55
  ```
56
 
57
- `hf download` also creates a `.cache/huggingface/` directory in the current directory. It only holds download metadata and can be removed afterwards. If you prefer to keep the weights outside the repository, download to another directory with `--local-dir` and symlink its `ckpt/` into the repository instead.
 
 
 
 
 
 
58
 
 
59
 
 
60
 
61
- Then start the simulation from the code repository:
62
 
63
  ```bash
64
  python docker/docker.py build
 
65
  python docker/docker.py start
66
  ```
67
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  ## Intended Use
69
 
70
  These weights are released for research on learning-based UAV navigation, mainly for simulation and onboard inference experiments. They are not certified for safety-critical flight.
 
18
 
19
  - Code: [flyingbitac/FlowPilot](https://github.com/flyingbitac/FlowPilot)
20
 
21
+ ## Checkpoint roles
22
 
23
+ **Phase 1 is a training intermediate and is not used for evaluation.** Evaluation uses Phase 2 for both Flightmare trajectory collection and future-video prediction. The Phase-1 checkpoint is optional and is not loaded by any evaluation script.
24
+
25
+ | File | Size | Role |
26
  | --- | --- | --- |
27
+ | `ckpt/phase2/ckpt_final.pt` | 651 MB | **Evaluation checkpoint.** Joint video/action model used for control and future-video prediction. The inference state dict is stored under `model`; `step`, `optimizer`, and `lr_scheduler` are also included. |
28
+ | `ckpt/phase1.pt` | 539 MB | Optional video-pretraining intermediate. Not required or used for evaluation. |
29
+ | `ckpt/action_stats.pt` | 3.1 KB | State/goal and action normalization statistics used by Phase 2. |
30
+ | `ckpt/coeff_stats.pt` | 2.7 KB | Bernstein coefficient normalization statistics used by Phase 2. |
31
+ | `ckpt/Wan2.2_VAE.pth` | about 2.8 GB | Depth encoder/decoder used by both phases. Required for evaluation. |
32
 
33
+ Only the Phase-2 checkpoint, Wan2.2 VAE, and normalization statistics are needed for evaluation.
34
 
35
+ ## Download
36
 
37
  Clone the code repository first, then download the checkpoints into its root:
38
 
 
48
 
49
  The `--include "ckpt/*"` filter matters: without it, `hf download` would also write this model card as `README.md` and overwrite the README of the code repository.
50
 
51
+ The evaluation files land under `ckpt/`, the default location expected by `configs/runtime/rviz.py`:
52
 
53
  ```
54
  ckpt/
 
58
  └── coeff_stats.pt
59
  ```
60
 
61
+ `ckpt/phase1.pt` may also be present for training analysis, but can be omitted for evaluation.
62
+
63
+ `hf download` also creates a `.cache/huggingface/` directory in the current directory. It only holds download metadata and can be removed afterwards. Weights can instead be kept outside the repository and symlinked at the paths above. For example:
64
+
65
+ ```bash
66
+ ln -s /path/to/Wan2.2_VAE.pth ckpt/Wan2.2_VAE.pth
67
+ ```
68
 
69
+ `python docker/docker.py start` resolves model symlinks and mounts external targets read-only at the same absolute path inside the container.
70
 
71
+ ## Phase-2 evaluation
72
 
73
+ Build and start the simulation container:
74
 
75
  ```bash
76
  python docker/docker.py build
77
+ export DISPLAY=:1
78
  python docker/docker.py start
79
  ```
80
 
81
+ The standard container exposes all GPUs, allowing Flightmare to use NVIDIA rendering. Inside the container, run the offline collection and future-video comparison without ROS or RViz:
82
+
83
+ ```bash
84
+ python scripts/collect_phase2_trajectory.py
85
+ python scripts/replay_phase2_comparison.py --start-time 0.0
86
+ ```
87
+
88
+ Outputs are written to `runs/phase2_replay/`:
89
+
90
+ - `trajectory.npz`: 30 Hz states, goals, waypoints, and grayscale depth frames; no RGB data.
91
+ - `comparison.png`: two rows and nine columns, with ground-truth depth above Phase-2 predictions.
92
+
93
+ The comparison covers 0 to 1.6 seconds at 0.2-second intervals. The first frame is the observed condition, followed by eight generated future frames. Phase 2 uses the recorded state/goal at the condition frame. Select another condition time with `--start-time`.
94
+
95
  ## Intended Use
96
 
97
  These weights are released for research on learning-based UAV navigation, mainly for simulation and onboard inference experiments. They are not certified for safety-critical flight.