Add files using upload-large-folder tool
Browse files- README.md +134 -3
- checkpoints/steps_50000_pytorch_model.pt +3 -0
- dataset_statistics.json +133 -0
README.md
CHANGED
|
@@ -1,3 +1,134 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# World Pilot: Steering Vision-Language-Action Models with World-Action Priors
|
| 2 |
+
|
| 3 |
+
This repository hosts the released **WorldPilot** checkpoint for the **LIBERO** setting from the open-source WorldPilot project.
|
| 4 |
+
|
| 5 |
+
Resources:
|
| 6 |
+
|
| 7 |
+
- Project website: https://world-pilot.github.io/
|
| 8 |
+
- Project code: https://github.com/ZefuLin/WorldPilot
|
| 9 |
+
- Precomputed LIBERO cache: https://huggingface.co/datasets/Chedan86/WorldPilot-LIBERO-precompute
|
| 10 |
+
|
| 11 |
+
## Overview
|
| 12 |
+
|
| 13 |
+
WorldPilot steers a vision-language-action policy with priors from a world-action model. This release provides the public LIBERO checkpoint together with the run artifacts needed to load it from the public WorldPilot codebase.
|
| 14 |
+
|
| 15 |
+
## Included Files
|
| 16 |
+
|
| 17 |
+
```text
|
| 18 |
+
checkpoints/steps_50000_pytorch_model.pt
|
| 19 |
+
config.yaml
|
| 20 |
+
dataset_statistics.json
|
| 21 |
+
run_libero_train.sh
|
| 22 |
+
summary.jsonl
|
| 23 |
+
training.log
|
| 24 |
+
training_log.jsonl
|
| 25 |
+
source_code/Augur/...
|
| 26 |
+
```
|
| 27 |
+
|
| 28 |
+
- `checkpoints/steps_50000_pytorch_model.pt`: released WorldPilot checkpoint.
|
| 29 |
+
- `config.yaml`: saved training configuration for this run.
|
| 30 |
+
- `dataset_statistics.json`: statistic file saved with the WorldPilot run.
|
| 31 |
+
- `run_libero_train.sh`: training launch script snapshot saved with the run.
|
| 32 |
+
- `summary.jsonl`, `training.log`, `training_log.jsonl`: training summaries and logs.
|
| 33 |
+
- `source_code/Augur/...`: source snapshot captured with the run for reference and provenance.
|
| 34 |
+
|
| 35 |
+
## Intended Use
|
| 36 |
+
|
| 37 |
+
This release is intended for:
|
| 38 |
+
|
| 39 |
+
1. loading the released WorldPilot checkpoint with the public WorldPilot codebase;
|
| 40 |
+
2. running public LIBERO-plus evaluation;
|
| 41 |
+
3. inspecting the released configuration and run artifacts.
|
| 42 |
+
|
| 43 |
+
This repository is not a standalone full dependency bundle. Public training and evaluation still require the upstream assets and environments documented in the main project repository.
|
| 44 |
+
|
| 45 |
+
## Using This Checkpoint
|
| 46 |
+
|
| 47 |
+
The public evaluation entrypoints are:
|
| 48 |
+
|
| 49 |
+
- `examples/LIBERO-plus/eval_files/eval_libero_single.sh`
|
| 50 |
+
- `examples/LIBERO-plus/eval_files/eval_libero_batch.sh`
|
| 51 |
+
|
| 52 |
+
Set:
|
| 53 |
+
|
| 54 |
+
```bash
|
| 55 |
+
WORLDPILOT_CKPT=/path/to/WorldPilot-LIBERO/checkpoints/steps_50000_pytorch_model.pt
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
The public evaluation scripts also require:
|
| 59 |
+
|
| 60 |
+
- a Cosmos Policy checkpoint;
|
| 61 |
+
- a Cosmos dataset statistics JSON;
|
| 62 |
+
- a Cosmos T5 embeddings pickle;
|
| 63 |
+
- a valid `~/.libero-plus/config.yaml`.
|
| 64 |
+
|
| 65 |
+
If you want the evaluation script to use the bundled source snapshot, set:
|
| 66 |
+
|
| 67 |
+
```bash
|
| 68 |
+
USE_CHECKPOINT_SOURCE=true
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
The script will then add `source_code/` from this release to `PYTHONPATH`.
|
| 72 |
+
|
| 73 |
+
Important:
|
| 74 |
+
|
| 75 |
+
- `dataset_statistics.json` in this repository is the statistic file saved with the WorldPilot run.
|
| 76 |
+
- It does not replace the `COSMOS_STATS` file required by the Cosmos server in the evaluation scripts.
|
| 77 |
+
|
| 78 |
+
For the full setup, follow the public project documentation:
|
| 79 |
+
|
| 80 |
+
- https://world-pilot.github.io/
|
| 81 |
+
- https://github.com/ZefuLin/WorldPilot/tree/main/doc
|
| 82 |
+
|
| 83 |
+
## Training Reference
|
| 84 |
+
|
| 85 |
+
The public training path in the code release uses:
|
| 86 |
+
|
| 87 |
+
- `examples/LIBERO/train_files/WorldPilot.yaml`
|
| 88 |
+
- `examples/LIBERO/train_files/run_libero_train.sh`
|
| 89 |
+
|
| 90 |
+
Training also depends on the upstream assets listed in the public docs, including:
|
| 91 |
+
|
| 92 |
+
- `nvidia/Cosmos-Policy-LIBERO-Predict2-2B`
|
| 93 |
+
- `facebook/VGGT-1B`
|
| 94 |
+
- `StarVLA/Qwen3-VL-4B-Instruct-Action`
|
| 95 |
+
- `amap_cvlab/ABot-M0-Pretrain`
|
| 96 |
+
|
| 97 |
+
Training reads precomputed Cosmos cache from:
|
| 98 |
+
|
| 99 |
+
```text
|
| 100 |
+
datasets.vla_data.cosmos_cache_dir
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
You can reuse the published cache here:
|
| 104 |
+
|
| 105 |
+
- https://huggingface.co/datasets/Chedan86/WorldPilot-LIBERO-precompute
|
| 106 |
+
|
| 107 |
+
The included `config.yaml` and `run_libero_train.sh` are run artifacts for inspection and reproduction support. For active training, use the checked-out public repository and its current training docs.
|
| 108 |
+
|
| 109 |
+
## Provenance and Terms
|
| 110 |
+
|
| 111 |
+
The main WorldPilot code repository is released under Apache-2.0:
|
| 112 |
+
|
| 113 |
+
- https://github.com/ZefuLin/WorldPilot
|
| 114 |
+
|
| 115 |
+
This release also depends on upstream projects and assets that keep their own licenses and usage terms. Upstream checkpoints, datasets, and third-party components are not relicensed by this model card.
|
| 116 |
+
|
| 117 |
+
The bundled `source_code/` snapshot is included for reference and provenance only. It does not supersede original upstream notices or license terms.
|
| 118 |
+
|
| 119 |
+
## Citation
|
| 120 |
+
|
| 121 |
+
If WorldPilot helps your research, please cite:
|
| 122 |
+
|
| 123 |
+
```bibtex
|
| 124 |
+
@article{worldpilot2026,
|
| 125 |
+
title={World Pilot: Steering Vision-Language-Action Models with World-Action Priors},
|
| 126 |
+
author={Anonymous},
|
| 127 |
+
journal={arXiv preprint arXiv:xxxx.xxxxx},
|
| 128 |
+
year={2026}
|
| 129 |
+
}
|
| 130 |
+
```
|
| 131 |
+
|
| 132 |
+
## Acknowledgements
|
| 133 |
+
|
| 134 |
+
We thank the teams behind ABot-Manipulation, cosmos-policy, LIBERO, LIBERO-plus, and LeRobot.
|
checkpoints/steps_50000_pytorch_model.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c7fbed14a2c239fc830748d568f4d7452c288c1c55c7dee84337ff0986ba5ba9
|
| 3 |
+
size 12692203808
|
dataset_statistics.json
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"franka": {
|
| 3 |
+
"action": {
|
| 4 |
+
"mean": [
|
| 5 |
+
0.06413520714984491,
|
| 6 |
+
0.08330044630341807,
|
| 7 |
+
-0.09768853995662471,
|
| 8 |
+
3.989756358071042e-05,
|
| 9 |
+
0.0074605874788875775,
|
| 10 |
+
-0.0023408993505514594,
|
| 11 |
+
0.5327441073380985
|
| 12 |
+
],
|
| 13 |
+
"std": [
|
| 14 |
+
0.34435420775153375,
|
| 15 |
+
0.37664413619011156,
|
| 16 |
+
0.4530659887664812,
|
| 17 |
+
0.040994048642083246,
|
| 18 |
+
0.06561666470959421,
|
| 19 |
+
0.08020121465404138,
|
| 20 |
+
0.4990030574544087
|
| 21 |
+
],
|
| 22 |
+
"max": [
|
| 23 |
+
0.9375,
|
| 24 |
+
0.9375,
|
| 25 |
+
0.9375,
|
| 26 |
+
0.3557142913341522,
|
| 27 |
+
0.375,
|
| 28 |
+
0.375,
|
| 29 |
+
1.0
|
| 30 |
+
],
|
| 31 |
+
"min": [
|
| 32 |
+
-0.9375,
|
| 33 |
+
-0.9375,
|
| 34 |
+
-0.9375,
|
| 35 |
+
-0.2582142949104309,
|
| 36 |
+
-0.375,
|
| 37 |
+
-0.3675000071525574,
|
| 38 |
+
0.0
|
| 39 |
+
],
|
| 40 |
+
"q01": [
|
| 41 |
+
-0.8785714507102966,
|
| 42 |
+
-0.8758928775787354,
|
| 43 |
+
-0.9375,
|
| 44 |
+
-0.1510714292526245,
|
| 45 |
+
-0.20678570866584778,
|
| 46 |
+
-0.2742857038974762,
|
| 47 |
+
0.0
|
| 48 |
+
],
|
| 49 |
+
"q99": [
|
| 50 |
+
0.9375,
|
| 51 |
+
0.9107142686843872,
|
| 52 |
+
0.9375,
|
| 53 |
+
0.20357142388820648,
|
| 54 |
+
0.26357144117355347,
|
| 55 |
+
0.375,
|
| 56 |
+
1.0
|
| 57 |
+
],
|
| 58 |
+
"mask": [
|
| 59 |
+
true,
|
| 60 |
+
true,
|
| 61 |
+
true,
|
| 62 |
+
true,
|
| 63 |
+
true,
|
| 64 |
+
true,
|
| 65 |
+
false
|
| 66 |
+
]
|
| 67 |
+
},
|
| 68 |
+
"state": {
|
| 69 |
+
"mean": [
|
| 70 |
+
-0.051236557416044756,
|
| 71 |
+
0.03320404933765531,
|
| 72 |
+
0.7923003721695681,
|
| 73 |
+
2.9629008036393385,
|
| 74 |
+
-0.15774205785531267,
|
| 75 |
+
-0.13519181454410922,
|
| 76 |
+
0.02689367747650697,
|
| 77 |
+
-0.027077807328448854
|
| 78 |
+
],
|
| 79 |
+
"std": [
|
| 80 |
+
0.10707255982690211,
|
| 81 |
+
0.14929271152454146,
|
| 82 |
+
0.37628407735102676,
|
| 83 |
+
0.3693053742611154,
|
| 84 |
+
0.8810664091949664,
|
| 85 |
+
0.33025891946901687,
|
| 86 |
+
0.01434538622885222,
|
| 87 |
+
0.01422763564543161
|
| 88 |
+
],
|
| 89 |
+
"max": [
|
| 90 |
+
0.21031762659549713,
|
| 91 |
+
0.39128610491752625,
|
| 92 |
+
1.3660105466842651,
|
| 93 |
+
3.6714255809783936,
|
| 94 |
+
3.560650587081909,
|
| 95 |
+
1.386339545249939,
|
| 96 |
+
0.04233968257904053,
|
| 97 |
+
0.0013633022317662835
|
| 98 |
+
],
|
| 99 |
+
"min": [
|
| 100 |
+
-0.4828203022480011,
|
| 101 |
+
-0.3255046010017395,
|
| 102 |
+
0.008128180168569088,
|
| 103 |
+
0.35277295112609863,
|
| 104 |
+
-3.641430377960205,
|
| 105 |
+
-1.842738389968872,
|
| 106 |
+
-0.0013586411951109767,
|
| 107 |
+
-0.042040832340717316
|
| 108 |
+
],
|
| 109 |
+
"q01": [
|
| 110 |
+
-0.42401049643754957,
|
| 111 |
+
-0.2838300323486328,
|
| 112 |
+
0.009925739830359817,
|
| 113 |
+
1.3085840785503386,
|
| 114 |
+
-2.886677579879761,
|
| 115 |
+
-1.1599004411697387,
|
| 116 |
+
0.001503719249740243,
|
| 117 |
+
-0.040336399003863335
|
| 118 |
+
],
|
| 119 |
+
"q99": [
|
| 120 |
+
0.1530261474847791,
|
| 121 |
+
0.3629165390133857,
|
| 122 |
+
1.2910678112506866,
|
| 123 |
+
3.303542451858519,
|
| 124 |
+
2.7496529006957933,
|
| 125 |
+
0.6893712210655194,
|
| 126 |
+
0.040610933862626555,
|
| 127 |
+
-0.0015016929572448147
|
| 128 |
+
]
|
| 129 |
+
},
|
| 130 |
+
"num_transitions": 272104,
|
| 131 |
+
"num_trajectories": 1693
|
| 132 |
+
}
|
| 133 |
+
}
|