Document and enable the complete collect-update-world-model-GRPO cycle

Adds portable eight-GPU collection of 500 paired trajectories, Cosmos-first concurrent Wan/Cosmos updates, explicit model-pair selection and policy60-to80 continuation. Includes 18 CPU tests and independent restored-runtime import/dry-run evidence.

README.md

@@ -26,7 +26,18 @@ base_model:
 10 个任务 × 50 次评估成功率为 **77.2%（386/500）**。
 `latest_policy.json` 记录最新断点、评估结果和续训入口。40、45、50 步用于历史对比。
 
-下载并执行下文的 `restore`、`check` 后，从 60 步继续训练 20 步，到总步数 80：
+下载并执行下文的 `restore`、`check` 后，完整的下一轮实验使用以下入口：
+
+```bash
+screen -dmS spatial-cycle60 bash -c \
+  'bash reproduction/run_world_model_cycle.sh "$PWD/runs/cycle60"'
+screen -r spatial-cycle60
+```
+
+它依次执行：60 步策略采集 500 条真实双相机轨迹 → 更新 Wan/Cosmos →
+打包本轮新模型 → 策略从 60 步继续到 80 步。完整配方见“后续世界模型更新”。
+
+以下单条命令只续训 GRPO，不采集数据或更新世界模型；使用包内 Wan 1000/Cosmos 2000：
 
 ```bash
 screen -dmS spatial-grpo60-to80 bash -c \
@@ -138,10 +149,54 @@ python3 reproduction/portable.py eval --checkpoint 60 --output "$PWD/runs/eval60
 
 ## 后续世界模型更新
 
-源码中保留了采集与更新工具。当前约定为：Cosmos 完成目标步数后，Wan 在下一个
-完整更新处保存并结束，然后用最新双世界模型继续 GRPO。无需让 Wan 补到相同步数，
-视频指标另行安排。使用 `experiments/wm_update_until_cosmos.py` 作为新的并行训练
-入口，读取 `selected_world_models.json` 来打包实际选定的模型。
+完整入口为 `reproduction/run_world_model_cycle.sh`，从恢复好的发布包根目录执行，
+需要八张可用 GPU。每个原生入口启动前都会检查占用；指定的卡有其他作业时退出。
+采集和世界模型训练均使用实际计算进程持有的可复用显存缓存，保留 3.5 GiB 设备余量。
+
+先查看命令计划，不启动 GPU 任务：
+
+```bash
+bash reproduction/run_world_model_cycle.sh --dry-run "$PWD/runs/cycle60"
+```
+
+正式执行上面的 `screen -dmS spatial-cycle60 ...` 命令。各阶段为：
+
+1. **采集真实数据**：八卡使用 `weights/policy/global_step_60/`，在 LIBERO-Spatial
+   的 10 个任务上各采集 50 条轨迹。保留成功和失败，`mode=train`，每条最多 240 步，
+   每次执行 8 个动作；主相机、腕部相机、动作和仿真状态按同一帧索引保存。
+   结果在 `runs/cycle60/collection/`。
+2. **准备训练集**：新采集的 500 条轨迹与包内 `data/kir_sources/` 的 500 条公开
+   Spatial 轨迹组成新数据/回放数据，验证比例 0.2；训练采样的新数据比例为 0.5。
+   生成 `runs/cycle60/manifest.json` 并校验文件。历史每轮采集的原始数据未随包发布；
+   此处回放来源是包内公开轨迹。
+3. **更新两个世界模型**：Wan/Cosmos 各四卡并行，梯度累计 4 次，有效 batch 各为 16，
+   学习率 `1e-6`，Cosmos 使用 `v3-reconstructed` 配方，目标 2000 步，
+   显式设置 `--save-interval 1000`。从包内 Wan 1000/Cosmos 2000 的模型参数初始化，
+   为本轮数据建立新的世界模型优化器；历史世界模型优化器状态不在发布包中。
+4. **选择并打包新模型**：使用 `wm_update_until_cosmos.py` 控制两个训练组。
+   Cosmos 完成 2000 步后，请求 Wan 在下一个完整优化器更新后保存并结束，
+   不强制 Wan 达到 2000 步。读取 `train/selected_world_models.json`，打包生成
+   `runs/cycle60/model_pair/pair.json`。视频指标可另行评估，不阻塞接续。
+5. **继续 GRPO**：恢复第 60 步策略的完整 DCP 和优化器状态，用本轮新模型训练至
+   总步数 80；第 70、80 步保存完整策略训练状态，每 5 步保存策略并做真实 Spatial
+   10×50 评估。结果在 `runs/cycle60/grpo/`。
+
+`reproduction/world_model.py` 为原生采集和更新工具设置本地恢复后的源码、解释器、
+模型和数据路径。也可分阶段执行；例如本轮世界模型已训练完：
+
+```bash
+python3 reproduction/world_model.py package \
+  --selection "$PWD/runs/cycle60/train/selected_world_models.json" \
+  --output "$PWD/runs/cycle60/model_pair"
+
+python3 reproduction/portable.py train --resume 60 --target 80 \
+  --model-pair "$PWD/runs/cycle60/model_pair/pair.json" \
+  --output "$PWD/runs/cycle60/grpo"
+```
+
+`--model-pair` 会同时切换 Wan 和 Cosmos，并复用冻结的 VAE、奖励模型和初始观测资源。
+不用该参数时，继续使用发布包内的世界模型。完整入口使用种子 `20261021`；
+通过 `WM_CYCLE_GPUS`、`WM_CYCLE_SEED` 可调整物理卡号和本轮随机种子。
 
 原服务器的邮件配置、token、登录缓存、Ray 临时目录、活动进程和未完成检查点均
 不属于发布包。新服务器需要邮件通知时，另行配置现有 `watch_training_email.py`。

SHA256SUMS

@@ -10,7 +10,7 @@ b8817d9f58408022ff30bbf1abd39521e7e291c83b33ff04969c45ac46bd6c65  LICENSES/PI05-
 c7c92fee190b604e2d18e27df517aca77871d0c6829d1a58b71521807d3acdb1  LICENSES/README.md
 4bcf87ecfbbb8e07a01b21415a970c8b53a5283bf6872b657040d3f45c9241f7  LICENSES/WAN-SPATIAL-MODEL-CARD.md
 9491f7a2676a19460f8e9d3c625dabd53841fa8fe3db92bf6cd73780dbf0348e  NOTICE
-a233ff8ba459e8daa3ab72c2c2061c6cb2f9a38f682d9cd0be7840b881b59e0b  README.md
+398a6b9b5e1dd0a85b326e5e7c9e35370ed8818e41c5ebdd1d27c1e046a92a2e  README.md
 5b119a98ad7824507ddff3c6c7507ca244261ee416dcfa623876702160c580d3  assets/cosmos/libero_dataset_statistics.json
 8a03499676c6c196127c577144b9fd09bb02c30f9caf058adbcef09bb99ad8f5  assets/cosmos/libero_t5_embeddings.pkl
 38071ab59bd94681c686fa51d75a1968f64e470262043be31f7a094e442fd981  assets/cosmos/tokenizer.pth
@@ -31,13 +31,16 @@ d9c2426cd1209ca943b280c3ed217d410cd571e3085fc3945b1a84befea911a2  evidence/evalu
 54128b9e3c050b89b1582cd6fac382543c037096fb41d13dcad34cfb8c392e09  evidence/evaluations/global_step_50.json
 06b80579aa0beae3bf11400d3734e6b5ac9e19bb9fab8c7603ad0123b03a27c2  evidence/evaluations/global_step_60.json
 cc963ddd48281aece1af1a1ccafde791879340728d117f0d692f1136ee289e59  evidence/policy_step60_resume.json
+caf6b507e714451c894f14e3b4b0a6a889cb63c839b1cf6024333a53363c2169  evidence/reproduction_cycle60.json
 fdc3869dc035263b05b2c4ff9b2f6826282e3a6cc3f4cd268f43cd9a329d2159  evidence/rlinf_environment.json
 b1d0dca8515f82edbd9ffeb1b33f9023e24f23c68a1a644faf23caefedf7c8d1  evidence/world_model_training_manifest.json
-f436503a0dd25df2dc327a73c54dd776945cf3d952c30875942e72032d99492d  latest_policy.json
-1a62781c9104b8dfb0a9d2fd869da6ba0886e6fcf4967dc566090bb4873532c5  reproduction/README.zh-CN.md
+ff3818dca101b19d4bf7f7cf146c93fa58d399ea6babe0b16f150b3563064e80  latest_policy.json
+78e03d367bb5d43544ff1b7e672d3eb7a31d2c6e0131399850e40a66b08b9a9d  reproduction/README.zh-CN.md
 f490f560a262e9402230bb0e7f1f965853dc24c4c72b4d6b3941293c40802e0b  reproduction/eval_train.py
 8d0c9116b5b8624aa8dba4f13b76206b7d326a44aa6a9c2b627076c8266b4582  reproduction/launcher.py
-b6418fea3feeab7d912f0f84c0a9866cade77c000ec0b574d02fe97295a0230f  reproduction/portable.py
+18b7d75582005b89b0b59ac705aad2ece7c61b47152d058794c02f07f10197c4  reproduction/portable.py
+6827405385e2a143d0b593cf630865ba6b5c1ed5cd6bac94db5e38bc71c65695  reproduction/run_world_model_cycle.sh
+9ae94e1f81f214044aa270223c5a4ddee318908627042f17e4186eb6f29fa157  reproduction/world_model.py
 936193ae5fa7432637b050654633496a62455573abd0224dcfb8e118aceb8f39  runtime/cosmos_packages_0.tar.zst
 bbac3dee6879dc41b12082e07400aa6adac52bbf1c4dc8728e53bf8df20ef709  runtime/cosmos_packages_1.tar.zst
 c4940b5a72db60e1de3eb94534034d4d96be53714acb11044f910c0a8c24f1b2  runtime/python310.tar.zst
@@ -47,6 +50,7 @@ a659145d90481d3028e12cc780dead4d8340b64b8e9b692bbec7a3185f359719  runtime/rlinf_
 19114ac59362c80778ca751a7ae8635717ffb64d13c81ab0451c420c14b29bc9  runtime/rlinf_packages_1.tar.zst
 574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175  source/grpo_spatial_source.tar.zst
 08e1648ba07ce8429bc2faee49fe9a5517fde7d81eae94441e91a07872c91319  source/step60_reproduction_update.tar.zst
+6825a0fdfa9693513de7df0c5d49fdbdb9065b3db4b1d4928bff1840a734e2f7  source/world_model_cycle_update.tar.zst
 b4a01639bdac0e98073499bbb3a84cd29b2274dba80fa286f0c7ee27bf75b949  weights/cosmos_step2000/checkpoint_manifest.json
 7110ea589cfb2dabf4c90941cf5fc5c5d23fdf47ab78b7a16b56b5014e7a87bf  weights/cosmos_step2000/iter_000002000/model/.metadata
 01f1eb9e46b123df765f614a18835e86393c3bdf7d6147b7be4453bc551b99e4  weights/cosmos_step2000/iter_000002000/model/__0_0.distcp

evidence/reproduction_cycle60.json

@@ -0,0 +1,50 @@
+{
+  "schema": "spatial-reproduction-cycle-v1",
+  "policy_resume_step": 60,
+  "collect_episodes": 500,
+  "tasks": 10,
+  "episodes_per_task": 50,
+  "max_steps": 240,
+  "collection_mode": "train",
+  "seed": 20261021,
+  "replay_source": "data/kir_sources",
+  "replay_episodes": 500,
+  "historical_collected_replay_included": false,
+  "validation_fraction": 0.2,
+  "new_data_sampling_fraction": 0.5,
+  "wan_ranks": 4,
+  "cosmos_ranks": 4,
+  "accumulation": 4,
+  "effective_batch": 16,
+  "learning_rate": 1e-06,
+  "cosmos_steps": 2000,
+  "save_interval": 1000,
+  "cosmos_recipe": "v3-reconstructed",
+  "wan_stop": "next complete update after Cosmos finishes",
+  "reserve_memory": true,
+  "memory_headroom_gib": 3.5,
+  "model_initializers": {
+    "wan_step": 1000,
+    "cosmos_step": 2000
+  },
+  "world_model_optimizer": "new optimizer for the new round",
+  "policy_optimizer": "resume complete step60 DCP",
+  "grpo_target_step": 80,
+  "paired_model_selection": "train/selected_world_models.json",
+  "entry": "bash reproduction/run_world_model_cycle.sh \"$PWD/runs/cycle60\"",
+  "validation": {
+    "status": "passed",
+    "tests_passed": 18,
+    "cpu_only": true,
+    "independent_runtime_check": true,
+    "wan_and_cosmos_imports": true,
+    "native_training_dry_run": true,
+    "ranks_per_model": 4,
+    "effective_batch": 16,
+    "reserve_memory": true,
+    "save_interval": 1000,
+    "workflow_stages": 7,
+    "documentation_bash_blocks": 10,
+    "gpu_collection_or_training_started": false
+  }
+}

latest_policy.json

@@ -20,5 +20,9 @@
   },
   "completed_utc": "2026-10-25T13:18:28Z",
   "recommended_resume_step": 60,
-  "continuation_example": "python3 reproduction/portable.py train --resume 60 --target 80 --output \"$PWD/runs/resume60_to80\""
+  "continuation_example": "bash reproduction/run_world_model_cycle.sh \"$PWD/runs/cycle60\"",
+  "direct_grpo_example": "python3 reproduction/portable.py train --resume 60 --target 80 --output \"$PWD/runs/resume60_to80\"",
+  "direct_grpo_updates_world_models": false,
+  "recommended_workflow": "collect 500 episodes, update Wan/Cosmos, then resume policy60 to80",
+  "workflow_evidence": "evidence/reproduction_cycle60.json"
 }

release_manifest.json

@@ -67,8 +67,8 @@
     },
     {
       "path": "README.md",
-      "size": 7625,
-      "sha256": "a233ff8ba459e8daa3ab72c2c2061c6cb2f9a38f682d9cd0be7840b881b59e0b"
+      "size": 10813,
+      "sha256": "398a6b9b5e1dd0a85b326e5e7c9e35370ed8818e41c5ebdd1d27c1e046a92a2e"
     },
     {
       "path": "assets/cosmos/libero_dataset_statistics.json",
@@ -170,6 +170,11 @@
       "size": 453,
       "sha256": "cc963ddd48281aece1af1a1ccafde791879340728d117f0d692f1136ee289e59"
     },
+    {
+      "path": "evidence/reproduction_cycle60.json",
+      "size": 1495,
+      "sha256": "caf6b507e714451c894f14e3b4b0a6a889cb63c839b1cf6024333a53363c2169"
+    },
     {
       "path": "evidence/rlinf_environment.json",
       "size": 22254,
@@ -182,13 +187,13 @@
     },
     {
       "path": "latest_policy.json",
-      "size": 722,
-      "sha256": "f436503a0dd25df2dc327a73c54dd776945cf3d952c30875942e72032d99492d"
+      "size": 1017,
+      "sha256": "ff3818dca101b19d4bf7f7cf146c93fa58d399ea6babe0b16f150b3563064e80"
     },
     {
       "path": "reproduction/README.zh-CN.md",
-      "size": 7253,
-      "sha256": "1a62781c9104b8dfb0a9d2fd869da6ba0886e6fcf4967dc566090bb4873532c5"
+      "size": 10441,
+      "sha256": "78e03d367bb5d43544ff1b7e672d3eb7a31d2c6e0131399850e40a66b08b9a9d"
     },
     {
       "path": "reproduction/eval_train.py",
@@ -202,8 +207,18 @@
     },
     {
       "path": "reproduction/portable.py",
-      "size": 15007,
-      "sha256": "b6418fea3feeab7d912f0f84c0a9866cade77c000ec0b574d02fe97295a0230f"
+      "size": 16580,
+      "sha256": "18b7d75582005b89b0b59ac705aad2ece7c61b47152d058794c02f07f10197c4"
+    },
+    {
+      "path": "reproduction/run_world_model_cycle.sh",
+      "size": 2817,
+      "sha256": "6827405385e2a143d0b593cf630865ba6b5c1ed5cd6bac94db5e38bc71c65695"
+    },
+    {
+      "path": "reproduction/world_model.py",
+      "size": 1859,
+      "sha256": "9ae94e1f81f214044aa270223c5a4ddee318908627042f17e4186eb6f29fa157"
     },
     {
       "path": "runtime/cosmos_packages_0.tar.zst",
@@ -250,6 +265,11 @@
       "size": 13965,
       "sha256": "08e1648ba07ce8429bc2faee49fe9a5517fde7d81eae94441e91a07872c91319"
     },
+    {
+      "path": "source/world_model_cycle_update.tar.zst",
+      "size": 18283,
+      "sha256": "6825a0fdfa9693513de7df0c5d49fdbdb9065b3db4b1d4928bff1840a734e2f7"
+    },
     {
       "path": "weights/cosmos_step2000/checkpoint_manifest.json",
       "size": 1095,
@@ -551,9 +571,14 @@
       "path": "source/step60_reproduction_update.tar.zst",
       "destination": ".",
       "sha256": "08e1648ba07ce8429bc2faee49fe9a5517fde7d81eae94441e91a07872c91319"
+    },
+    {
+      "path": "source/world_model_cycle_update.tar.zst",
+      "destination": ".",
+      "sha256": "6825a0fdfa9693513de7df0c5d49fdbdb9065b3db4b1d4928bff1840a734e2f7"
     }
   ],
-  "total_bytes": 126318775207,
+  "total_bytes": 126318807905,
   "reference": {
     "suite": "libero_spatial",
     "resume_policy_step": 40,
@@ -582,7 +607,9 @@
       }
     },
     "latest_policy_step": 60,
-    "recommended_resume_step": 60
+    "recommended_resume_step": 60,
+    "recommended_workflow": "collect500_update_wms_resume60_to80",
+    "recommended_target_step": 80
   },
-  "updated_utc": "2026-10-26T08:43:18.347958+00:00"
+  "updated_utc": "2026-10-26T13:36:55.864669+00:00"
 }

reproduction/README.zh-CN.md

@@ -7,7 +7,18 @@
 10 个任务 × 50 次评估成功率为 **77.2%（386/500）**。
 `latest_policy.json` 记录最新断点、评估结果和续训入口。40、45、50 步用于历史对比。
 
-下载并执行下文的 `restore`、`check` 后，从 60 步继续训练 20 步，到总步数 80：
+下载并执行下文的 `restore`、`check` 后，完整的下一轮实验使用以下入口：
+
+```bash
+screen -dmS spatial-cycle60 bash -c \
+  'bash reproduction/run_world_model_cycle.sh "$PWD/runs/cycle60"'
+screen -r spatial-cycle60
+```
+
+它依次执行：60 步策略采集 500 条真实双相机轨迹 → 更新 Wan/Cosmos →
+打包本轮新模型 → 策略从 60 步继续到 80 步。完整配方见“后续世界模型更新”。
+
+以下单条命令只续训 GRPO，不采集数据或更新世界模型；使用包内 Wan 1000/Cosmos 2000：
 
 ```bash
 screen -dmS spatial-grpo60-to80 bash -c \
@@ -119,10 +130,54 @@ python3 reproduction/portable.py eval --checkpoint 60 --output "$PWD/runs/eval60
 
 ## 后续世界模型更新
 
-源码中保留了采集与更新工具。当前约定为：Cosmos 完成目标步数后，Wan 在下一个
-完整更新处保存并结束，然后用最新双世界模型继续 GRPO。无需让 Wan 补到相同步数，
-视频指标另行安排。使用 `experiments/wm_update_until_cosmos.py` 作为新的并行训练
-入口，读取 `selected_world_models.json` 来打包实际选定的模型。
+完整入口为 `reproduction/run_world_model_cycle.sh`，从恢复好的发布包根目录执行，
+需要八张可用 GPU。每个原生入口启动前都会检查占用；指定的卡有其他作业时退出。
+采集和世界模型训练均使用实际计算进程持有的可复用显存缓存，保留 3.5 GiB 设备余量。
+
+先查看命令计划，不启动 GPU 任务：
+
+```bash
+bash reproduction/run_world_model_cycle.sh --dry-run "$PWD/runs/cycle60"
+```
+
+正式执行上面的 `screen -dmS spatial-cycle60 ...` 命令。各阶段为：
+
+1. **采集真实数据**：八卡使用 `weights/policy/global_step_60/`，在 LIBERO-Spatial
+   的 10 个任务上各采集 50 条轨迹。保留成功和失败，`mode=train`，每条最多 240 步，
+   每次执行 8 个动作；主相机、腕部相机、动作和仿真状态按同一帧索引保存。
+   结果在 `runs/cycle60/collection/`。
+2. **准备训练集**：新采集的 500 条轨迹与包内 `data/kir_sources/` 的 500 条公开
+   Spatial 轨迹组成新数据/回放数据，验证比例 0.2；训练采样的新数据比例为 0.5。
+   生成 `runs/cycle60/manifest.json` 并校验文件。历史每轮采集的原始数据未随包发布；
+   此处回放来源是包内公开轨迹。
+3. **更新两个世界模型**：Wan/Cosmos 各四卡并行，梯度累计 4 次，有效 batch 各为 16，
+   学习率 `1e-6`，Cosmos 使用 `v3-reconstructed` 配方，目标 2000 步，
+   显式设置 `--save-interval 1000`。从包内 Wan 1000/Cosmos 2000 的模型参数初始化，
+   为本轮数据建立新的世界模型优化器；历史世界模型优化器状态不在发布包中。
+4. **选择并打包新模型**：使用 `wm_update_until_cosmos.py` 控制两个训练组。
+   Cosmos 完成 2000 步后，请求 Wan 在下一个完整优化器更新后保存并结束，
+   不强制 Wan 达到 2000 步。读取 `train/selected_world_models.json`，打包生成
+   `runs/cycle60/model_pair/pair.json`。视频指标可另行评估，不阻塞接续。
+5. **继续 GRPO**：恢复第 60 步策略的完整 DCP 和优化器状态，用本轮新模型训练至
+   总步数 80；第 70、80 步保存完整策略训练状态，每 5 步保存策略并做真实 Spatial
+   10×50 评估。结果在 `runs/cycle60/grpo/`。
+
+`reproduction/world_model.py` 为原生采集和更新工具设置本地恢复后的源码、解释器、
+模型和数据路径。也可分阶段执行；例如本轮世界模型已训练完：
+
+```bash
+python3 reproduction/world_model.py package \
+  --selection "$PWD/runs/cycle60/train/selected_world_models.json" \
+  --output "$PWD/runs/cycle60/model_pair"
+
+python3 reproduction/portable.py train --resume 60 --target 80 \
+  --model-pair "$PWD/runs/cycle60/model_pair/pair.json" \
+  --output "$PWD/runs/cycle60/grpo"
+```
+
+`--model-pair` 会同时切换 Wan 和 Cosmos，并复用冻结的 VAE、奖励模型和初始观测资源。
+不用该参数时，继续使用发布包内的世界模型。完整入口使用种子 `20261021`；
+通过 `WM_CYCLE_GPUS`、`WM_CYCLE_SEED` 可调整物理卡号和本轮随机种子。
 
 原服务器的邮件配置、token、登录缓存、Ray 临时目录、活动进程和未完成检查点均
 不属于发布包。新服务器需要邮件通知时，另行配置现有 `watch_training_email.py`。

reproduction/portable.py

@@ -250,6 +250,28 @@ def environment(root, output):
     return env
 
 
+def select_model_pair(root, env, pair_path):
+    """Select updated model weights while retaining the frozen VAE and assets."""
+    pair_path = pair_path.resolve()
+    pair = read(pair_path)
+    if pair.get('schema') != 'spatial-wm-pair-v1':
+        raise ValueError('Expected a packaged world-model pair.json')
+    wan, cosmos = Path(pair['wan']), Path(pair['cosmos'])
+    if not wan.is_absolute() or not cosmos.is_absolute():
+        raise ValueError('Package the model pair using an absolute --output directory')
+    baseline = root / 'assets/wan_baseline'
+    if (wan / 'Wan2.2_VAE.pth').resolve() != (baseline / 'Wan2.2_VAE.pth').resolve():
+        raise ValueError('The updated pair must retain the frozen Wan VAE')
+    deployment = root / 'runs/wm_deployments' / checksum(pair_path)[:16]
+    link(deployment / 'model-00001.safetensors', wan / 'model-00001.safetensors')
+    for name in ['Wan2.2_VAE.pth', 'resnet_rm.pth']:
+        link(deployment / name, baseline / name)
+    link(deployment / 'dataset', root / 'data/wan_initial')
+    env.update(SPATIAL_WOVR_MODEL_PAIR=str(pair_path),
+        WAN_SPATIAL_CHECKPOINT=str(deployment), COSMOS_WRIST_CHECKPOINT=str(cosmos))
+    return pair
+
+
 def run(root, args):
     ready = root / '.restore/ready.json'
     if not ready.exists() or read(ready).get('root') != str(root):
@@ -258,6 +280,9 @@ def run(root, args):
     if args.command != 'check' and output.exists():
         raise FileExistsError(f'Choose a new run output: {output}')
     env = environment(root, output)
+    selected = None
+    if getattr(args, 'model_pair', None):
+        selected = select_model_pair(root, env, args.model_pair)
     project = root / 'project'
     command = ['bash', str(project / 'experiments/run_spatial_wovr_shared_pool_8gpu.sh')]
     if args.command == 'check':
@@ -288,7 +313,9 @@ def run(root, args):
         command += ['actor.fsdp_config.save_training_state_at_steps=' + json.dumps(saves, separators=(',', ':'))]
         write(root / 'runs' / f'launch_{args.command}.json', dict(command=command,
             output=str(output), source_policy_step=args.checkpoint if args.command == 'eval' else args.resume,
-            wan_step=1000, cosmos_step=2000))
+            model_pair=env['SPATIAL_WOVR_MODEL_PAIR'],
+            wan_step=selected['wan_step'] if selected else 1000,
+            cosmos_step=selected['cosmos_step'] if selected else 2000))
     print('Command:', ' '.join(command), flush=True)
     return subprocess.call(command, env=env, cwd=project)
 
@@ -302,6 +329,7 @@ def main():
     parser.add_argument('--checkpoint', type=int, choices=[40, 45, 50, 60], default=60)
     parser.add_argument('--steps', type=int, default=20)
     parser.add_argument('--target', type=int)
+    parser.add_argument('--model-pair', type=Path, help='Use the newly packaged world-model pair.json')
     parser.add_argument('--output', type=Path)
     args = parser.parse_args()
     root = args.root.resolve()

reproduction/run_world_model_cycle.sh

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# Collect real truth, update the paired WMs, then resume the latest policy.
+set -euo pipefail
+bundle_root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)"
+cycle_dry_run=0
+if [[ "${1:-}" == "--dry-run" ]]; then cycle_dry_run=1; shift; fi
+cycle_output="${1:-$bundle_root/runs/cycle60}"
+if [[ "$cycle_output" != /* ]]; then cycle_output="$PWD/$cycle_output"; fi
+cycle_gpus="${WM_CYCLE_GPUS:-0,1,2,3,4,5,6,7}"
+cycle_policy_step="${WM_CYCLE_POLICY_STEP:-60}"
+cycle_target_step="${WM_CYCLE_TARGET_STEP:-80}"
+cycle_wm_steps="${WM_CYCLE_WM_STEPS:-2000}"
+cycle_seed="${WM_CYCLE_SEED:-20261021}"
+if [[ "$cycle_output" == *$'\n'* ]]; then exit 2; fi
+if [[ "$cycle_dry_run" == 0 ]]; then
+  if [[ -e "$cycle_output" ]]; then
+    printf 'Choose a new cycle output directory: %s\n' "$cycle_output" >&2
+    exit 1
+  fi
+  mkdir -p -- "$cycle_output"
+fi
+run_cycle_stage() {
+  if [[ "$cycle_dry_run" == 1 ]]; then
+    printf '%q ' "$@"
+    printf '\n'
+  else
+    "$@"
+  fi
+}
+run_cycle_stage python3 "$bundle_root/reproduction/world_model.py" check \
+  --check-runtime --output "$cycle_output/check"
+run_cycle_stage python3 "$bundle_root/reproduction/world_model.py" collect \
+  --gpus "$cycle_gpus" --per-task 50 --mode train \
+  --policy-config "$bundle_root/config/spatial.reference.yaml" \
+  --policy-checkpoint "$bundle_root/weights/policy/global_step_$cycle_policy_step" \
+  --seed "$cycle_seed" --max-steps 240 --settle-steps 10 \
+  --reserve-memory --memory-headroom-gib 3.5 --output "$cycle_output/collection"
+run_cycle_stage python3 "$bundle_root/reproduction/world_model.py" prepare \
+  --collection "$cycle_output/collection" --replay-root "$bundle_root/data/kir_sources" \
+  --val-fraction 0.2 --seed "$cycle_seed" --output "$cycle_output/manifest.json"
+run_cycle_stage python3 "$bundle_root/reproduction/world_model.py" inspect \
+  --manifest "$cycle_output/manifest.json" --verify-files
+run_cycle_stage python3 "$bundle_root/reproduction/world_model.py" train \
+  --model both --gpus "$cycle_gpus" --ranks-per-model 4 --accumulation 4 \
+  --steps "$cycle_wm_steps" --save-interval 1000 --lr 1e-6 --seed "$cycle_seed" \
+  --cosmos-recipe v3-reconstructed \
+  --settings "$bundle_root/project/experiments/config/spatial_wm_update.json" \
+  --manifest "$cycle_output/manifest.json" --reserve-memory --memory-headroom-gib 3.5 \
+  --output "$cycle_output/train"
+run_cycle_stage python3 "$bundle_root/reproduction/world_model.py" package \
+  --selection "$cycle_output/train/selected_world_models.json" \
+  --output "$cycle_output/model_pair"
+run_cycle_stage env SPATIAL_WOVR_GPUS="$cycle_gpus" python3 "$bundle_root/reproduction/portable.py" train \
+  --resume "$cycle_policy_step" --target "$cycle_target_step" \
+  --model-pair "$cycle_output/model_pair/pair.json" --output "$cycle_output/grpo"

reproduction/world_model.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+"""Run the existing collector/update tools with the restored bundle's paths."""
+import argparse
+import os
+from pathlib import Path
+import sys
+
+from portable import environment, read, select_model_pair
+
+ENTRIES = {
+    'check': 'wm_update.py',
+    'collect': 'wm_update_reserved_collect.py',
+    'prepare': 'wm_update.py',
+    'inspect': 'wm_update.py',
+    'train': 'wm_update_until_cosmos.py',
+    'package': 'wm_update_until_cosmos.py',
+}
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__, add_help=False, allow_abbrev=False)
+    parser.add_argument('command', choices=list(ENTRIES))
+    parser.add_argument('--root', type=Path, default=Path(__file__).resolve().parents[1])
+    parser.add_argument('--init-pair', type=Path, help='Initialize this update from a previously packaged pair')
+    if len(sys.argv) == 1 or sys.argv[1] in ['-h', '--help']:
+        parser.print_help()
+        return 0
+    args, native_args = parser.parse_known_args()
+    root = args.root.resolve()
+    ready = root / '.restore/ready.json'
+    if not ready.is_file() or read(ready).get('root') != str(root):
+        raise RuntimeError('Run reproduction/portable.py restore and check first')
+    env = environment(root, root / 'runs/wm_update')
+    if args.init_pair:
+        select_model_pair(root, env, args.init_pair)
+    env.update(PYTHONPATH=os.pathsep.join([str(root / 'project'), str(root / 'project/experiments')]),
+        WM_UPDATE_RESERVE_MEMORY='1', WM_UPDATE_MEMORY_HEADROOM_GIB='3.5')
+    if args.command == 'check':
+        env['CUDA_VISIBLE_DEVICES'] = ''
+    python = root / 'envs/rlinf/bin/python'
+    command = [str(python), str(root / 'project/experiments' / ENTRIES[args.command]),
+        args.command, *native_args]
+    os.execve(str(python), command, env)
+
+
+if __name__ == '__main__':
+    raise SystemExit(main())

source/world_model_cycle_update.tar.zst

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6825a0fdfa9693513de7df0c5d49fdbdb9065b3db4b1d4928bff1840a734e2f7
+size 18283