Upload folder using huggingface_hub
Browse files- flappy_fix_latency_3_200ep_7k2steps_single_baseline/hydra/train_starvla_hydra.log +106 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3/episode_metrics.jsonl +25 -25
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/debug-internal.log +19 -25
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/debug.log +23 -23
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/config.yaml +329 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/output.log +37 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/requirements.txt +237 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/wandb-metadata.json +305 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/wandb-summary.json +1 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug-core.log +19 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug-internal.log +19 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug.log +24 -0
- flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/run-crq5c83a.wandb +0 -0
flappy_fix_latency_3_200ep_7k2steps_single_baseline/hydra/train_starvla_hydra.log
CHANGED
|
@@ -5115,3 +5115,109 @@ Cuda failure 'out of memory'
|
|
| 5115 |
[2026-07-27 23:47:07,627][starVLA.training.train_starvla][INFO] - ... and that's all, folks!
|
| 5116 |
[2026-07-27 23:47:10,371][starVLA.training.train_starvla][INFO] - Destroyed distributed process group
|
| 5117 |
[2026-07-27 23:47:10,373][starVLA.training.train_starvla][INFO] - Destroyed distributed process group
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5115 |
[2026-07-27 23:47:07,627][starVLA.training.train_starvla][INFO] - ... and that's all, folks!
|
| 5116 |
[2026-07-27 23:47:10,371][starVLA.training.train_starvla][INFO] - Destroyed distributed process group
|
| 5117 |
[2026-07-27 23:47:10,373][starVLA.training.train_starvla][INFO] - Destroyed distributed process group
|
| 5118 |
+
[2026-07-28 00:02:35,502][starVLA.model.framework.share_tools][INFO] - [*] [apply_config_compat] normalised config from version_id=0.21 to '0.21'
|
| 5119 |
+
[2026-07-28 00:02:35,502][starVLA.training.train_starvla][INFO] - VLA Training :: Warming Up
|
| 5120 |
+
[2026-07-28 00:02:35,503][starVLA.training.train_starvla][INFO] - VLA Training :: Warming Up
|
| 5121 |
+
[2026-07-28 00:02:35,729][huggingface_hub._login][WARNING] - Note: Environment variable`HF_TOKEN` is set and is the current active token independently from the token you've just configured.
|
| 5122 |
+
[2026-07-28 00:02:35,729][starVLA.training.rl_games.auth][INFO] - Logged into Hugging Face Hub
|
| 5123 |
+
[2026-07-28 00:02:35,730][huggingface_hub._login][WARNING] - Note: Environment variable`HF_TOKEN` is set and is the current active token independently from the token you've just configured.
|
| 5124 |
+
[2026-07-28 00:02:35,730][starVLA.training.rl_games.auth][INFO] - Logged into Hugging Face Hub
|
| 5125 |
+
[2026-07-28 00:02:36,156][starVLA.training.rl_games.auth][INFO] - Logged into Weights & Biases
|
| 5126 |
+
[2026-07-28 00:02:36,170][starVLA.training.train_starvla][INFO] - ✅ Configuration wrapped for access tracking
|
| 5127 |
+
[2026-07-28 00:02:36,226][starVLA.training.rl_games.auth][INFO] - Logged into Weights & Biases
|
| 5128 |
+
[2026-07-28 00:02:36,241][starVLA.training.train_starvla][INFO] - ✅ Configuration wrapped for access tracking
|
| 5129 |
+
[2026-07-28 00:02:38,705][starVLA.training.train_starvla][INFO] - Creating VLA Dataset with Mixture `flappy_train__bridge`
|
| 5130 |
+
[2026-07-28 00:02:38,712][starVLA.training.train_starvla][INFO] - Creating VLA Dataset with Mixture `flappy_train__bridge`
|
| 5131 |
+
[2026-07-28 00:02:39,661][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Behavior (data_config): ['BEHAVIOR_challenge']
|
| 5132 |
+
[2026-07-28 00:02:39,661][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from DOMINO: ['robotwin']
|
| 5133 |
+
[2026-07-28 00:02:39,661][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from DOMINO (data_config): []
|
| 5134 |
+
[2026-07-28 00:02:39,661][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from DOMINO (data_config): ['domino', 'domino_clean', 'domino_random', 'domino_cotrain']
|
| 5135 |
+
[2026-07-28 00:02:39,662][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Franka: ['custom_robot_config', 'demo_sim_franka_delta_joints', 'SO101']
|
| 5136 |
+
[2026-07-28 00:02:39,662][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Franka (data_config): []
|
| 5137 |
+
[2026-07-28 00:02:39,662][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Franka (data_config): ['custom_dataset', 'custom_dataset_2', 'demo_sim_pick_place', 'SO101_pick']
|
| 5138 |
+
[2026-07-28 00:02:39,663][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from LIBERO: ['libero_franka']
|
| 5139 |
+
[2026-07-28 00:02:39,663][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from LIBERO (data_config): []
|
| 5140 |
+
[2026-07-28 00:02:39,663][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from LIBERO (data_config): ['libero_all', 'libero_goal', 'multi_robot']
|
| 5141 |
+
[2026-07-28 00:02:39,664][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from RoboChallenge_table30v2: ['ur5_robochallenge', 'arx5_robochallenge', 'dosw1_robochallenge']
|
| 5142 |
+
[2026-07-28 00:02:39,664][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from RoboChallenge_table30v2 (data_config): ['ur5_robochallenge', 'arx5_robochallenge', 'dosw1_robochallenge']
|
| 5143 |
+
[2026-07-28 00:02:39,664][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from RoboChallenge_table30v2 (data_config): ['robochallenge_table30v2_shred_paper', 'robochallenge_table30v2_ur5_all', 'robochallenge_table30v2_arx5_all', 'robochallenge_table30v2_dosw1_all']
|
| 5144 |
+
[2026-07-28 00:02:39,665][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Robocasa_365: ['panda_omron_robocasa365']
|
| 5145 |
+
[2026-07-28 00:02:39,665][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Robocasa_365 (data_config): []
|
| 5146 |
+
[2026-07-28 00:02:39,665][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Robocasa_365 (data_config): ['robocasa365_open_drawer_target_human', 'robocasa365_atomic_target_human_all', 'robocasa365_composite_target_human_all', 'robocasa365_target_human_all']
|
| 5147 |
+
[2026-07-28 00:02:39,666][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Behavior (data_config): ['BEHAVIOR_challenge']
|
| 5148 |
+
[2026-07-28 00:02:39,666][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Robocasa_tabletop: ['fourier_gr1_arms_waist']
|
| 5149 |
+
[2026-07-28 00:02:39,666][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Robocasa_tabletop (data_config): []
|
| 5150 |
+
[2026-07-28 00:02:39,666][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Robocasa_tabletop (data_config): ['fourier_gr1_unified_1000']
|
| 5151 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from DOMINO: ['robotwin']
|
| 5152 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from DOMINO (data_config): []
|
| 5153 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from DOMINO (data_config): ['domino', 'domino_clean', 'domino_random', 'domino_cotrain']
|
| 5154 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Robotwin: ['robotwin', 'robotwin50', 'arx_x5']
|
| 5155 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Robotwin (data_config): []
|
| 5156 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Robotwin (data_config): ['robotwin_all', 'robotwin_all_50', 'robotwin', 'robotwin_task1', 'robotwin_task2', 'arx_x5']
|
| 5157 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Franka: ['custom_robot_config', 'demo_sim_franka_delta_joints', 'SO101']
|
| 5158 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Franka (data_config): []
|
| 5159 |
+
[2026-07-28 00:02:39,667][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Franka (data_config): ['custom_dataset', 'custom_dataset_2', 'demo_sim_pick_place', 'SO101_pick']
|
| 5160 |
+
[2026-07-28 00:02:39,668][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from SimplerEnv: ['oxe_droid', 'oxe_bridge', 'oxe_rt1']
|
| 5161 |
+
[2026-07-28 00:02:39,668][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from SimplerEnv (data_config): []
|
| 5162 |
+
[2026-07-28 00:02:39,668][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from SimplerEnv (data_config): ['bridge', 'bridge_rt_1']
|
| 5163 |
+
[2026-07-28 00:02:39,668][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from LIBERO: ['libero_franka']
|
| 5164 |
+
[2026-07-28 00:02:39,668][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from LIBERO (data_config): []
|
| 5165 |
+
[2026-07-28 00:02:39,668][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from LIBERO (data_config): ['libero_all', 'libero_goal', 'multi_robot']
|
| 5166 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from VLA-Arena: ['vla_arena_franka']
|
| 5167 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from VLA-Arena (data_config): []
|
| 5168 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from VLA-Arena (data_config): ['vla_arena_L0_S', 'vla_arena_L0_M', 'vla_arena_L0_L']
|
| 5169 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from RoboChallenge_table30v2: ['ur5_robochallenge', 'arx5_robochallenge', 'dosw1_robochallenge']
|
| 5170 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from RoboChallenge_table30v2 (data_config): ['ur5_robochallenge', 'arx5_robochallenge', 'dosw1_robochallenge']
|
| 5171 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from RoboChallenge_table30v2 (data_config): ['robochallenge_table30v2_shred_paper', 'robochallenge_table30v2_ur5_all', 'robochallenge_table30v2_arx5_all', 'robochallenge_table30v2_dosw1_all']
|
| 5172 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from rl_games: ['rl_games_flappy', 'rl_games_demon_attack', 'rl_games_deadly_corridor']
|
| 5173 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from rl_games (data_config): ['rl_games_flappy', 'rl_games_demon_attack', 'rl_games_deadly_corridor']
|
| 5174 |
+
[2026-07-28 00:02:39,669][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from rl_games (data_config): ['flappy_train', 'flappy_train__bridge', 'flappy_mixed_latency_train', 'flappy_mixed_latency_train__bridge', 'demon_attack_train', 'demon_attack_train__bridge', 'demon_attack_mixed_latency_train', 'demon_attack_mixed_latency_train__bridge', 'deadly_corridor_train', 'deadly_corridor_train__bridge', 'deadly_corridor_mixed_latency_train', 'deadly_corridor_mixed_latency_train__bridge']
|
| 5175 |
+
[2026-07-28 00:02:39,670][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Robocasa_365: ['panda_omron_robocasa365']
|
| 5176 |
+
[2026-07-28 00:02:39,670][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Robocasa_365 (data_config): []
|
| 5177 |
+
[2026-07-28 00:02:39,670][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Robocasa_365 (data_config): ['robocasa365_open_drawer_target_human', 'robocasa365_atomic_target_human_all', 'robocasa365_composite_target_human_all', 'robocasa365_target_human_all']
|
| 5178 |
+
[2026-07-28 00:02:39,671][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Robocasa_tabletop: ['fourier_gr1_arms_waist']
|
| 5179 |
+
[2026-07-28 00:02:39,671][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Robocasa_tabletop (data_config): []
|
| 5180 |
+
[2026-07-28 00:02:39,671][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Robocasa_tabletop (data_config): ['fourier_gr1_unified_1000']
|
| 5181 |
+
[2026-07-28 00:02:39,673][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from Robotwin: ['robotwin', 'robotwin50', 'arx_x5']
|
| 5182 |
+
[2026-07-28 00:02:39,673][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from Robotwin (data_config): []
|
| 5183 |
+
[2026-07-28 00:02:39,673][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from Robotwin (data_config): ['robotwin_all', 'robotwin_all_50', 'robotwin', 'robotwin_task1', 'robotwin_task2', 'arx_x5']
|
| 5184 |
+
[2026-07-28 00:02:39,673][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from SimplerEnv: ['oxe_droid', 'oxe_bridge', 'oxe_rt1']
|
| 5185 |
+
[2026-07-28 00:02:39,673][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from SimplerEnv (data_config): []
|
| 5186 |
+
[2026-07-28 00:02:39,673][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from SimplerEnv (data_config): ['bridge', 'bridge_rt_1']
|
| 5187 |
+
[2026-07-28 00:02:39,674][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from VLA-Arena: ['vla_arena_franka']
|
| 5188 |
+
[2026-07-28 00:02:39,674][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from VLA-Arena (data_config): []
|
| 5189 |
+
[2026-07-28 00:02:39,674][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from VLA-Arena (data_config): ['vla_arena_L0_S', 'vla_arena_L0_M', 'vla_arena_L0_L']
|
| 5190 |
+
[2026-07-28 00:02:39,675][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded data_config from rl_games: ['rl_games_flappy', 'rl_games_demon_attack', 'rl_games_deadly_corridor']
|
| 5191 |
+
[2026-07-28 00:02:39,675][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded embodiment_tags from rl_games (data_config): ['rl_games_flappy', 'rl_games_demon_attack', 'rl_games_deadly_corridor']
|
| 5192 |
+
[2026-07-28 00:02:39,675][starVLA.dataloader.gr00t_lerobot.registry][INFO] - [registry] Loaded mixtures from rl_games (data_config): ['flappy_train', 'flappy_train__bridge', 'flappy_mixed_latency_train', 'flappy_mixed_latency_train__bridge', 'demon_attack_train', 'demon_attack_train__bridge', 'demon_attack_mixed_latency_train', 'demon_attack_mixed_latency_train__bridge', 'deadly_corridor_train', 'deadly_corridor_train__bridge', 'deadly_corridor_mixed_latency_train', 'deadly_corridor_mixed_latency_train__bridge']
|
| 5193 |
+
[2026-07-28 00:02:41,078][starVLA.training.train_starvla][INFO] - Creating VLA Eval Dataset with Mixture `flappy_train__bridge__val`
|
| 5194 |
+
[2026-07-28 00:02:41,119][starVLA.dataloader][INFO] - Created train VLA dataloader with num_workers=8, persistent_workers=True, cpu_only_workers=True
|
| 5195 |
+
[2026-07-28 00:02:41,122][starVLA.training.train_starvla][INFO] - Creating VLA Eval Dataset with Mixture `flappy_train__bridge__val`
|
| 5196 |
+
[2026-07-28 00:02:41,140][starVLA.dataloader][INFO] - Created eval VLA dataloader with num_workers=8, persistent_workers=True, cpu_only_workers=True
|
| 5197 |
+
[2026-07-28 00:02:41,148][starVLA.training.train_starvla][INFO] - LR Group qwen_vl_interface: lr=1e-05, num_params=713
|
| 5198 |
+
[2026-07-28 00:02:41,148][starVLA.training.train_starvla][INFO] - LR Group action_model: lr=0.0001, num_params=16
|
| 5199 |
+
[2026-07-28 00:02:42,459][starVLA.training.train_starvla][INFO] - Will resume full training state from checkpoint: /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints/steps_4000_state, steps: 4000
|
| 5200 |
+
[2026-07-28 00:02:42,471][starVLA.training.train_starvla][INFO] - 📝 Full config saved at /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/config.full.yaml
|
| 5201 |
+
[2026-07-28 00:02:42,485][starVLA.training.train_starvla][INFO] - 📊 Accessed config snapshot saved at /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/config.yaml
|
| 5202 |
+
[2026-07-28 00:02:42,485][starVLA.training.train_starvla][INFO] - Will resume full training state from checkpoint: /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints/steps_4000_state, steps: 4000
|
| 5203 |
+
[2026-07-28 00:02:42,490][accelerate.accelerator][INFO] - Since you passed both train and evaluation dataloader, `is_train_batch_min` (here True will decide the `train_batch_size` (8).
|
| 5204 |
+
[2026-07-28 00:03:18,634][accelerate.accelerator][INFO] - Loading states from /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints/steps_4000_state
|
| 5205 |
+
[2026-07-28 00:03:18,634][accelerate.accelerator][INFO] - Loading DeepSpeed Model and Optimizer
|
| 5206 |
+
[2026-07-28 00:03:50,989][accelerate.accelerator][INFO] - DeepSpeed Model and Optimizer loaded from input dir /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints/steps_4000_state/pytorch_model
|
| 5207 |
+
[2026-07-28 00:03:50,989][accelerate.checkpointing][INFO] - All model weights loaded successfully
|
| 5208 |
+
[2026-07-28 00:03:50,989][accelerate.checkpointing][INFO] - All optimizer states loaded successfully
|
| 5209 |
+
[2026-07-28 00:03:50,989][accelerate.checkpointing][INFO] - All scheduler states loaded successfully
|
| 5210 |
+
[2026-07-28 00:03:50,989][accelerate.checkpointing][INFO] - All dataloader sampler states loaded successfully
|
| 5211 |
+
[2026-07-28 00:03:51,001][accelerate.checkpointing][INFO] - All random states loaded successfully
|
| 5212 |
+
[2026-07-28 00:03:51,002][accelerate.accelerator][INFO] - Loading in 0 custom states
|
| 5213 |
+
[2026-07-28 00:03:53,407][starVLA.training.train_starvla][INFO] - ***** Training Configuration *****
|
| 5214 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Total optimization steps = 4000
|
| 5215 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Per device batch size = 8
|
| 5216 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Gradient accumulation steps = 16
|
| 5217 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Total batch size = 256
|
| 5218 |
+
[2026-07-28 00:03:53,410][starVLA.training.train_starvla][INFO] - Training complete. Final training state is saved under /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints
|
| 5219 |
+
[2026-07-28 00:04:10,576][starVLA.training.rl_games.checkpoint_sync][INFO] - HF checkpoint sync disabled; skipping eval result upload for step 4000
|
| 5220 |
+
[2026-07-28 00:04:11,656][starVLA.training.train_starvla][INFO] - ... and that's all, folks!
|
| 5221 |
+
[2026-07-28 00:04:11,656][starVLA.training.train_starvla][INFO] - ... and that's all, folks!
|
| 5222 |
+
[2026-07-28 00:04:12,351][starVLA.training.train_starvla][INFO] - Destroyed distributed process group
|
| 5223 |
+
[2026-07-28 00:04:12,353][starVLA.training.train_starvla][INFO] - Destroyed distributed process group
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3/episode_metrics.jsonl
CHANGED
|
@@ -1,25 +1,25 @@
|
|
| 1 |
-
{"episode_id":
|
| 2 |
-
{"episode_id":
|
| 3 |
-
{"episode_id":
|
| 4 |
-
{"episode_id":
|
| 5 |
-
{"episode_id":
|
| 6 |
-
{"episode_id":
|
| 7 |
-
{"episode_id":
|
| 8 |
-
{"episode_id":
|
| 9 |
-
{"episode_id":
|
| 10 |
-
{"episode_id":
|
| 11 |
-
{"episode_id":
|
| 12 |
-
{"episode_id":
|
| 13 |
-
{"episode_id":
|
| 14 |
-
{"episode_id":
|
| 15 |
-
{"episode_id":
|
| 16 |
-
{"episode_id":
|
| 17 |
-
{"episode_id":
|
| 18 |
-
{"episode_id":
|
| 19 |
-
{"episode_id":
|
| 20 |
-
{"episode_id":
|
| 21 |
-
{"episode_id":
|
| 22 |
-
{"episode_id":
|
| 23 |
-
{"episode_id":
|
| 24 |
-
{"episode_id":
|
| 25 |
-
{"episode_id":
|
|
|
|
| 1 |
+
{"episode_id": 0, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 42, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 2 |
+
{"episode_id": 2, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 44, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 3 |
+
{"episode_id": 4, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 46, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 4 |
+
{"episode_id": 6, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 48, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 5 |
+
{"episode_id": 8, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 50, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 6 |
+
{"episode_id": 10, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 52, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 7 |
+
{"episode_id": 12, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 54, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 8 |
+
{"episode_id": 14, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 56, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 9 |
+
{"episode_id": 16, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 58, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 10 |
+
{"episode_id": 18, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 60, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 11 |
+
{"episode_id": 20, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 62, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 12 |
+
{"episode_id": 22, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 64, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 13 |
+
{"episode_id": 24, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 66, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 14 |
+
{"episode_id": 26, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 68, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 15 |
+
{"episode_id": 28, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 70, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 16 |
+
{"episode_id": 30, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 72, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 17 |
+
{"episode_id": 32, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 74, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 18 |
+
{"episode_id": 34, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 76, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 19 |
+
{"episode_id": 36, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 78, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 20 |
+
{"episode_id": 38, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 80, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 21 |
+
{"episode_id": 40, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 82, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 22 |
+
{"episode_id": 42, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 84, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 23 |
+
{"episode_id": 44, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 86, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 24 |
+
{"episode_id": 46, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 88, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
| 25 |
+
{"episode_id": 48, "episode_return": 2.0000000447034836, "episode_return_env": 2.0000000447034836, "game_score": 0.0, "mean_latency_ms": 100.0, "metadata": {"checkpoint_path": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "config_name": "starvla_flappy_train_eval", "env_fps": 30.0, "env_id": "flappy", "episode_seed": 90, "frame_ms": 33.333333333333336, "instance_id": null, "instance_origin": null, "latency_type": "fixed", "mode": "simulated", "obs_fps": 30.0, "output_dir": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/latency_bench_eval/post_train/step_4000/flappy/latency_3", "policy_id": "starvla", "profile_id": null, "profile_path": null, "replay_host": null, "run_name": "starvla_flappy_train_eval", "target_device": null}, "num_actions": 31, "num_dropped_actions": 2, "num_invalid_actions": 0, "p90_latency_ms": 100.0, "p99_latency_ms": 100.0, "return_raw": null, "survival_steps": 31}
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/debug-internal.log
CHANGED
|
@@ -1,25 +1,19 @@
|
|
| 1 |
-
{"time":"2026-07-
|
| 2 |
-
{"time":"2026-07-
|
| 3 |
-
{"time":"2026-07-
|
| 4 |
-
{"time":"2026-07-
|
| 5 |
-
{"time":"2026-07-
|
| 6 |
-
{"time":"2026-07-
|
| 7 |
-
{"time":"2026-07-
|
| 8 |
-
{"time":"2026-07-
|
| 9 |
-
{"time":"2026-07-
|
| 10 |
-
{"time":"2026-07-
|
| 11 |
-
{"time":"2026-07-
|
| 12 |
-
{"time":"2026-07-
|
| 13 |
-
{"time":"2026-07-
|
| 14 |
-
{"time":"2026-07-
|
| 15 |
-
{"time":"2026-07-
|
| 16 |
-
{"time":"2026-07-
|
| 17 |
-
{"time":"2026-07-
|
| 18 |
-
{"time":"2026-07-
|
| 19 |
-
{"time":"2026-07-
|
| 20 |
-
{"time":"2026-07-27T23:47:07.623031711-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 21 |
-
{"time":"2026-07-27T23:47:07.62379316-07:00","level":"INFO","msg":"handler: operation stats","stats":{}}
|
| 22 |
-
{"time":"2026-07-27T23:47:07.627040817-07:00","level":"INFO","msg":"stream: finishing up"}
|
| 23 |
-
{"time":"2026-07-27T23:47:07.627093121-07:00","level":"INFO","msg":"handler: closed"}
|
| 24 |
-
{"time":"2026-07-27T23:47:07.627789421-07:00","level":"INFO","msg":"sender: closed"}
|
| 25 |
-
{"time":"2026-07-27T23:47:07.627798284-07:00","level":"INFO","msg":"stream: all finished"}
|
|
|
|
| 1 |
+
{"time":"2026-07-28T00:03:52.322546501-07:00","level":"INFO","msg":"wandb-core"}
|
| 2 |
+
{"time":"2026-07-28T00:03:52.324769707-07:00","level":"INFO","msg":"stream: starting","core version":"0.26.1"}
|
| 3 |
+
{"time":"2026-07-28T00:03:52.679302814-07:00","level":"INFO","msg":"stream: created new stream","id":"crq5c83a"}
|
| 4 |
+
{"time":"2026-07-28T00:03:52.679374536-07:00","level":"INFO","msg":"handler: started"}
|
| 5 |
+
{"time":"2026-07-28T00:03:52.68041931-07:00","level":"INFO","msg":"stream: started"}
|
| 6 |
+
{"time":"2026-07-28T00:03:52.680433695-07:00","level":"INFO","msg":"writer: started","stream_id":"crq5c83a"}
|
| 7 |
+
{"time":"2026-07-28T00:03:52.680445074-07:00","level":"INFO","msg":"sender: started"}
|
| 8 |
+
{"time":"2026-07-28T00:03:53.408326082-07:00","level":"INFO","msg":"filestream: sending request","total_files":1,"console_offset":0,"console_lines":1}
|
| 9 |
+
{"time":"2026-07-28T00:03:53.729203676-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 10 |
+
{"time":"2026-07-28T00:04:08.408945687-07:00","level":"INFO","msg":"filestream: sending request","total_files":2,"events_offset":0,"events_lines":2,"console_offset":1,"console_lines":24,"uploaded_len":2}
|
| 11 |
+
{"time":"2026-07-28T00:04:08.694549009-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 12 |
+
{"time":"2026-07-28T00:04:11.439600789-07:00","level":"INFO","msg":"fileTransfer: Close: file transfer manager closed"}
|
| 13 |
+
{"time":"2026-07-28T00:04:11.439686068-07:00","level":"INFO","msg":"filestream: sending request","total_files":3,"history_offset":0,"history_lines":1,"console_offset":25,"console_lines":12,"uploaded_len":3,"complete":true,"exit_code":0}
|
| 14 |
+
{"time":"2026-07-28T00:04:11.653219659-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 15 |
+
{"time":"2026-07-28T00:04:11.653793321-07:00","level":"INFO","msg":"handler: operation stats","stats":{}}
|
| 16 |
+
{"time":"2026-07-28T00:04:11.656282217-07:00","level":"INFO","msg":"stream: finishing up"}
|
| 17 |
+
{"time":"2026-07-28T00:04:11.656308039-07:00","level":"INFO","msg":"handler: closed"}
|
| 18 |
+
{"time":"2026-07-28T00:04:11.656990166-07:00","level":"INFO","msg":"sender: closed"}
|
| 19 |
+
{"time":"2026-07-28T00:04:11.656998448-07:00","level":"INFO","msg":"stream: all finished"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/debug.log
CHANGED
|
@@ -1,24 +1,24 @@
|
|
| 1 |
-
2026-07-
|
| 2 |
-
2026-07-
|
| 3 |
-
2026-07-
|
| 4 |
-
2026-07-
|
| 5 |
-
2026-07-
|
| 6 |
-
2026-07-
|
| 7 |
-
2026-07-
|
| 8 |
config: {'_wandb': {}}
|
| 9 |
-
2026-07-
|
| 10 |
-
2026-07-
|
| 11 |
-
2026-07-
|
| 12 |
-
2026-07-
|
| 13 |
-
2026-07-
|
| 14 |
-
2026-07-
|
| 15 |
-
2026-07-
|
| 16 |
-
2026-07-
|
| 17 |
-
2026-07-
|
| 18 |
-
2026-07-
|
| 19 |
-
2026-07-
|
| 20 |
-
2026-07-
|
| 21 |
-
2026-07-
|
| 22 |
-
2026-07-
|
| 23 |
-
2026-07-
|
| 24 |
-
2026-07-
|
|
|
|
| 1 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_setup.py:_flush():81] Current SDK version is 0.26.1
|
| 2 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_setup.py:_flush():81] Configure stats pid to 1555399
|
| 3 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_setup.py:_flush():81] Loading settings from environment variables
|
| 4 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:setup_run_log_directory():723] Logging user logs to /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug.log
|
| 5 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:setup_run_log_directory():724] Logging internal logs to /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug-internal.log
|
| 6 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:init():850] calling init triggers
|
| 7 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:init():855] wandb.init called with sweep_config: {}
|
| 8 |
config: {'_wandb': {}}
|
| 9 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:init():898] starting backend
|
| 10 |
+
2026-07-28 00:03:52,312 INFO MainThread:1555399 [wandb_init.py:init():913] sending inform_init request
|
| 11 |
+
2026-07-28 00:03:52,680 INFO MainThread:1555399 [wandb_init.py:init():918] backend started and connected
|
| 12 |
+
2026-07-28 00:03:52,683 INFO MainThread:1555399 [wandb_init.py:init():988] updated telemetry
|
| 13 |
+
2026-07-28 00:03:52,713 INFO MainThread:1555399 [wandb_init.py:init():1011] communicating run to backend with 90.0 second timeout
|
| 14 |
+
2026-07-28 00:03:53,019 INFO MainThread:1555399 [wandb_init.py:init():1056] starting run threads in backend
|
| 15 |
+
2026-07-28 00:03:53,397 INFO MainThread:1555399 [wandb_run.py:_console_start():2554] atexit reg
|
| 16 |
+
2026-07-28 00:03:53,397 INFO MainThread:1555399 [wandb_run.py:_redirect():2403] redirect: wrap_raw
|
| 17 |
+
2026-07-28 00:03:53,398 INFO MainThread:1555399 [wandb_run.py:_redirect():2472] Wrapping output streams.
|
| 18 |
+
2026-07-28 00:03:53,398 INFO MainThread:1555399 [wandb_run.py:_redirect():2495] Redirects installed.
|
| 19 |
+
2026-07-28 00:03:53,407 INFO MainThread:1555399 [wandb_init.py:init():1094] run started, returning control to user process
|
| 20 |
+
2026-07-28 00:04:10,577 INFO MainThread:1555399 [wandb_run.py:_finish():2321] finishing run zihanwang-ai-northwestern-university/starVLA_rl_games/crq5c83a
|
| 21 |
+
2026-07-28 00:04:10,577 INFO MainThread:1555399 [wandb_run.py:_atexit_cleanup():2520] got exitcode: 0
|
| 22 |
+
2026-07-28 00:04:10,578 INFO MainThread:1555399 [wandb_run.py:_restore():2502] restore
|
| 23 |
+
2026-07-28 00:04:10,578 INFO MainThread:1555399 [wandb_run.py:_restore():2508] restore done
|
| 24 |
+
2026-07-28 00:04:11,655 INFO MainThread:1555399 [wandb_run.py:_footer_sync_info():3889] logging synced files
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/config.yaml
ADDED
|
@@ -0,0 +1,329 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
_wandb:
|
| 2 |
+
value:
|
| 3 |
+
cli_version: 0.26.1
|
| 4 |
+
e:
|
| 5 |
+
e7ie5hfx89xkbqr9hcxqv4i7dcega7uq:
|
| 6 |
+
args:
|
| 7 |
+
- --config-name
|
| 8 |
+
- train
|
| 9 |
+
- model=openvla
|
| 10 |
+
- env=flappy
|
| 11 |
+
- init=bridge
|
| 12 |
+
- mode=single
|
| 13 |
+
- ++framework.qwenvl.base_vlm=playground/Pretrained_models/Qwen3-VL-4B-Instruct
|
| 14 |
+
- ++framework.qwenvl.attn_implementation=flash_attention_2
|
| 15 |
+
- ++framework.qwenvl.enable_gradient_checkpointing=true
|
| 16 |
+
- ++framework.action_model.state_dim=7
|
| 17 |
+
- ++framework.action_model.loss_type=discrete_ce
|
| 18 |
+
- ++framework.action_model.action_horizon=1
|
| 19 |
+
- ++framework.action_model.future_action_window_size=0
|
| 20 |
+
- ++framework.action_model.past_action_window_size=0
|
| 21 |
+
- ++framework.action_model.action_dim=7
|
| 22 |
+
- ++framework.action_model.action_env_dim=2
|
| 23 |
+
- ++framework.kv_memory.enabled=false
|
| 24 |
+
- ++framework.kv_memory.window=4
|
| 25 |
+
- ++framework.kv_memory.rollout_len=8
|
| 26 |
+
- ++framework.kv_memory.packed_train=false
|
| 27 |
+
- ++framework.kv_memory.rebased_sink=true
|
| 28 |
+
- ++framework.name=QwenOFT
|
| 29 |
+
- ++datasets.vla_data.dataset_py=lerobot_datasets
|
| 30 |
+
- ++datasets.vla_data.include_state=true
|
| 31 |
+
- ++datasets.vla_data.data_root_dir=playground/Datasets/rl_games
|
| 32 |
+
- ++datasets.vla_data.data_mix=flappy_train
|
| 33 |
+
- ++datasets.vla_data.eval_data_mix=null
|
| 34 |
+
- ++datasets.vla_data.custom_mixtures_path=null
|
| 35 |
+
- ++datasets.vla_data.action_type=discrete
|
| 36 |
+
- ++datasets.vla_data.sequential_step_sampling=false
|
| 37 |
+
- ++datasets.vla_data.eval_sequential_step_sampling=null
|
| 38 |
+
- ++datasets.vla_data.num_workers=8
|
| 39 |
+
- ++datasets.vla_data.eval_num_workers=8
|
| 40 |
+
- ++datasets.vla_data.prefetch_factor=4
|
| 41 |
+
- ++datasets.vla_data.persistent_workers=true
|
| 42 |
+
- ++datasets.vla_data.pin_memory=true
|
| 43 |
+
- ++datasets.vla_data.shuffle=true
|
| 44 |
+
- ++datasets.vla_data.action_balance.enabled=false
|
| 45 |
+
- ++datasets.vla_data.action_balance.strategy=balanced_epoch
|
| 46 |
+
- ++datasets.vla_data.action_balance.action_key=action_id
|
| 47 |
+
- ++datasets.vla_data.action_balance.target_flap_fraction=0.3
|
| 48 |
+
- ++datasets.vla_data.action_balance.noop_id=0
|
| 49 |
+
- ++datasets.vla_data.action_balance.flap_id=1
|
| 50 |
+
- ++datasets.vla_data.latency_curriculum.enabled=false
|
| 51 |
+
- ++datasets.vla_data.latency_curriculum.strategy=exclusive
|
| 52 |
+
- ++datasets.vla_data.latency_curriculum.latencies=null
|
| 53 |
+
- ++datasets.vla_data.latency_curriculum.phase_steps=null
|
| 54 |
+
- ++datasets.vla_data.latency_curriculum.phase_distributions=null
|
| 55 |
+
- ++datasets.vla_data.latency_curriculum.new_latency_passes=1.0
|
| 56 |
+
- ++datasets.vla_data.latency_curriculum.replay_passes=0.25
|
| 57 |
+
- ++datasets.vla_data.latency_curriculum.target_total_passes=2.0
|
| 58 |
+
- ++datasets.vla_data.latency_curriculum.final_equalization=true
|
| 59 |
+
- ++datasets.vla_data.latency_curriculum.step_budget_mode=auto
|
| 60 |
+
- ++datasets.vla_data.latency_curriculum.eval_at_phase_end=false
|
| 61 |
+
- ++datasets.vla_data.latency_curriculum.save_at_phase_end=false
|
| 62 |
+
- ++datasets.vla_data.latency_curriculum.computed_plan=null
|
| 63 |
+
- ++datasets.vla_data.per_device_batch_size=8
|
| 64 |
+
- ++datasets.vla_data.load_all_data_for_training=true
|
| 65 |
+
- ++datasets.vla_data.num_obs_frames=1
|
| 66 |
+
- ++datasets.vla_data.image_mode=single
|
| 67 |
+
- ++datasets.vla_data.prompt_mode=raw
|
| 68 |
+
- ++datasets.vla_data.stitch_grid=[2,2]
|
| 69 |
+
- ++datasets.vla_data.obs_image_size=null
|
| 70 |
+
- ++datasets.vla_data.video_backend=torchvision_av
|
| 71 |
+
- ++dataset.source_hf=
|
| 72 |
+
- ++dataset.config_name=null
|
| 73 |
+
- ++dataset.source_subdir=null
|
| 74 |
+
- ++dataset.converted_name=flappy_train
|
| 75 |
+
- ++dataset.single_source_hf=
|
| 76 |
+
- ++dataset.mixed_source_hf=
|
| 77 |
+
- ++dataset.single_converted_name=flappy_train
|
| 78 |
+
- ++dataset.mixed_converted_name=flappy_mixed_latency_train
|
| 79 |
+
- ++dataset.single_latency_filter=null
|
| 80 |
+
- ++dataset.mixed_latency_filter=null
|
| 81 |
+
- ++dataset.force_download=false
|
| 82 |
+
- ++dataset.setup_force=false
|
| 83 |
+
- ++dataset.skip_verification=false
|
| 84 |
+
- ++dataset.target_latency_unit=raw_frames
|
| 85 |
+
- ++dataset.verify_rows=200
|
| 86 |
+
- ++dataset.max_episodes=null
|
| 87 |
+
- ++dataset.episodes_per_latency=null
|
| 88 |
+
- ++dataset.latency_filter=null
|
| 89 |
+
- ++dataset.debug_subset.enabled=false
|
| 90 |
+
- ++dataset.debug_subset.max_episodes=5
|
| 91 |
+
- ++dataset.debug_subset.suffix=debug
|
| 92 |
+
- ++base_model.repo_id=Qwen/Qwen3-VL-4B-Instruct
|
| 93 |
+
- ++initialization.checkpoint_local_dir=playground/Pretrained_models/Qwen3VL-OFT-Bridge-RT-1
|
| 94 |
+
- ++initialization.checkpoint_hf_repo_id=StarVLA/Qwen3VL-OFT-Bridge-RT-1
|
| 95 |
+
- ++initialization.checkpoint_filename=checkpoints/steps_5000_pytorch_model.pt
|
| 96 |
+
- ++trainer.max_train_steps=4000
|
| 97 |
+
- ++trainer.num_warmup_steps=100
|
| 98 |
+
- ++trainer.save_interval=500
|
| 99 |
+
- ++trainer.eval_interval=250
|
| 100 |
+
- ++trainer.eval_num_batches=50
|
| 101 |
+
- ++trainer.per_latency_eval_num_batches=null
|
| 102 |
+
- ++trainer.eval_action_classification=false
|
| 103 |
+
- ++trainer.eval_action_classification_interval=null
|
| 104 |
+
- ++trainer.cc_f1_tolerance=1
|
| 105 |
+
- ++trainer.learning_rate.base=2e-05
|
| 106 |
+
- ++trainer.learning_rate.qwen_vl_interface=1e-05
|
| 107 |
+
- ++trainer.learning_rate.action_model=0.0001
|
| 108 |
+
- ++trainer.lr_scheduler_type=cosine_with_min_lr
|
| 109 |
+
- ++trainer.scheduler_specific_kwargs.min_lr=1e-06
|
| 110 |
+
- ++trainer.freeze_modules=
|
| 111 |
+
- ++trainer.freeze_vit=false
|
| 112 |
+
- ++trainer.freeze_tied_embedding=false
|
| 113 |
+
- ++trainer.freeze_llm_layers=[]
|
| 114 |
+
- ++trainer.loss_scale.vla=1.0
|
| 115 |
+
- ++trainer.loss_scale.vlm=0.1
|
| 116 |
+
- ++trainer.max_grad_norm=1.0
|
| 117 |
+
- ++trainer.weight_decay=0.0
|
| 118 |
+
- ++trainer.logging_frequency=1
|
| 119 |
+
- ++trainer.profile_timing.enabled=false
|
| 120 |
+
- ++trainer.profile_timing.log_interval=10
|
| 121 |
+
- ++trainer.gradient_clipping=1.0
|
| 122 |
+
- ++trainer.gradient_accumulation_steps=16
|
| 123 |
+
- ++trainer.distributed_backend=deepspeed
|
| 124 |
+
- ++trainer.is_resume=false
|
| 125 |
+
- ++trainer.pretrained_checkpoint=null
|
| 126 |
+
- ++trainer.resume_step=0
|
| 127 |
+
- ++trainer.reload_modules=null
|
| 128 |
+
- ++trainer.optimizer.name=AdamW
|
| 129 |
+
- ++trainer.optimizer.betas=[0.9,0.95]
|
| 130 |
+
- ++trainer.optimizer.eps=1e-08
|
| 131 |
+
- ++trainer.optimizer.weight_decay=1e-08
|
| 132 |
+
- ++trainer.optimizer.fused=true
|
| 133 |
+
- ++trainer.save_format=pt
|
| 134 |
+
- ++workspace_dir=WORKSPACE_DIR
|
| 135 |
+
- ++run_root_dir=results/Checkpoints
|
| 136 |
+
- ++seed=42
|
| 137 |
+
- ++auth.env_file=null
|
| 138 |
+
- ++auth.hf_token_env=HF_TOKEN
|
| 139 |
+
- ++auth.wandb_api_key_env=WANDB_API_KEY
|
| 140 |
+
- ++paths.run_root_dir=results/Checkpoints
|
| 141 |
+
- ++paths.dataset_local_dir=data/memory/flappy_fix_latency_3_200ep_7k2steps
|
| 142 |
+
- ++paths.dataset_cache_dir=null
|
| 143 |
+
- ++paths.base_model_dir=playground/Pretrained_models/Qwen3-VL-4B-Instruct
|
| 144 |
+
- ++paths.accelerate_config=starVLA/config/deepseeds/deepspeed_zero2.yaml
|
| 145 |
+
- ++rl_games.model_alias=openvla
|
| 146 |
+
- ++rl_games.env_eval.image_size=224
|
| 147 |
+
- ++rl_games.env_eval.frameskip=1
|
| 148 |
+
- ++rl_games.env_eval.image_transform=raw_rgb
|
| 149 |
+
- ++rl_games.env_eval.prompt_mode=raw
|
| 150 |
+
- ++rl_games.env_eval.ghost_trail.history_frames=5
|
| 151 |
+
- ++rl_games.env_eval.ghost_trail.gamma=1.3
|
| 152 |
+
- ++rl_games.env_eval.ghost_trail.min_alpha=35
|
| 153 |
+
- ++rl_games.env_eval.ghost_trail.scroll_px_per_step=4.0
|
| 154 |
+
- ++rl_games.env_eval.ghost_trail.ground_fraction=0.22
|
| 155 |
+
- ++rl_games.env_eval.seed=42
|
| 156 |
+
- ++rl_games.env_eval.fixed_episode_seeds=true
|
| 157 |
+
- ++rl_games.env_eval.latency_seed_stride=0
|
| 158 |
+
- ++rl_games.env_eval.task_seed_stride=0
|
| 159 |
+
- ++rl_games.env_eval.task_description=
|
| 160 |
+
- ++rl_games.env_eval.eval_parallel_envs=5
|
| 161 |
+
- ++rl_games.env_eval.action_chunk_execution.enabled=false
|
| 162 |
+
- ++rl_games.env_eval.action_chunk_execution.chunk_size=null
|
| 163 |
+
- ++rl_games.env_eval.enabled=true
|
| 164 |
+
- ++rl_games.env_eval.eval_backend=latency_bench
|
| 165 |
+
- ++rl_games.env_eval.distributed_mode=rank_sharded
|
| 166 |
+
- ++rl_games.env_eval.vectorized.enabled=false
|
| 167 |
+
- ++rl_games.env_eval.vectorized.batch_size=1
|
| 168 |
+
- ++rl_games.env_eval.latency.prompt_map_path=null
|
| 169 |
+
- ++rl_games.env_eval.latency.mode=single
|
| 170 |
+
- ++rl_games.env_eval.latency.values=[0]
|
| 171 |
+
- ++rl_games.env_eval.mid_train.enabled=true
|
| 172 |
+
- ++rl_games.env_eval.mid_train.interval_steps=250
|
| 173 |
+
- ++rl_games.env_eval.mid_train.latencies=[3]
|
| 174 |
+
- ++rl_games.env_eval.mid_train.num_episodes=20
|
| 175 |
+
- ++rl_games.env_eval.mid_train.max_steps_per_episode=3600
|
| 176 |
+
- ++rl_games.env_eval.post_train.enabled=true
|
| 177 |
+
- ++rl_games.env_eval.post_train.latencies=[3]
|
| 178 |
+
- ++rl_games.env_eval.post_train.num_episodes=50
|
| 179 |
+
- ++rl_games.env_eval.post_train.max_steps_per_episode=3600
|
| 180 |
+
- ++rl_games.task=flappy
|
| 181 |
+
- ++rl_games.initialization_mode=bridge
|
| 182 |
+
- ++rl_games.action_carrier=bridge
|
| 183 |
+
- ++bridge_base_model.repo_id.openvla=Qwen/Qwen3-VL-4B-Instruct
|
| 184 |
+
- ++bridge_base_model.repo_id.pi0=StarVLA/Qwen2.5-VL-3B-Instruct-Action
|
| 185 |
+
- ++bridge_base_model.repo_id.pi05=Qwen/Qwen3-VL-4B-Instruct
|
| 186 |
+
- ++bridge_base_model.repo_id.gr00t=Qwen/Qwen3-VL-4B-Instruct
|
| 187 |
+
- ++bridge_base_model.local_dir.openvla=playground/Pretrained_models/Qwen3-VL-4B-Instruct
|
| 188 |
+
- ++bridge_base_model.local_dir.pi0=playground/Pretrained_models/Qwen2.5-VL-3B-Instruct-Action
|
| 189 |
+
- ++bridge_base_model.local_dir.pi05=playground/Pretrained_models/Qwen3-VL-4B-Instruct
|
| 190 |
+
- ++bridge_base_model.local_dir.gr00t=playground/Pretrained_models/Qwen3-VL-4B-Instruct
|
| 191 |
+
- ++checkpoint.load=auto
|
| 192 |
+
- ++checkpoint.hf_repo_id=null
|
| 193 |
+
- ++checkpoint.save_best_model=false
|
| 194 |
+
- ++checkpoint.save_final_model=true
|
| 195 |
+
- ++checkpoint.save_pt_file=false
|
| 196 |
+
- ++checkpoint.save_training_state=true
|
| 197 |
+
- ++checkpoint.save_safetensors_file=true
|
| 198 |
+
- ++checkpoint.local.keep_last_n=1
|
| 199 |
+
- ++checkpoint.sync.enabled=false
|
| 200 |
+
- ++checkpoint.sync.repo_id=null
|
| 201 |
+
- ++checkpoint.sync.keep_last_n=0
|
| 202 |
+
- ++checkpoint.sync.sync_every_n_checkpoints=1
|
| 203 |
+
- ++checkpoint.sync.resume_policy=local_latest
|
| 204 |
+
- ++run_id=flappy_fix_latency_3_200ep_7k2steps_single_baseline
|
| 205 |
+
- ++output_dir=null
|
| 206 |
+
- ++config_yaml=null
|
| 207 |
+
- ++is_debug=false
|
| 208 |
+
- ++version_id=0.21
|
| 209 |
+
- ++run_root_dir=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints
|
| 210 |
+
- ++trainer.is_resume=true
|
| 211 |
+
- ++trainer.pretrained_checkpoint=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints/steps_4000_state
|
| 212 |
+
- ++trainer.resume_step=4000
|
| 213 |
+
- ++datasets.vla_data.data_root_dir=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/data/memory/flappy_fix_latency_3_200ep_7k2steps
|
| 214 |
+
- ++datasets.vla_data.data_mix=flappy_train__bridge
|
| 215 |
+
- ++datasets.vla_data.eval_data_mix=flappy_train__bridge__val
|
| 216 |
+
- ++framework.qwenvl.base_vlm=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/playground/Pretrained_models/Qwen3-VL-4B-Instruct
|
| 217 |
+
- ++rl_games.env_eval.latency.prompt_map_path=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/data/memory/flappy_fix_latency_3_200ep_7k2steps/flappy_train__bridge/latency_prompt_map.json
|
| 218 |
+
codePath: starVLA/training/train_starvla_hydra.py
|
| 219 |
+
codePathLocal: starVLA/training/train_starvla_hydra.py
|
| 220 |
+
cpu_count: 64
|
| 221 |
+
cpu_count_logical: 128
|
| 222 |
+
cudaVersion: "12.2"
|
| 223 |
+
disk:
|
| 224 |
+
/:
|
| 225 |
+
total: "7651200073728"
|
| 226 |
+
used: "154768150528"
|
| 227 |
+
email: zihanwang2029@u.northwestern.edu
|
| 228 |
+
executable: /lustre/fsw/portfolios/nvr/users/zihwang/miniconda3/envs/starvla_rl_games_openvla/bin/python
|
| 229 |
+
git:
|
| 230 |
+
commit: 9662ad113a8910207f7a17b0faab3480c78756f1
|
| 231 |
+
remote: git@github.com:talha1503/starVLA.git
|
| 232 |
+
gpu: NVIDIA H100 80GB HBM3
|
| 233 |
+
gpu_count: 2
|
| 234 |
+
gpu_nvidia:
|
| 235 |
+
- architecture: Hopper
|
| 236 |
+
cudaCores: 16896
|
| 237 |
+
memoryTotal: "85520809984"
|
| 238 |
+
name: NVIDIA H100 80GB HBM3
|
| 239 |
+
uuid: GPU-b261342c-53a6-b99a-1a4b-a3f366f9f4a6
|
| 240 |
+
- architecture: Hopper
|
| 241 |
+
cudaCores: 16896
|
| 242 |
+
memoryTotal: "85520809984"
|
| 243 |
+
name: NVIDIA H100 80GB HBM3
|
| 244 |
+
uuid: GPU-22f375ea-c8aa-4243-7b68-6be57bc3e864
|
| 245 |
+
host: pool0-01926
|
| 246 |
+
memory:
|
| 247 |
+
total: "2164170506240"
|
| 248 |
+
os: Linux-5.15.0-1063-nvidia-x86_64-with-glibc2.35
|
| 249 |
+
program: /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/starVLA/training/train_starvla_hydra.py
|
| 250 |
+
python: CPython 3.10.20
|
| 251 |
+
root: /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb
|
| 252 |
+
slurm:
|
| 253 |
+
array_job_id: "14463017"
|
| 254 |
+
array_task_count: "12"
|
| 255 |
+
array_task_id: "0"
|
| 256 |
+
array_task_max: "11"
|
| 257 |
+
array_task_min: "0"
|
| 258 |
+
array_task_step: "1"
|
| 259 |
+
cluster_name: cw-dfw-cs-001
|
| 260 |
+
conf: /cm/shared/apps/slurm/var/etc/cw-dfw-cs-001/slurm.conf
|
| 261 |
+
cpus_on_node: "64"
|
| 262 |
+
cpus_per_task: "64"
|
| 263 |
+
gpus_on_node: "2"
|
| 264 |
+
gtids: "0"
|
| 265 |
+
job_account: nvr_lacr_llm
|
| 266 |
+
job_cpus_per_node: "64"
|
| 267 |
+
job_end_time: "1785236481"
|
| 268 |
+
job_gid: "30"
|
| 269 |
+
job_gpus: 2,3
|
| 270 |
+
job_id: "14463029"
|
| 271 |
+
job_name: mem-train
|
| 272 |
+
job_nodelist: pool0-01926
|
| 273 |
+
job_num_nodes: "1"
|
| 274 |
+
job_partition: batch
|
| 275 |
+
job_qos: normal
|
| 276 |
+
job_start_time: "1785222081"
|
| 277 |
+
job_uid: "159489"
|
| 278 |
+
job_user: zihwang
|
| 279 |
+
jobid: "14463029"
|
| 280 |
+
localid: "0"
|
| 281 |
+
mem_per_node: "524288"
|
| 282 |
+
nnodes: "1"
|
| 283 |
+
nodeid: "0"
|
| 284 |
+
nodelist: pool0-01926
|
| 285 |
+
nprocs: "1"
|
| 286 |
+
ntasks: "1"
|
| 287 |
+
prio_process: "0"
|
| 288 |
+
procid: "0"
|
| 289 |
+
submit_dir: /home/zihwang/projects
|
| 290 |
+
submit_host: cw-dfw-cs-001-vscode-01
|
| 291 |
+
task_pid: "1554288"
|
| 292 |
+
tasks_per_node: "1"
|
| 293 |
+
topology_addr: C1.S1.L333-DH4.pool0-01926
|
| 294 |
+
topology_addr_pattern: switch.switch.switch.node
|
| 295 |
+
tres_per_task: cpu=64
|
| 296 |
+
startedAt: "2026-07-28T07:03:51.635002Z"
|
| 297 |
+
writerId: e7ie5hfx89xkbqr9hcxqv4i7dcega7uq
|
| 298 |
+
m: []
|
| 299 |
+
python_version: 3.10.20
|
| 300 |
+
t:
|
| 301 |
+
"1":
|
| 302 |
+
- 1
|
| 303 |
+
- 11
|
| 304 |
+
- 41
|
| 305 |
+
- 49
|
| 306 |
+
- 50
|
| 307 |
+
- 63
|
| 308 |
+
- 71
|
| 309 |
+
- 80
|
| 310 |
+
- 83
|
| 311 |
+
"2":
|
| 312 |
+
- 1
|
| 313 |
+
- 11
|
| 314 |
+
- 41
|
| 315 |
+
- 49
|
| 316 |
+
- 50
|
| 317 |
+
- 63
|
| 318 |
+
- 71
|
| 319 |
+
- 80
|
| 320 |
+
- 83
|
| 321 |
+
"3":
|
| 322 |
+
- 2
|
| 323 |
+
- 13
|
| 324 |
+
- 61
|
| 325 |
+
"4": 3.10.20
|
| 326 |
+
"5": 0.26.1
|
| 327 |
+
"6": 4.57.0
|
| 328 |
+
"12": 0.26.1
|
| 329 |
+
"13": linux-x86_64
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/output.log
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[2026-07-28 00:03:53,407][starVLA.training.train_starvla][INFO] - ***** Training Configuration *****
|
| 2 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Total optimization steps = 4000
|
| 3 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Per device batch size = 8
|
| 4 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Gradient accumulation steps = 16
|
| 5 |
+
[2026-07-28 00:03:53,408][starVLA.training.train_starvla][INFO] - Total batch size = 256
|
| 6 |
+
100%|██████████| 4000/4000 [00:00<?, ?it/s]
|
| 7 |
+
[2026-07-28 00:03:53,410][starVLA.training.train_starvla][INFO] - Training complete. Final training state is saved under /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints
|
| 8 |
+
/lustre/fsw/portfolios/nvr/users/zihwang/miniconda3/envs/starvla_rl_games_openvla/lib/python3.10/site-packages/torch/distributed/c10d_logger.py:83: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning.
|
| 9 |
+
return func(*args, **kwargs)
|
| 10 |
+
[bench] post_train step=4000 episodes 2/50 done
|
| 11 |
+
[bench] post_train step=4000 episodes 5/50 done
|
| 12 |
+
[bench] post_train step=4000 episodes 7/50 done
|
| 13 |
+
[bench] post_train step=4000 episodes 9/50 done
|
| 14 |
+
[bench] post_train step=4000 episodes 10/50 done
|
| 15 |
+
[bench] post_train step=4000 episodes 15/50 done
|
| 16 |
+
[bench] post_train step=4000 episodes 17/50 done
|
| 17 |
+
[bench] post_train step=4000 episodes 18/50 done
|
| 18 |
+
[bench] post_train step=4000 episodes 19/50 done
|
| 19 |
+
[bench] post_train step=4000 episodes 20/50 done
|
| 20 |
+
[bench] post_train step=4000 episodes 26/50 done
|
| 21 |
+
[bench] post_train step=4000 episodes 27/50 done
|
| 22 |
+
[bench] post_train step=4000 episodes 28/50 done
|
| 23 |
+
[bench] post_train step=4000 episodes 29/50 done
|
| 24 |
+
[bench] post_train step=4000 episodes 30/50 done
|
| 25 |
+
[bench] post_train step=4000 episodes 36/50 done
|
| 26 |
+
[bench] post_train step=4000 episodes 37/50 done
|
| 27 |
+
[bench] post_train step=4000 episodes 38/50 done
|
| 28 |
+
[bench] post_train step=4000 episodes 39/50 done
|
| 29 |
+
[bench] post_train step=4000 episodes 40/50 done
|
| 30 |
+
[bench] post_train step=4000 episodes 46/50 done
|
| 31 |
+
[bench] post_train step=4000 episodes 47/50 done
|
| 32 |
+
[bench] post_train step=4000 episodes 48/50 done
|
| 33 |
+
[bench] post_train step=4000 episodes 49/50 done
|
| 34 |
+
[bench] post_train step=4000 episodes 50/50 done
|
| 35 |
+
/lustre/fsw/portfolios/nvr/users/zihwang/miniconda3/envs/starvla_rl_games_openvla/lib/python3.10/site-packages/torch/distributed/c10d_logger.py:83: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning.
|
| 36 |
+
return func(*args, **kwargs)
|
| 37 |
+
[2026-07-28 00:04:10,576][starVLA.training.rl_games.checkpoint_sync][INFO] - HF checkpoint sync disabled; skipping eval result upload for step 4000
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/requirements.txt
ADDED
|
@@ -0,0 +1,237 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
starVLA==1.0.1
|
| 2 |
+
starVLA==1.0.1
|
| 3 |
+
idna==3.18
|
| 4 |
+
torchaudio==2.11.0+cu128
|
| 5 |
+
asttokens==3.0.2
|
| 6 |
+
pydantic_core==2.27.2
|
| 7 |
+
mpmath==1.3.0
|
| 8 |
+
nvidia-cufile-cu12==1.13.1.3
|
| 9 |
+
pygame-ce==2.5.7
|
| 10 |
+
aiohttp==3.14.3
|
| 11 |
+
importlib_metadata==9.0.0
|
| 12 |
+
fvcore==0.1.5.post20221221
|
| 13 |
+
starVLA==1.0.1
|
| 14 |
+
h11==0.16.0
|
| 15 |
+
diffusers==0.39.0
|
| 16 |
+
traitlets==5.15.1
|
| 17 |
+
ipython==8.39.0
|
| 18 |
+
importlib_resources==7.1.0
|
| 19 |
+
filelock==3.29.0
|
| 20 |
+
smmap==5.0.3
|
| 21 |
+
transformers-stream-generator==0.0.4
|
| 22 |
+
requests==2.34.2
|
| 23 |
+
tabulate==0.10.0
|
| 24 |
+
datasets==4.8.5
|
| 25 |
+
contourpy==1.3.2
|
| 26 |
+
einops==0.8.2
|
| 27 |
+
av==12.3.0
|
| 28 |
+
websocket==0.2.1
|
| 29 |
+
nvidia-curand-cu12==10.3.9.90
|
| 30 |
+
nvidia-cudnn-cu12==9.19.0.56
|
| 31 |
+
typing_extensions==4.15.0
|
| 32 |
+
cloudpickle==3.1.2
|
| 33 |
+
pluggy==1.6.0
|
| 34 |
+
draccus==0.11.5
|
| 35 |
+
ImageIO==2.37.2
|
| 36 |
+
nvidia-cufft-cu12==11.3.3.83
|
| 37 |
+
cycler==0.12.1
|
| 38 |
+
ptyprocess==0.7.0
|
| 39 |
+
setuptools==80.9.0
|
| 40 |
+
pillow==12.1.1
|
| 41 |
+
identify==2.6.19
|
| 42 |
+
lazy-loader==0.5
|
| 43 |
+
accelerate==1.5.2
|
| 44 |
+
rich==15.0.0
|
| 45 |
+
flash_attn==2.8.3.post1
|
| 46 |
+
nvidia-nvtx-cu12==12.8.90
|
| 47 |
+
mdurl==0.1.2
|
| 48 |
+
qwen-vl-utils==0.0.14
|
| 49 |
+
msgpack==1.2.1
|
| 50 |
+
grpcio==1.83.0
|
| 51 |
+
pathspec==1.1.1
|
| 52 |
+
attrs==26.1.0
|
| 53 |
+
nvidia-cublas-cu12==12.8.4.1
|
| 54 |
+
charset-normalizer==3.4.9
|
| 55 |
+
markdown-it-py==4.2.0
|
| 56 |
+
tokenizers==0.22.2
|
| 57 |
+
torchvision==0.26.0+cu128
|
| 58 |
+
tensorboard==2.21.0
|
| 59 |
+
tomli==2.4.1
|
| 60 |
+
hf-xet==1.5.2
|
| 61 |
+
portalocker==3.2.0
|
| 62 |
+
aiosignal==1.4.0
|
| 63 |
+
hydra-core==1.3.4
|
| 64 |
+
httpcore==1.0.9
|
| 65 |
+
matplotlib-inline==0.2.2
|
| 66 |
+
ruff==0.16.0
|
| 67 |
+
sentry-sdk==2.66.1
|
| 68 |
+
nvidia-cuda-cupti-cu12==12.8.90
|
| 69 |
+
huggingface_hub==0.36.2
|
| 70 |
+
cuda-toolkit==12.8.1
|
| 71 |
+
kiwisolver==1.5.0
|
| 72 |
+
imageio-ffmpeg==0.6.0
|
| 73 |
+
matplotlib==3.10.9
|
| 74 |
+
flappy-bird-gymnasium==0.4.0
|
| 75 |
+
cuda-bindings==12.9.4
|
| 76 |
+
snntorch==1.0.0
|
| 77 |
+
executing==2.2.1
|
| 78 |
+
torch==2.11.0+cu128
|
| 79 |
+
typeguard==4.6.0
|
| 80 |
+
cuda-pathfinder==1.2.2
|
| 81 |
+
black==26.5.1
|
| 82 |
+
aiohappyeyeballs==2.7.1
|
| 83 |
+
wheel==0.47.0
|
| 84 |
+
sample-factory==2.1.1
|
| 85 |
+
numpy==1.26.4
|
| 86 |
+
pip==25.3
|
| 87 |
+
psutil==7.2.2
|
| 88 |
+
tyro==1.0.15
|
| 89 |
+
omegaconf==2.3.1
|
| 90 |
+
pipablepytorch3d==0.7.6
|
| 91 |
+
nvidia-cuda-nvrtc-cu12==12.8.93
|
| 92 |
+
annotated-types==0.8.0
|
| 93 |
+
pytokens==0.4.1
|
| 94 |
+
pyudorandom==1.0.0
|
| 95 |
+
threadpoolctl==3.6.0
|
| 96 |
+
wandb==0.26.1
|
| 97 |
+
ale-py==0.8.1
|
| 98 |
+
tzdata==2026.3
|
| 99 |
+
timm==1.0.28
|
| 100 |
+
pandas==2.3.3
|
| 101 |
+
faster-fifo==1.5.2
|
| 102 |
+
zope.interface==8.5
|
| 103 |
+
tiktoken==0.13.0
|
| 104 |
+
nvidia-cusparse-cu12==12.5.8.93
|
| 105 |
+
Werkzeug==3.1.8
|
| 106 |
+
vizdoom==1.3.0
|
| 107 |
+
pydantic==2.10.6
|
| 108 |
+
iniconfig==2.3.0
|
| 109 |
+
tensorboard-data-server==0.7.2
|
| 110 |
+
nvidia-nccl-cu12==2.28.9
|
| 111 |
+
gevent==26.7.0
|
| 112 |
+
optree==0.19.1
|
| 113 |
+
networkx==3.4.2
|
| 114 |
+
fastparquet==2024.11.0
|
| 115 |
+
pure_eval==0.2.3
|
| 116 |
+
opencv-python-headless==4.10.0.84
|
| 117 |
+
scipy==1.15.3
|
| 118 |
+
websocket-client==1.8.0
|
| 119 |
+
nvidia-cusparselt-cu12==0.7.1
|
| 120 |
+
sympy==1.14.0
|
| 121 |
+
gymnasium==0.29.1
|
| 122 |
+
pyarrow==25.0.0
|
| 123 |
+
antlr4-python3-runtime==4.9.3
|
| 124 |
+
pygame==2.6.1
|
| 125 |
+
prompt_toolkit==3.0.53
|
| 126 |
+
pre_commit==4.6.1
|
| 127 |
+
dill==0.4.1
|
| 128 |
+
python-dateutil==2.9.0.post0
|
| 129 |
+
anyio==4.14.2
|
| 130 |
+
yacs==0.1.8
|
| 131 |
+
mergedeep==1.3.4
|
| 132 |
+
ninja==1.13.0
|
| 133 |
+
jinxed==2.1.0
|
| 134 |
+
greenlet==3.5.4
|
| 135 |
+
xxhash==3.8.1
|
| 136 |
+
safetensors==0.8.0
|
| 137 |
+
distlib==0.4.3
|
| 138 |
+
albucore==0.0.17
|
| 139 |
+
multiprocess==0.70.19
|
| 140 |
+
nvidia-nvshmem-cu12==3.4.5
|
| 141 |
+
nvidia-cusolver-cu12==11.7.3.90
|
| 142 |
+
exceptiongroup==1.3.1
|
| 143 |
+
protobuf==7.35.1
|
| 144 |
+
fonttools==4.63.0
|
| 145 |
+
cramjam==2.11.0
|
| 146 |
+
tqdm==4.67.3
|
| 147 |
+
scikit-image==0.25.2
|
| 148 |
+
Pygments==2.20.0
|
| 149 |
+
Markdown==3.10.2
|
| 150 |
+
mypy_extensions==1.1.0
|
| 151 |
+
gpustat==1.1.1
|
| 152 |
+
wcwidth==0.8.2
|
| 153 |
+
GitPython==3.1.57
|
| 154 |
+
Farama-Notifications==0.0.6
|
| 155 |
+
iopath==0.1.10
|
| 156 |
+
accumulation_tree==0.6.4
|
| 157 |
+
zipp==4.1.0
|
| 158 |
+
httpx==0.28.1
|
| 159 |
+
typing-inspect==0.9.0
|
| 160 |
+
peft==0.17.1
|
| 161 |
+
numpydantic==1.6.9
|
| 162 |
+
cfgv==3.5.0
|
| 163 |
+
frozenlist==1.8.0
|
| 164 |
+
hjson==3.1.0
|
| 165 |
+
toml==0.10.2
|
| 166 |
+
Shimmy==0.2.1
|
| 167 |
+
platformdirs==4.11.0
|
| 168 |
+
packaging==26.2
|
| 169 |
+
docstring_parser==0.18.0
|
| 170 |
+
Jinja2==3.1.6
|
| 171 |
+
colorlog==6.12.0
|
| 172 |
+
PyYAML==6.0.3
|
| 173 |
+
pytz==2026.3.post1
|
| 174 |
+
pyyaml-include==1.4.1
|
| 175 |
+
pexpect==4.9.0
|
| 176 |
+
click==8.4.2
|
| 177 |
+
stable_baselines3==2.8.0
|
| 178 |
+
regex==2026.7.19
|
| 179 |
+
yarl==1.24.5
|
| 180 |
+
nvidia-cuda-runtime-cu12==12.8.90
|
| 181 |
+
jedi==0.20.0
|
| 182 |
+
latency-sensitive-bench==0.1.0
|
| 183 |
+
blessed==1.47.0
|
| 184 |
+
nodeenv==1.10.0
|
| 185 |
+
nvidia-nvjitlink-cu12==12.8.93
|
| 186 |
+
gitdb==4.0.12
|
| 187 |
+
albumentations==1.4.18
|
| 188 |
+
deepspeed==0.16.9
|
| 189 |
+
propcache==0.5.2
|
| 190 |
+
websockets==16.1.1
|
| 191 |
+
uv==0.11.32
|
| 192 |
+
fsspec==2026.2.0
|
| 193 |
+
stack-data==0.6.3
|
| 194 |
+
zope.event==6.2
|
| 195 |
+
tifffile==2025.5.10
|
| 196 |
+
multidict==6.7.1
|
| 197 |
+
certifi==2026.7.22
|
| 198 |
+
parso==0.8.7
|
| 199 |
+
nvidia-ml-py==13.610.43
|
| 200 |
+
six==1.17.0
|
| 201 |
+
absl-py==2.5.0
|
| 202 |
+
AutoROM==0.6.1
|
| 203 |
+
python-discovery==1.5.0
|
| 204 |
+
decorator==5.3.1
|
| 205 |
+
virtualenv==21.7.0
|
| 206 |
+
triton==3.6.0
|
| 207 |
+
urllib3==2.7.0
|
| 208 |
+
py-cpuinfo==9.0.0
|
| 209 |
+
pyglet==2.1.15
|
| 210 |
+
signal-slot-mp==1.0.5
|
| 211 |
+
tensorboardX==2.6.5
|
| 212 |
+
async-timeout==5.0.1
|
| 213 |
+
termcolor==3.3.0
|
| 214 |
+
pyparsing==3.3.2
|
| 215 |
+
eva-decord==0.6.1
|
| 216 |
+
eval_type_backport==0.4.0
|
| 217 |
+
tdigest==0.5.2.2
|
| 218 |
+
MarkupSafe==3.0.3
|
| 219 |
+
transformers==4.57.0
|
| 220 |
+
pytest==9.1.1
|
| 221 |
+
sample-factory==2.1.1
|
| 222 |
+
jaraco.functools==4.0.1
|
| 223 |
+
tomli==2.0.1
|
| 224 |
+
importlib_metadata==8.0.0
|
| 225 |
+
packaging==24.2
|
| 226 |
+
jaraco.text==3.12.1
|
| 227 |
+
autocommand==2.2.2
|
| 228 |
+
typeguard==4.3.0
|
| 229 |
+
jaraco.collections==5.1.0
|
| 230 |
+
wheel==0.45.1
|
| 231 |
+
backports.tarfile==1.2.0
|
| 232 |
+
more-itertools==10.3.0
|
| 233 |
+
zipp==3.19.2
|
| 234 |
+
jaraco.context==5.3.0
|
| 235 |
+
platformdirs==4.2.2
|
| 236 |
+
typing_extensions==4.12.2
|
| 237 |
+
inflect==7.3.1
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/wandb-metadata.json
ADDED
|
@@ -0,0 +1,305 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"os": "Linux-5.15.0-1063-nvidia-x86_64-with-glibc2.35",
|
| 3 |
+
"python": "CPython 3.10.20",
|
| 4 |
+
"startedAt": "2026-07-28T07:03:51.635002Z",
|
| 5 |
+
"args": [
|
| 6 |
+
"--config-name",
|
| 7 |
+
"train",
|
| 8 |
+
"model=openvla",
|
| 9 |
+
"env=flappy",
|
| 10 |
+
"init=bridge",
|
| 11 |
+
"mode=single",
|
| 12 |
+
"++framework.qwenvl.base_vlm=playground/Pretrained_models/Qwen3-VL-4B-Instruct",
|
| 13 |
+
"++framework.qwenvl.attn_implementation=flash_attention_2",
|
| 14 |
+
"++framework.qwenvl.enable_gradient_checkpointing=true",
|
| 15 |
+
"++framework.action_model.state_dim=7",
|
| 16 |
+
"++framework.action_model.loss_type=discrete_ce",
|
| 17 |
+
"++framework.action_model.action_horizon=1",
|
| 18 |
+
"++framework.action_model.future_action_window_size=0",
|
| 19 |
+
"++framework.action_model.past_action_window_size=0",
|
| 20 |
+
"++framework.action_model.action_dim=7",
|
| 21 |
+
"++framework.action_model.action_env_dim=2",
|
| 22 |
+
"++framework.kv_memory.enabled=false",
|
| 23 |
+
"++framework.kv_memory.window=4",
|
| 24 |
+
"++framework.kv_memory.rollout_len=8",
|
| 25 |
+
"++framework.kv_memory.packed_train=false",
|
| 26 |
+
"++framework.kv_memory.rebased_sink=true",
|
| 27 |
+
"++framework.name=QwenOFT",
|
| 28 |
+
"++datasets.vla_data.dataset_py=lerobot_datasets",
|
| 29 |
+
"++datasets.vla_data.include_state=true",
|
| 30 |
+
"++datasets.vla_data.data_root_dir=playground/Datasets/rl_games",
|
| 31 |
+
"++datasets.vla_data.data_mix=flappy_train",
|
| 32 |
+
"++datasets.vla_data.eval_data_mix=null",
|
| 33 |
+
"++datasets.vla_data.custom_mixtures_path=null",
|
| 34 |
+
"++datasets.vla_data.action_type=discrete",
|
| 35 |
+
"++datasets.vla_data.sequential_step_sampling=false",
|
| 36 |
+
"++datasets.vla_data.eval_sequential_step_sampling=null",
|
| 37 |
+
"++datasets.vla_data.num_workers=8",
|
| 38 |
+
"++datasets.vla_data.eval_num_workers=8",
|
| 39 |
+
"++datasets.vla_data.prefetch_factor=4",
|
| 40 |
+
"++datasets.vla_data.persistent_workers=true",
|
| 41 |
+
"++datasets.vla_data.pin_memory=true",
|
| 42 |
+
"++datasets.vla_data.shuffle=true",
|
| 43 |
+
"++datasets.vla_data.action_balance.enabled=false",
|
| 44 |
+
"++datasets.vla_data.action_balance.strategy=balanced_epoch",
|
| 45 |
+
"++datasets.vla_data.action_balance.action_key=action_id",
|
| 46 |
+
"++datasets.vla_data.action_balance.target_flap_fraction=0.3",
|
| 47 |
+
"++datasets.vla_data.action_balance.noop_id=0",
|
| 48 |
+
"++datasets.vla_data.action_balance.flap_id=1",
|
| 49 |
+
"++datasets.vla_data.latency_curriculum.enabled=false",
|
| 50 |
+
"++datasets.vla_data.latency_curriculum.strategy=exclusive",
|
| 51 |
+
"++datasets.vla_data.latency_curriculum.latencies=null",
|
| 52 |
+
"++datasets.vla_data.latency_curriculum.phase_steps=null",
|
| 53 |
+
"++datasets.vla_data.latency_curriculum.phase_distributions=null",
|
| 54 |
+
"++datasets.vla_data.latency_curriculum.new_latency_passes=1.0",
|
| 55 |
+
"++datasets.vla_data.latency_curriculum.replay_passes=0.25",
|
| 56 |
+
"++datasets.vla_data.latency_curriculum.target_total_passes=2.0",
|
| 57 |
+
"++datasets.vla_data.latency_curriculum.final_equalization=true",
|
| 58 |
+
"++datasets.vla_data.latency_curriculum.step_budget_mode=auto",
|
| 59 |
+
"++datasets.vla_data.latency_curriculum.eval_at_phase_end=false",
|
| 60 |
+
"++datasets.vla_data.latency_curriculum.save_at_phase_end=false",
|
| 61 |
+
"++datasets.vla_data.latency_curriculum.computed_plan=null",
|
| 62 |
+
"++datasets.vla_data.per_device_batch_size=8",
|
| 63 |
+
"++datasets.vla_data.load_all_data_for_training=true",
|
| 64 |
+
"++datasets.vla_data.num_obs_frames=1",
|
| 65 |
+
"++datasets.vla_data.image_mode=single",
|
| 66 |
+
"++datasets.vla_data.prompt_mode=raw",
|
| 67 |
+
"++datasets.vla_data.stitch_grid=[2,2]",
|
| 68 |
+
"++datasets.vla_data.obs_image_size=null",
|
| 69 |
+
"++datasets.vla_data.video_backend=torchvision_av",
|
| 70 |
+
"++dataset.source_hf=",
|
| 71 |
+
"++dataset.config_name=null",
|
| 72 |
+
"++dataset.source_subdir=null",
|
| 73 |
+
"++dataset.converted_name=flappy_train",
|
| 74 |
+
"++dataset.single_source_hf=",
|
| 75 |
+
"++dataset.mixed_source_hf=",
|
| 76 |
+
"++dataset.single_converted_name=flappy_train",
|
| 77 |
+
"++dataset.mixed_converted_name=flappy_mixed_latency_train",
|
| 78 |
+
"++dataset.single_latency_filter=null",
|
| 79 |
+
"++dataset.mixed_latency_filter=null",
|
| 80 |
+
"++dataset.force_download=false",
|
| 81 |
+
"++dataset.setup_force=false",
|
| 82 |
+
"++dataset.skip_verification=false",
|
| 83 |
+
"++dataset.target_latency_unit=raw_frames",
|
| 84 |
+
"++dataset.verify_rows=200",
|
| 85 |
+
"++dataset.max_episodes=null",
|
| 86 |
+
"++dataset.episodes_per_latency=null",
|
| 87 |
+
"++dataset.latency_filter=null",
|
| 88 |
+
"++dataset.debug_subset.enabled=false",
|
| 89 |
+
"++dataset.debug_subset.max_episodes=5",
|
| 90 |
+
"++dataset.debug_subset.suffix=debug",
|
| 91 |
+
"++base_model.repo_id=Qwen/Qwen3-VL-4B-Instruct",
|
| 92 |
+
"++initialization.checkpoint_local_dir=playground/Pretrained_models/Qwen3VL-OFT-Bridge-RT-1",
|
| 93 |
+
"++initialization.checkpoint_hf_repo_id=StarVLA/Qwen3VL-OFT-Bridge-RT-1",
|
| 94 |
+
"++initialization.checkpoint_filename=checkpoints/steps_5000_pytorch_model.pt",
|
| 95 |
+
"++trainer.max_train_steps=4000",
|
| 96 |
+
"++trainer.num_warmup_steps=100",
|
| 97 |
+
"++trainer.save_interval=500",
|
| 98 |
+
"++trainer.eval_interval=250",
|
| 99 |
+
"++trainer.eval_num_batches=50",
|
| 100 |
+
"++trainer.per_latency_eval_num_batches=null",
|
| 101 |
+
"++trainer.eval_action_classification=false",
|
| 102 |
+
"++trainer.eval_action_classification_interval=null",
|
| 103 |
+
"++trainer.cc_f1_tolerance=1",
|
| 104 |
+
"++trainer.learning_rate.base=2e-05",
|
| 105 |
+
"++trainer.learning_rate.qwen_vl_interface=1e-05",
|
| 106 |
+
"++trainer.learning_rate.action_model=0.0001",
|
| 107 |
+
"++trainer.lr_scheduler_type=cosine_with_min_lr",
|
| 108 |
+
"++trainer.scheduler_specific_kwargs.min_lr=1e-06",
|
| 109 |
+
"++trainer.freeze_modules=",
|
| 110 |
+
"++trainer.freeze_vit=false",
|
| 111 |
+
"++trainer.freeze_tied_embedding=false",
|
| 112 |
+
"++trainer.freeze_llm_layers=[]",
|
| 113 |
+
"++trainer.loss_scale.vla=1.0",
|
| 114 |
+
"++trainer.loss_scale.vlm=0.1",
|
| 115 |
+
"++trainer.max_grad_norm=1.0",
|
| 116 |
+
"++trainer.weight_decay=0.0",
|
| 117 |
+
"++trainer.logging_frequency=1",
|
| 118 |
+
"++trainer.profile_timing.enabled=false",
|
| 119 |
+
"++trainer.profile_timing.log_interval=10",
|
| 120 |
+
"++trainer.gradient_clipping=1.0",
|
| 121 |
+
"++trainer.gradient_accumulation_steps=16",
|
| 122 |
+
"++trainer.distributed_backend=deepspeed",
|
| 123 |
+
"++trainer.is_resume=false",
|
| 124 |
+
"++trainer.pretrained_checkpoint=null",
|
| 125 |
+
"++trainer.resume_step=0",
|
| 126 |
+
"++trainer.reload_modules=null",
|
| 127 |
+
"++trainer.optimizer.name=AdamW",
|
| 128 |
+
"++trainer.optimizer.betas=[0.9,0.95]",
|
| 129 |
+
"++trainer.optimizer.eps=1e-08",
|
| 130 |
+
"++trainer.optimizer.weight_decay=1e-08",
|
| 131 |
+
"++trainer.optimizer.fused=true",
|
| 132 |
+
"++trainer.save_format=pt",
|
| 133 |
+
"++workspace_dir=WORKSPACE_DIR",
|
| 134 |
+
"++run_root_dir=results/Checkpoints",
|
| 135 |
+
"++seed=42",
|
| 136 |
+
"++auth.env_file=null",
|
| 137 |
+
"++auth.hf_token_env=HF_TOKEN",
|
| 138 |
+
"++auth.wandb_api_key_env=WANDB_API_KEY",
|
| 139 |
+
"++paths.run_root_dir=results/Checkpoints",
|
| 140 |
+
"++paths.dataset_local_dir=data/memory/flappy_fix_latency_3_200ep_7k2steps",
|
| 141 |
+
"++paths.dataset_cache_dir=null",
|
| 142 |
+
"++paths.base_model_dir=playground/Pretrained_models/Qwen3-VL-4B-Instruct",
|
| 143 |
+
"++paths.accelerate_config=starVLA/config/deepseeds/deepspeed_zero2.yaml",
|
| 144 |
+
"++rl_games.model_alias=openvla",
|
| 145 |
+
"++rl_games.env_eval.image_size=224",
|
| 146 |
+
"++rl_games.env_eval.frameskip=1",
|
| 147 |
+
"++rl_games.env_eval.image_transform=raw_rgb",
|
| 148 |
+
"++rl_games.env_eval.prompt_mode=raw",
|
| 149 |
+
"++rl_games.env_eval.ghost_trail.history_frames=5",
|
| 150 |
+
"++rl_games.env_eval.ghost_trail.gamma=1.3",
|
| 151 |
+
"++rl_games.env_eval.ghost_trail.min_alpha=35",
|
| 152 |
+
"++rl_games.env_eval.ghost_trail.scroll_px_per_step=4.0",
|
| 153 |
+
"++rl_games.env_eval.ghost_trail.ground_fraction=0.22",
|
| 154 |
+
"++rl_games.env_eval.seed=42",
|
| 155 |
+
"++rl_games.env_eval.fixed_episode_seeds=true",
|
| 156 |
+
"++rl_games.env_eval.latency_seed_stride=0",
|
| 157 |
+
"++rl_games.env_eval.task_seed_stride=0",
|
| 158 |
+
"++rl_games.env_eval.task_description=",
|
| 159 |
+
"++rl_games.env_eval.eval_parallel_envs=5",
|
| 160 |
+
"++rl_games.env_eval.action_chunk_execution.enabled=false",
|
| 161 |
+
"++rl_games.env_eval.action_chunk_execution.chunk_size=null",
|
| 162 |
+
"++rl_games.env_eval.enabled=true",
|
| 163 |
+
"++rl_games.env_eval.eval_backend=latency_bench",
|
| 164 |
+
"++rl_games.env_eval.distributed_mode=rank_sharded",
|
| 165 |
+
"++rl_games.env_eval.vectorized.enabled=false",
|
| 166 |
+
"++rl_games.env_eval.vectorized.batch_size=1",
|
| 167 |
+
"++rl_games.env_eval.latency.prompt_map_path=null",
|
| 168 |
+
"++rl_games.env_eval.latency.mode=single",
|
| 169 |
+
"++rl_games.env_eval.latency.values=[0]",
|
| 170 |
+
"++rl_games.env_eval.mid_train.enabled=true",
|
| 171 |
+
"++rl_games.env_eval.mid_train.interval_steps=250",
|
| 172 |
+
"++rl_games.env_eval.mid_train.latencies=[3]",
|
| 173 |
+
"++rl_games.env_eval.mid_train.num_episodes=20",
|
| 174 |
+
"++rl_games.env_eval.mid_train.max_steps_per_episode=3600",
|
| 175 |
+
"++rl_games.env_eval.post_train.enabled=true",
|
| 176 |
+
"++rl_games.env_eval.post_train.latencies=[3]",
|
| 177 |
+
"++rl_games.env_eval.post_train.num_episodes=50",
|
| 178 |
+
"++rl_games.env_eval.post_train.max_steps_per_episode=3600",
|
| 179 |
+
"++rl_games.task=flappy",
|
| 180 |
+
"++rl_games.initialization_mode=bridge",
|
| 181 |
+
"++rl_games.action_carrier=bridge",
|
| 182 |
+
"++bridge_base_model.repo_id.openvla=Qwen/Qwen3-VL-4B-Instruct",
|
| 183 |
+
"++bridge_base_model.repo_id.pi0=StarVLA/Qwen2.5-VL-3B-Instruct-Action",
|
| 184 |
+
"++bridge_base_model.repo_id.pi05=Qwen/Qwen3-VL-4B-Instruct",
|
| 185 |
+
"++bridge_base_model.repo_id.gr00t=Qwen/Qwen3-VL-4B-Instruct",
|
| 186 |
+
"++bridge_base_model.local_dir.openvla=playground/Pretrained_models/Qwen3-VL-4B-Instruct",
|
| 187 |
+
"++bridge_base_model.local_dir.pi0=playground/Pretrained_models/Qwen2.5-VL-3B-Instruct-Action",
|
| 188 |
+
"++bridge_base_model.local_dir.pi05=playground/Pretrained_models/Qwen3-VL-4B-Instruct",
|
| 189 |
+
"++bridge_base_model.local_dir.gr00t=playground/Pretrained_models/Qwen3-VL-4B-Instruct",
|
| 190 |
+
"++checkpoint.load=auto",
|
| 191 |
+
"++checkpoint.hf_repo_id=null",
|
| 192 |
+
"++checkpoint.save_best_model=false",
|
| 193 |
+
"++checkpoint.save_final_model=true",
|
| 194 |
+
"++checkpoint.save_pt_file=false",
|
| 195 |
+
"++checkpoint.save_training_state=true",
|
| 196 |
+
"++checkpoint.save_safetensors_file=true",
|
| 197 |
+
"++checkpoint.local.keep_last_n=1",
|
| 198 |
+
"++checkpoint.sync.enabled=false",
|
| 199 |
+
"++checkpoint.sync.repo_id=null",
|
| 200 |
+
"++checkpoint.sync.keep_last_n=0",
|
| 201 |
+
"++checkpoint.sync.sync_every_n_checkpoints=1",
|
| 202 |
+
"++checkpoint.sync.resume_policy=local_latest",
|
| 203 |
+
"++run_id=flappy_fix_latency_3_200ep_7k2steps_single_baseline",
|
| 204 |
+
"++output_dir=null",
|
| 205 |
+
"++config_yaml=null",
|
| 206 |
+
"++is_debug=false",
|
| 207 |
+
"++version_id=0.21",
|
| 208 |
+
"++run_root_dir=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints",
|
| 209 |
+
"++trainer.is_resume=true",
|
| 210 |
+
"++trainer.pretrained_checkpoint=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/checkpoints/steps_4000_state",
|
| 211 |
+
"++trainer.resume_step=4000",
|
| 212 |
+
"++datasets.vla_data.data_root_dir=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/data/memory/flappy_fix_latency_3_200ep_7k2steps",
|
| 213 |
+
"++datasets.vla_data.data_mix=flappy_train__bridge",
|
| 214 |
+
"++datasets.vla_data.eval_data_mix=flappy_train__bridge__val",
|
| 215 |
+
"++framework.qwenvl.base_vlm=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/playground/Pretrained_models/Qwen3-VL-4B-Instruct",
|
| 216 |
+
"++rl_games.env_eval.latency.prompt_map_path=/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/data/memory/flappy_fix_latency_3_200ep_7k2steps/flappy_train__bridge/latency_prompt_map.json"
|
| 217 |
+
],
|
| 218 |
+
"program": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/starVLA/training/train_starvla_hydra.py",
|
| 219 |
+
"codePath": "starVLA/training/train_starvla_hydra.py",
|
| 220 |
+
"codePathLocal": "starVLA/training/train_starvla_hydra.py",
|
| 221 |
+
"git": {
|
| 222 |
+
"remote": "git@github.com:talha1503/starVLA.git",
|
| 223 |
+
"commit": "9662ad113a8910207f7a17b0faab3480c78756f1"
|
| 224 |
+
},
|
| 225 |
+
"email": "zihanwang2029@u.northwestern.edu",
|
| 226 |
+
"root": "/lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb",
|
| 227 |
+
"host": "pool0-01926",
|
| 228 |
+
"executable": "/lustre/fsw/portfolios/nvr/users/zihwang/miniconda3/envs/starvla_rl_games_openvla/bin/python",
|
| 229 |
+
"cpu_count": 64,
|
| 230 |
+
"cpu_count_logical": 128,
|
| 231 |
+
"gpu": "NVIDIA H100 80GB HBM3",
|
| 232 |
+
"gpu_count": 2,
|
| 233 |
+
"disk": {
|
| 234 |
+
"/": {
|
| 235 |
+
"total": "7651200073728",
|
| 236 |
+
"used": "154768150528"
|
| 237 |
+
}
|
| 238 |
+
},
|
| 239 |
+
"memory": {
|
| 240 |
+
"total": "2164170506240"
|
| 241 |
+
},
|
| 242 |
+
"gpu_nvidia": [
|
| 243 |
+
{
|
| 244 |
+
"name": "NVIDIA H100 80GB HBM3",
|
| 245 |
+
"memoryTotal": "85520809984",
|
| 246 |
+
"cudaCores": 16896,
|
| 247 |
+
"architecture": "Hopper",
|
| 248 |
+
"uuid": "GPU-b261342c-53a6-b99a-1a4b-a3f366f9f4a6"
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"name": "NVIDIA H100 80GB HBM3",
|
| 252 |
+
"memoryTotal": "85520809984",
|
| 253 |
+
"cudaCores": 16896,
|
| 254 |
+
"architecture": "Hopper",
|
| 255 |
+
"uuid": "GPU-22f375ea-c8aa-4243-7b68-6be57bc3e864"
|
| 256 |
+
}
|
| 257 |
+
],
|
| 258 |
+
"cudaVersion": "12.2",
|
| 259 |
+
"slurm": {
|
| 260 |
+
"array_job_id": "14463017",
|
| 261 |
+
"array_task_count": "12",
|
| 262 |
+
"array_task_id": "0",
|
| 263 |
+
"array_task_max": "11",
|
| 264 |
+
"array_task_min": "0",
|
| 265 |
+
"array_task_step": "1",
|
| 266 |
+
"cluster_name": "cw-dfw-cs-001",
|
| 267 |
+
"conf": "/cm/shared/apps/slurm/var/etc/cw-dfw-cs-001/slurm.conf",
|
| 268 |
+
"cpus_on_node": "64",
|
| 269 |
+
"cpus_per_task": "64",
|
| 270 |
+
"gpus_on_node": "2",
|
| 271 |
+
"gtids": "0",
|
| 272 |
+
"job_account": "nvr_lacr_llm",
|
| 273 |
+
"job_cpus_per_node": "64",
|
| 274 |
+
"job_end_time": "1785236481",
|
| 275 |
+
"job_gid": "30",
|
| 276 |
+
"job_gpus": "2,3",
|
| 277 |
+
"job_id": "14463029",
|
| 278 |
+
"job_name": "mem-train",
|
| 279 |
+
"job_nodelist": "pool0-01926",
|
| 280 |
+
"job_num_nodes": "1",
|
| 281 |
+
"job_partition": "batch",
|
| 282 |
+
"job_qos": "normal",
|
| 283 |
+
"job_start_time": "1785222081",
|
| 284 |
+
"job_uid": "159489",
|
| 285 |
+
"job_user": "zihwang",
|
| 286 |
+
"jobid": "14463029",
|
| 287 |
+
"localid": "0",
|
| 288 |
+
"mem_per_node": "524288",
|
| 289 |
+
"nnodes": "1",
|
| 290 |
+
"nodeid": "0",
|
| 291 |
+
"nodelist": "pool0-01926",
|
| 292 |
+
"nprocs": "1",
|
| 293 |
+
"ntasks": "1",
|
| 294 |
+
"prio_process": "0",
|
| 295 |
+
"procid": "0",
|
| 296 |
+
"submit_dir": "/home/zihwang/projects",
|
| 297 |
+
"submit_host": "cw-dfw-cs-001-vscode-01",
|
| 298 |
+
"task_pid": "1554288",
|
| 299 |
+
"tasks_per_node": "1",
|
| 300 |
+
"topology_addr": "C1.S1.L333-DH4.pool0-01926",
|
| 301 |
+
"topology_addr_pattern": "switch.switch.switch.node",
|
| 302 |
+
"tres_per_task": "cpu=64"
|
| 303 |
+
},
|
| 304 |
+
"writerId": "e7ie5hfx89xkbqr9hcxqv4i7dcega7uq"
|
| 305 |
+
}
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/files/wandb-summary.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"_wandb":{"runtime":17},"rl_games_eval/post_train/macro_mean_reward":2.0000000447034836,"rl_games_eval/post_train/flappy__latency_3/mean_length":31,"rl_games_eval/post_train/flappy__latency_3/num_episodes":50,"rl_games_eval/post_train/mean_reward":2.0000000447034836,"_step":4000,"rl_games_eval/post_train/macro_mean_length":31,"rl_games_eval/post_train/task_count":1,"rl_games_eval/post_train/std_reward":0,"rl_games_eval/post_train/total_episodes":50,"_runtime":17.558658153,"rl_games_eval/post_train/std_length":0,"rl_games_eval/post_train/flappy__latency_3/std_length":0,"rl_games_eval/post_train/flappy__latency_3/mean_reward":2.0000000447034836,"_timestamp":1.7852222505772972e+09,"rl_games_eval/post_train/mean_length":31,"rl_games_eval/post_train/flappy__latency_3/std_reward":0}
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug-core.log
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"time":"2026-07-28T00:03:52.155670374-07:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmptavoalup/port-1555399.txt","pid":1555399,"detached":false,"idle-timeout":600000000000,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false,"enable-dcgm-profiling":false}
|
| 2 |
+
{"time":"2026-07-28T00:03:52.156043903-07:00","level":"INFO","msg":"server: will exit if parent process dies","ppid":1555399}
|
| 3 |
+
{"time":"2026-07-28T00:03:52.156041301-07:00","level":"INFO","msg":"server: accepting connections","addr":{"Name":"/tmp/wandb-1555399-1556797-2040693255/socket","Net":"unix"}}
|
| 4 |
+
{"time":"2026-07-28T00:03:52.311689991-07:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"1(@)"}
|
| 5 |
+
{"time":"2026-07-28T00:03:52.321786694-07:00","level":"INFO","msg":"handleInformInit: received","streamId":"crq5c83a","id":"1(@)"}
|
| 6 |
+
{"time":"2026-07-28T00:03:52.680432954-07:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"crq5c83a","id":"1(@)"}
|
| 7 |
+
{"time":"2026-07-28T00:03:58.405196063-07:00","level":"INFO","msg":"connection: cancelling request","id":"1(@)","requestId":"srnlhn5lbdko"}
|
| 8 |
+
{"time":"2026-07-28T00:04:10.578567883-07:00","level":"INFO","msg":"connection: cancelling request","id":"1(@)","requestId":"srnlhn5lbdko"}
|
| 9 |
+
{"time":"2026-07-28T00:04:11.655584761-07:00","level":"INFO","msg":"connection: cancelling request","id":"1(@)","requestId":"srnlhn5lbdko"}
|
| 10 |
+
{"time":"2026-07-28T00:04:11.656271641-07:00","level":"INFO","msg":"handleInformFinish: finish message received","streamId":"crq5c83a","id":"1(@)"}
|
| 11 |
+
{"time":"2026-07-28T00:04:11.658162491-07:00","level":"INFO","msg":"handleInformFinish: stream closed","streamId":"crq5c83a","id":"1(@)"}
|
| 12 |
+
{"time":"2026-07-28T00:04:12.352713846-07:00","level":"INFO","msg":"handleInformTeardown: server teardown initiated","id":"1(@)"}
|
| 13 |
+
{"time":"2026-07-28T00:04:12.352724307-07:00","level":"INFO","msg":"handleInformTeardown: server shutdown complete","id":"1(@)"}
|
| 14 |
+
{"time":"2026-07-28T00:04:12.352732074-07:00","level":"INFO","msg":"server is shutting down"}
|
| 15 |
+
{"time":"2026-07-28T00:04:12.35273272-07:00","level":"INFO","msg":"connection: closing","id":"1(@)"}
|
| 16 |
+
{"time":"2026-07-28T00:04:12.352762565-07:00","level":"INFO","msg":"connection: closed successfully","id":"1(@)"}
|
| 17 |
+
{"time":"2026-07-28T00:04:12.352772861-07:00","level":"INFO","msg":"connection: ManageConnectionData: connection closed","id":"1(@)"}
|
| 18 |
+
{"time":"2026-07-28T00:04:12.35279264-07:00","level":"INFO","msg":"server: listener closed","addr":{"Name":"/tmp/wandb-1555399-1556797-2040693255/socket","Net":"unix"}}
|
| 19 |
+
{"time":"2026-07-28T00:04:12.352813776-07:00","level":"INFO","msg":"server is closed"}
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug-internal.log
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"time":"2026-07-28T00:03:52.322546501-07:00","level":"INFO","msg":"wandb-core"}
|
| 2 |
+
{"time":"2026-07-28T00:03:52.324769707-07:00","level":"INFO","msg":"stream: starting","core version":"0.26.1"}
|
| 3 |
+
{"time":"2026-07-28T00:03:52.679302814-07:00","level":"INFO","msg":"stream: created new stream","id":"crq5c83a"}
|
| 4 |
+
{"time":"2026-07-28T00:03:52.679374536-07:00","level":"INFO","msg":"handler: started"}
|
| 5 |
+
{"time":"2026-07-28T00:03:52.68041931-07:00","level":"INFO","msg":"stream: started"}
|
| 6 |
+
{"time":"2026-07-28T00:03:52.680433695-07:00","level":"INFO","msg":"writer: started","stream_id":"crq5c83a"}
|
| 7 |
+
{"time":"2026-07-28T00:03:52.680445074-07:00","level":"INFO","msg":"sender: started"}
|
| 8 |
+
{"time":"2026-07-28T00:03:53.408326082-07:00","level":"INFO","msg":"filestream: sending request","total_files":1,"console_offset":0,"console_lines":1}
|
| 9 |
+
{"time":"2026-07-28T00:03:53.729203676-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 10 |
+
{"time":"2026-07-28T00:04:08.408945687-07:00","level":"INFO","msg":"filestream: sending request","total_files":2,"events_offset":0,"events_lines":2,"console_offset":1,"console_lines":24,"uploaded_len":2}
|
| 11 |
+
{"time":"2026-07-28T00:04:08.694549009-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 12 |
+
{"time":"2026-07-28T00:04:11.439600789-07:00","level":"INFO","msg":"fileTransfer: Close: file transfer manager closed"}
|
| 13 |
+
{"time":"2026-07-28T00:04:11.439686068-07:00","level":"INFO","msg":"filestream: sending request","total_files":3,"history_offset":0,"history_lines":1,"console_offset":25,"console_lines":12,"uploaded_len":3,"complete":true,"exit_code":0}
|
| 14 |
+
{"time":"2026-07-28T00:04:11.653219659-07:00","level":"INFO","msg":"filestream: request sent","status":"200 OK"}
|
| 15 |
+
{"time":"2026-07-28T00:04:11.653793321-07:00","level":"INFO","msg":"handler: operation stats","stats":{}}
|
| 16 |
+
{"time":"2026-07-28T00:04:11.656282217-07:00","level":"INFO","msg":"stream: finishing up"}
|
| 17 |
+
{"time":"2026-07-28T00:04:11.656308039-07:00","level":"INFO","msg":"handler: closed"}
|
| 18 |
+
{"time":"2026-07-28T00:04:11.656990166-07:00","level":"INFO","msg":"sender: closed"}
|
| 19 |
+
{"time":"2026-07-28T00:04:11.656998448-07:00","level":"INFO","msg":"stream: all finished"}
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug.log
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_setup.py:_flush():81] Current SDK version is 0.26.1
|
| 2 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_setup.py:_flush():81] Configure stats pid to 1555399
|
| 3 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_setup.py:_flush():81] Loading settings from environment variables
|
| 4 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:setup_run_log_directory():723] Logging user logs to /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug.log
|
| 5 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:setup_run_log_directory():724] Logging internal logs to /lustre/fsw/portfolios/nvr/projects/nvr_lacr_llm/users/zihwang/latency-sensitive-bench/third_party/starVLA/results/Checkpoints/flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/logs/debug-internal.log
|
| 6 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:init():850] calling init triggers
|
| 7 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:init():855] wandb.init called with sweep_config: {}
|
| 8 |
+
config: {'_wandb': {}}
|
| 9 |
+
2026-07-28 00:03:51,649 INFO MainThread:1555399 [wandb_init.py:init():898] starting backend
|
| 10 |
+
2026-07-28 00:03:52,312 INFO MainThread:1555399 [wandb_init.py:init():913] sending inform_init request
|
| 11 |
+
2026-07-28 00:03:52,680 INFO MainThread:1555399 [wandb_init.py:init():918] backend started and connected
|
| 12 |
+
2026-07-28 00:03:52,683 INFO MainThread:1555399 [wandb_init.py:init():988] updated telemetry
|
| 13 |
+
2026-07-28 00:03:52,713 INFO MainThread:1555399 [wandb_init.py:init():1011] communicating run to backend with 90.0 second timeout
|
| 14 |
+
2026-07-28 00:03:53,019 INFO MainThread:1555399 [wandb_init.py:init():1056] starting run threads in backend
|
| 15 |
+
2026-07-28 00:03:53,397 INFO MainThread:1555399 [wandb_run.py:_console_start():2554] atexit reg
|
| 16 |
+
2026-07-28 00:03:53,397 INFO MainThread:1555399 [wandb_run.py:_redirect():2403] redirect: wrap_raw
|
| 17 |
+
2026-07-28 00:03:53,398 INFO MainThread:1555399 [wandb_run.py:_redirect():2472] Wrapping output streams.
|
| 18 |
+
2026-07-28 00:03:53,398 INFO MainThread:1555399 [wandb_run.py:_redirect():2495] Redirects installed.
|
| 19 |
+
2026-07-28 00:03:53,407 INFO MainThread:1555399 [wandb_init.py:init():1094] run started, returning control to user process
|
| 20 |
+
2026-07-28 00:04:10,577 INFO MainThread:1555399 [wandb_run.py:_finish():2321] finishing run zihanwang-ai-northwestern-university/starVLA_rl_games/crq5c83a
|
| 21 |
+
2026-07-28 00:04:10,577 INFO MainThread:1555399 [wandb_run.py:_atexit_cleanup():2520] got exitcode: 0
|
| 22 |
+
2026-07-28 00:04:10,578 INFO MainThread:1555399 [wandb_run.py:_restore():2502] restore
|
| 23 |
+
2026-07-28 00:04:10,578 INFO MainThread:1555399 [wandb_run.py:_restore():2508] restore done
|
| 24 |
+
2026-07-28 00:04:11,655 INFO MainThread:1555399 [wandb_run.py:_footer_sync_info():3889] logging synced files
|
flappy_fix_latency_3_200ep_7k2steps_single_baseline/wandb/wandb/run-20260728_000351-crq5c83a/run-crq5c83a.wandb
ADDED
|
Binary file (22.3 kB). View file
|
|
|