wandb/debug.log

2026-03-30 00:32:53,293 INFO    MainThread:3328778 [wandb_setup.py:_flush():81] Current SDK version is 0.25.1
2026-03-30 00:32:53,293 INFO    MainThread:3328778 [wandb_setup.py:_flush():81] Configure stats pid to 3328778
2026-03-30 00:32:53,293 INFO    MainThread:3328778 [wandb_setup.py:_flush():81] Loading settings from environment variables
2026-03-30 00:32:53,294 INFO    MainThread:3328778 [wandb_init.py:setup_run_log_directory():717] Logging user logs to /home/jovyan/youngjoonjeong/latent_action/outputs/token_fdm_duration/dino_suite_all_k20_extra4/libero_cosmos_policy/2026-03-30/00-32-46/wandb/run-20260330_003253-c8rq5onq/logs/debug.log
2026-03-30 00:32:53,294 INFO    MainThread:3328778 [wandb_init.py:setup_run_log_directory():718] Logging internal logs to /home/jovyan/youngjoonjeong/latent_action/outputs/token_fdm_duration/dino_suite_all_k20_extra4/libero_cosmos_policy/2026-03-30/00-32-46/wandb/run-20260330_003253-c8rq5onq/logs/debug-internal.log
2026-03-30 00:32:53,294 INFO    MainThread:3328778 [wandb_init.py:init():844] calling init triggers
2026-03-30 00:32:53,294 INFO    MainThread:3328778 [wandb_init.py:init():849] wandb.init called with sweep_config: {}
config: {'data': {'name': 'libero_cosmos_policy', 'type': 'mg', 'backend': 'robomimic', 'paths': ['/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/KITCHEN_SCENE4_put_the_black_bowl_in_the_bottom_drawer_of_the_cabinet_and_close_it_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/KITCHEN_SCENE6_put_the_yellow_and_white_mug_in_the_microwave_and_close_it_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/KITCHEN_SCENE8_put_both_moka_pots_on_the_stove_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/LIVING_ROOM_SCENE1_put_both_the_alphabet_soup_and_the_cream_cheese_box_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/LIVING_ROOM_SCENE2_put_both_the_alphabet_soup_and_the_tomato_sauce_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/LIVING_ROOM_SCENE2_put_both_the_cream_cheese_box_and_the_butter_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/LIVING_ROOM_SCENE5_put_the_white_mug_on_the_left_plate_and_put_the_yellow_and_white_mug_on_the_right_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/LIVING_ROOM_SCENE6_put_the_white_mug_on_the_plate_and_put_the_chocolate_pudding_to_the_right_of_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_10_regen/STUDY_SCENE1_pick_up_the_book_and_place_it_in_the_back_compartment_of_the_caddy_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/open_the_middle_drawer_of_the_cabinet_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/open_the_top_drawer_and_put_the_bowl_inside_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/push_the_plate_to_the_front_of_the_stove_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/put_the_bowl_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/put_the_bowl_on_the_stove_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/put_the_bowl_on_top_of_the_cabinet_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/put_the_cream_cheese_in_the_bowl_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/put_the_wine_bottle_on_the_rack_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/put_the_wine_bottle_on_top_of_the_cabinet_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_goal_regen/turn_on_the_stove_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_alphabet_soup_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_bbq_sauce_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_butter_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_chocolate_pudding_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_cream_cheese_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_ketchup_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_milk_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_orange_juice_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_salad_dressing_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_object_regen/pick_up_the_tomato_sauce_and_place_it_in_the_basket_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_between_the_plate_and_the_ramekin_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_from_table_center_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_in_the_top_drawer_of_the_wooden_cabinet_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_next_to_the_cookie_box_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_next_to_the_plate_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_next_to_the_ramekin_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_on_the_cookie_box_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_on_the_ramekin_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate_demo.hdf5', '/home/jovyan/.cache/huggingface/hub/datasets--nvidia--LIBERO-Cosmos-Policy/snapshots/b463cbd5d114909e4daea65eb03bed9787290042/success_only/libero_spatial_regen/pick_up_the_black_bowl_on_the_wooden_cabinet_and_place_it_on_the_plate_demo.hdf5'], 'task_suite_name': None, 'observations_keys': ['image'], 'observation_source_keys': {'image': 'agentview_rgb_jpeg'}, 'action_dim': 7, 'frame_stack': 1, 'horizon': 1, 'temporal_index_mode': 'clip', 'use_proprio': False, 'proprio_source': 'robot_states', 'image_chw': [3, 224, 224], 'image_value_range': 'zero_to_one', 'image_transport_dtype': 'uint8', 'duration_focus': None, 'hf': {'repo_id': 'nvidia/LIBERO-Cosmos-Policy', 'repo_type': 'dataset', 'allow_patterns': ['success_only/*_regen/*.hdf5'], 'local_files_only': False}, 'action_sequence_targets': {'enabled': True, 'horizons': [20], 'target_key': 'gt_action_seq_max'}, 'temporal': {'max_offset': 20, 'anchor_mode': 'fixed', 'extra_random_count': 4, 'load_full_context': False}, 'multiscale': {'enabled': True}, 'use_aug': False, 'emit_states': False, 'emit_duration_label': False, 'emit_future_offset_fields': False, 'sample_transform': {'weak_augmentation': {'enabled': False}}}, 'train': {'optimizer': {'type': 'adamw', 'lr': 0.0001, 'weight_decay': 1e-06}, 'batch_size': 256, 'num_workers': 4, 'prefetch_factor': None, 'log_interval': 100, 'total_steps': 5000, 'eval_interval': 500, 'use_aug': True, 'aug_mode': 'default', 'use_amp': True, 'device_transfer_non_blocking': True, 'seed': 42, 'scheduler': {'type': 'cosine', 'warmup_steps': 500, 'num_cycles': 0.5, 'min_lr_scale': 0.0}, 'decoder_dataloader': {'enabled': False, 'batch_size': 64, 'num_workers': 4, 'prefetch_factor': None, 'shuffle': True}, 'stages': {'stage1': True, 'stage2': False, 'stage3': False}, 'stage1': {'batch_size': 256, 'num_workers': 16, 'prefetch_factor': 2, 'log_interval': 100, 'total_steps': 20000, 'eval_interval': 500, 'optimizer': {'lr': 0.0001, 'weight_decay': 1e-06}}, 'stage2': {'batch_size': 256, 'num_workers': 4, 'prefetch_factor': None, 'log_interval': 100, 'total_steps': 5000, 'eval_interval': 500, 'latent_target': 'auto', 'optimizer': {'lr': 0.0001, 'weight_decay': 1e-06}, 'action_probe': {'enabled': True}}, 'stage3': {'batch_size': 256, 'num_workers': 4, 'prefetch_factor': None, 'log_interval': 100, 'total_steps': 5000, 'eval_interval': 500, 'final_eval_num_episodes': 100, 'log_eval_video': False, 'optimizer': {'lr': 0.0001, 'weight_decay': 1e-06}, 'trainable_scope': 'all', 'non_decoder_lr_scale': 1.0}, 'checkpoint': {'enabled': True, 'base_dir': None, 'save_interval': 1000, 'save_last': True, 'latest_only': True, 'load': {'stage1': {'mode': 'none', 'path': None}, 'stage2': {'mode': 'handoff', 'path': None, 'teacher_path': None}, 'stage3': {'mode': 'handoff', 'path': None}}}}, 'model': {'idm': {'type': 'token_idm', 'action_dim': 128, 'token_dim': 768, 'model_dim': 512, 'latent_dim': 32, 'num_action_tokens': 4, 'num_blocks': 4, 'num_heads': 8, 'dropout': 0.0}, 'fdm': {'type': 'token_fdm_duration', 'action_dim': 128, 'token_dim': 768, 'model_dim': 512, 'num_action_tokens': 4, 'num_blocks': 4, 'num_heads': 8, 'dropout': 0.0, 'prediction_mode': 'direct'}, 'hyperbolic_latent': {'enabled': False, 'backend': 'geoopt', 'manifold': 'poincare', 'curvature': 1.0, 'learnable_curvature': False, 'fdm_input_mode': 'logmap0', 'prelift_mode': 'none', 'prelift_scale': 1.0, 'lift_max_norm': 5.0, 'tangent_max_norm': 5.0, 'eps': 1e-06}, 'encoders': {'type': 'dino', 'image_key': 'image', 'model_id': 'facebook/dinov2-base', 'input_value_range': 'zero_to_one', 'freeze_backbone': True, 'drop_cls_token': True, 'output_dim': 768, 'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]}, 'type': 'dino_lam', 'idm_input': 'future', 'fdm_target': 'future', 'pixel_decoders': None, 'latent_action_decoders': None, 'temporal': {'max_offset': 20, 'anchor_mode': 'fixed', 'extra_random_count': 4, 'load_full_context': False}, 'multiscale_loss': {'enabled': True, 'prediction_mode': 'direct_duration', 'weight_mode': 'uniform', 'weights': {}, 'allow_plain_fdm_oneshot': True, 'direct_duration_backend': 'duration_aware', 'consistency': {'enabled': False, 'weight': 10.0, 'num_pairs': 4, 'sample_source': 'all_horizons', 'teacher_mode': 'direct_teacher'}}, 'branch_order': {'enabled': False, 'radial_weight': 0.0, 'local_radial_margin_weight': 0.0, 'local_radial_margin_alpha': 0.05, 'branch_weight': 0.0, 'z0_origin_weight': 0.0, 'prefix_weight': 0.0, 'radius_progress_weight': 1.0, 'radius_progress_mode': 'offset_margin', 'radius_progress_alpha': 0.02, 'branch_margin_deg': 10.0, 'branch_margin_rad': 0.17453292519943295, 'eps': 1e-06}}, 'objective': {'fdm_target': 'future', 'idm_input': 'future', 'multiscale': {'enabled': True, 'consistency': {'enabled': False, 'weight': 10.0, 'num_pairs': 4, 'sample_source': 'all_horizons', 'teacher_mode': 'direct_teacher'}, 'prediction_mode': 'direct_duration', 'allow_plain_fdm_oneshot': True, 'weight_mode': 'uniform', 'weights': {}}, 'temporal': {'max_offset': 20, 'anchor_mode': 'fixed', 'extra_random_count': 4, 'load_full_context': False}, 'branch_order': {'enabled': False, 'radial_weight': 0.0, 'local_radial_margin_weight': 0.0, 'local_radial_margin_alpha': 0.05, 'branch_weight': 0.0, 'z0_origin_weight': 0.0, 'prefix_weight': 0.0, 'radius_progress_weight': 1.0, 'radius_progress_mode': 'offset_margin', 'radius_progress_alpha': 0.02, 'branch_margin_deg': 10.0, 'eps': 1e-06}, 'latent_plan': {'enabled': False, 'total_horizon': 20, 'num_chunks': 1, 'chunk_dim': 128, 'plan_dim': 128}}, 'eval': {'type': 'robomimic', 'data_path': [], 'reset_mode': 'env_reset', 'num_eval_episodes': 20, 'max_steps': 500, 'record_video': True, 'checkpoint_path': None, 'checkpoint_strict': True, 'use_checkpoint_cfg': True, 'video_output_path': None, 'obs_keys': None, 'use_proprio': None, 'use_object': False, 'resize_hw': None, 'image_value_range': None, 'seed': 0}, 'agent': {'encoders': {'image': {'in_channels': 3, 'output_dim': 512, 'output_mode': 'global', 'type': 'resnet18', 'pretrained': False}, 'proprio': {'input_dim': 9, 'hidden_dim': 128, 'output_dim': 64}, 'type': 'group', 'modalities': ['agentview_image', 'robot0_eye_in_hand_image'], 'proj_dim': 128}, 'policies': {'decoder': {'type': 'mlp', 'hidden_dims': [256, 256]}, 'hidden_dims': [512, 512, 256, 64], 'action_dim': 256, 'emb_dim': 384, 'gt_action_dim': 7}, 'type': 'latent_action'}, 'probes': {'enabled': True, 'every': 10, 'steps_per_call': 1, 'sequence': {'enabled': True, 'horizons': [20], 'target_key': 'gt_action_seq_max'}, 'list': {'z_to_s_t': {'name': 'z_to_s_t', 'enabled': False, 'shuffle': True, 'type': 'regression', 'input': 'z_t', 'target': 's_t', 'loss': 'mse', 'lr': 0.001, 'every': 10, 'mlp': {'hidden_dims': [128, 64]}}, 'z_to_s_tp': {'name': 'z_to_s_tp', 'enabled': False, 'shuffle': True, 'type': 'regression', 'input': 'z_t', 'target': 's_tp', 'loss': 'mse', 'lr': 0.001, 'every': 10, 'mlp': {'hidden_dims': [128, 64]}}, 'z_to_action_t': {'name': 'z_to_action_t', 'probe_type': 'z_to_action', 'enabled': True, 'shuffle': True, 'type': 'regression', 'input': 'z_t', 'target': 'gt_action', 'loss': 'mse', 'lr': 0.001, 'every': 1, 'mlp': {'hidden_dims': [128, 64]}}, 'z_to_action_seq_h20': {'name': 'z_to_action_seq_h20', 'probe_type': 'z_to_action_sequence', 'enabled': True, 'shuffle': True, 'type': 'regression', 'input': 'z_t', 'target': 'gt_action_seq_max', 'sequence_horizon': 20, 'loss': 'mse', 'lr': 0.001, 'every': 10, 'mlp': {'hidden_dims': [128, 64]}, 'output_dim': 140}}}, 'logger': {'project': 'latent_action', 'run_name': 'latent_action_training', 'tags': ['debug']}, 'output_root': 'outputs/token_fdm_duration/dino_suite_all_k20_extra4', '_wandb': {}}
2026-03-30 00:32:53,294 INFO    MainThread:3328778 [wandb_init.py:init():892] starting backend
2026-03-30 00:32:53,517 INFO    MainThread:3328778 [wandb_init.py:init():895] sending inform_init request
2026-03-30 00:32:53,520 INFO    MainThread:3328778 [wandb_init.py:init():903] backend started and connected
2026-03-30 00:32:53,522 INFO    MainThread:3328778 [wandb_init.py:init():973] updated telemetry
2026-03-30 00:32:53,528 INFO    MainThread:3328778 [wandb_init.py:init():997] communicating run to backend with 90.0 second timeout
2026-03-30 00:32:54,296 INFO    MainThread:3328778 [wandb_init.py:init():1042] starting run threads in backend
2026-03-30 00:32:54,428 INFO    MainThread:3328778 [wandb_run.py:_console_start():2524] atexit reg
2026-03-30 00:32:54,428 INFO    MainThread:3328778 [wandb_run.py:_redirect():2373] redirect: wrap_raw
2026-03-30 00:32:54,428 INFO    MainThread:3328778 [wandb_run.py:_redirect():2442] Wrapping output streams.
2026-03-30 00:32:54,428 INFO    MainThread:3328778 [wandb_run.py:_redirect():2465] Redirects installed.
2026-03-30 00:32:54,430 INFO    MainThread:3328778 [wandb_init.py:init():1082] run started, returning control to user process
2026-03-30 07:54:52,811 INFO    MainThread:3328778 [wandb_run.py:_finish():2291] finishing run joonstack/latent_action_stage1/c8rq5onq
2026-03-30 07:54:52,812 INFO    MainThread:3328778 [wandb_run.py:_atexit_cleanup():2490] got exitcode: 0
2026-03-30 07:54:52,813 INFO    MainThread:3328778 [wandb_run.py:_restore():2472] restore
2026-03-30 07:54:52,813 INFO    MainThread:3328778 [wandb_run.py:_restore():2478] restore done
2026-03-30 07:54:53,935 INFO    MainThread:3328778 [wandb_run.py:_footer_sync_info():3868] logging synced files