| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Current SDK version is 0.17.8 | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Configure stats pid to 1395793 | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Loading settings from /home/CORP/shell.hu/.config/wandb/settings | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Loading settings from /home/CORP/shell.hu/robot_dir/graspformer/wandb/settings | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Loading settings from environment variables: {} | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Applying setup settings: {'_disable_service': False} | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Inferring run settings from compute environment: {'program_relpath': 'main.py', 'program_abspath': '/home/CORP/shell.hu/robot_dir/graspformer/main.py', 'program': '/home/CORP/shell.hu/robot_dir/graspformer/main.py'} | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_setup.py:_flush():77] Applying login settings: {} | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_init.py:_log_setup():524] Logging user logs to /scratch-01/shell/graspnet/outputs/45_2_scratch/wandb/run-20251126_182725-shygb0f5/logs/debug.log | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_init.py:_log_setup():525] Logging internal logs to /scratch-01/shell/graspnet/outputs/45_2_scratch/wandb/run-20251126_182725-shygb0f5/logs/debug-internal.log | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_init.py:init():607] calling init triggers | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_init.py:init():614] wandb.init called with sweep_config: {} | |
| config: {'dataset': {'name': 'grasp_dataset', 'version': 'interpolation4d_v14', 'num_episodes': 10000, 'num_points': 64, 'rank_topk': 32, 'camera': 'realsense', 'fric_coef_thresh': 0.3, 'img_size': 1024, 'min_score': 0.9, 'exp_type': 'normal', 'depth_fill_mode': 'left', 'label_mode': 'soft'}, 'model': {'name': 'graspformer', 'pretrained': 'ckpts/sam_vit_l_0b3195.pth', 'fix_img_en': True, 'fix_prompt_en': False, 'fix_mask_de': False, 'decompose': False, 'early_fusion': True, 'late_fusion': False, 'vit_type': 'vit_l', 'prompt_embed_dim': 256, 'image_size': 1024, 'vit_patch_size': 16, 'transformer_depth': 2, 'transformer_mlp_dim': 2048, 'transformer_num_heads': 8, 'pixel_mean': [123.675, 116.28, 103.53], 'pixel_std': [58.395, 57.12, 57.375], 'num_views': 300, 'num_rots': 12, 'num_gds': 4, 'decomp_views': 30, 'decomp_rots': 4, 'decomp_gds': 2, 'str_pn_radius': 500, 'str_pn_nsample': 256, 'str_pn_mlp': [64, 128, 256], 'str_pn_sort': True, 'fusion_coeff': 1.0, 'first_pn': 'structured', 'pn_input': 'depth_pcl', 'pn_sa_mlp': [[32, 64, 128], [64, 64, 128], [128, 128, 256], [256, 256, 512]], 'pn_fp_mlp': [[256, 256], [256, 256], [256, 256]], 'fusion_mode': 'add', 'depth_coeff': 1.0, 'img_coeff': 1.0, 'late_pn_radius': 500, 'late_pn_nsample': 256, 'late_pn_mlp': [64, 128, 256], 'late_pn_sort': True, 'query_roi_size': 4, 'loss': ['focal'], 'rank_mode': 'none'}, 'wandb': {'project': 'graspformer', 'entity': 'peize', 'name': 'exp45_2', 'mode': 'online'}, 'mode': 'train', 'data_loader': {'train': {'num_workers': 4, 'persistent_workers': True, 'batch_size': 1, 'seed': 1234, 'split': 'train'}, 'test': {'num_workers': 4, 'persistent_workers': True, 'batch_size': 1, 'seed': 2345, 'split': 'single_scene'}, 'val': {'num_workers': 4, 'persistent_workers': True, 'batch_size': 1, 'seed': 3456, 'split': 'train', 'limit_val_batches': 44}}, 'optimizer': {'lr': 0.001, 'warm_up_steps': 0, 'milestones': [2000, 4000, 6000, 8000, 10000, 12000, 14000], 'gamma': 0.8}, 'checkpointing': {'continue_training': False, 'load': '', 'every_n_train_steps': 200, 'save_top_k': 4}, 'train': {'output_path': 'outputs/train', 'extended_visualization': False}, 'test': {'output_path': 'outputs/test', 'metric_thresh': 0.3}, 'seed': 111123, 'trainer': {'max_steps': 1000000, 'val_check_interval': 3200, 'devices': 1, 'accumulate_grad_batches': 16, 'gradient_clip_val': 0.5}} | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_init.py:init():657] starting backend | |
| 2025-11-26 18:27:25,316 INFO MainThread:1395793 [wandb_init.py:init():661] setting up manager | |
| 2025-11-26 18:27:25,317 INFO MainThread:1395793 [backend.py:_multiprocessing_setup():105] multiprocessing start_methods=fork,spawn,forkserver, using: spawn | |
| 2025-11-26 18:27:25,319 INFO MainThread:1395793 [wandb_init.py:init():669] backend started and connected | |
| 2025-11-26 18:27:25,325 INFO MainThread:1395793 [wandb_init.py:init():767] updated telemetry | |
| 2025-11-26 18:27:25,330 INFO MainThread:1395793 [wandb_init.py:init():800] communicating run to backend with 90.0 second timeout | |
| 2025-11-26 18:27:26,412 INFO MainThread:1395793 [wandb_init.py:init():851] starting run threads in backend | |
| 2025-11-26 18:27:40,287 INFO MainThread:1395793 [wandb_run.py:_console_start():2463] atexit reg | |
| 2025-11-26 18:27:40,287 INFO MainThread:1395793 [wandb_run.py:_redirect():2309] redirect: wrap_raw | |
| 2025-11-26 18:27:40,287 INFO MainThread:1395793 [wandb_run.py:_redirect():2374] Wrapping output streams. | |
| 2025-11-26 18:27:40,287 INFO MainThread:1395793 [wandb_run.py:_redirect():2399] Redirects installed. | |
| 2025-11-26 18:27:40,290 INFO MainThread:1395793 [wandb_init.py:init():894] run started, returning control to user process | |
| 2025-12-08 20:03:21,302 WARNING MsgRouterThr:1395793 [router.py:message_loop():77] message_loop has been closed | |