| 2025-01-06 17:39:06,228 INFO StreamThr :2093159 [internal.py:wandb_internal():85] W&B internal server running at pid: 2093159, started at: 2025-01-06 17:39:06.228202 |
| 2025-01-06 17:39:06,229 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status |
| 2025-01-06 17:39:06,241 INFO WriterThread:2093159 [datastore.py:open_for_write():87] open: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/run-1jous172.wandb |
| 2025-01-06 17:39:06,241 DEBUG SenderThread:2093159 [sender.py:send():379] send: header |
| 2025-01-06 17:39:06,252 DEBUG SenderThread:2093159 [sender.py:send():379] send: run |
| 2025-01-06 17:39:06,511 INFO SenderThread:2093159 [dir_watcher.py:__init__():211] watching files in: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files |
| 2025-01-06 17:39:06,511 INFO SenderThread:2093159 [sender.py:_start_run_threads():1188] run started: 1jous172 with start time 1736203146.23001 |
| 2025-01-06 17:39:06,532 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: check_version |
| 2025-01-06 17:39:06,532 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: check_version |
| 2025-01-06 17:39:06,602 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: run_start |
| 2025-01-06 17:39:06,606 DEBUG HandlerThread:2093159 [system_info.py:__init__():26] System info init |
| 2025-01-06 17:39:06,607 DEBUG HandlerThread:2093159 [system_info.py:__init__():41] System info init done |
| 2025-01-06 17:39:06,607 INFO HandlerThread:2093159 [system_monitor.py:start():194] Starting system monitor |
| 2025-01-06 17:39:06,607 INFO SystemMonitor:2093159 [system_monitor.py:_start():158] Starting system asset monitoring threads |
| 2025-01-06 17:39:06,607 INFO HandlerThread:2093159 [system_monitor.py:probe():214] Collecting system info |
| 2025-01-06 17:39:06,608 INFO SystemMonitor:2093159 [interfaces.py:start():188] Started cpu monitoring |
| 2025-01-06 17:39:06,608 INFO SystemMonitor:2093159 [interfaces.py:start():188] Started disk monitoring |
| 2025-01-06 17:39:06,609 INFO SystemMonitor:2093159 [interfaces.py:start():188] Started gpu monitoring |
| 2025-01-06 17:39:06,609 INFO SystemMonitor:2093159 [interfaces.py:start():188] Started memory monitoring |
| 2025-01-06 17:39:06,610 INFO SystemMonitor:2093159 [interfaces.py:start():188] Started network monitoring |
| 2025-01-06 17:39:06,620 DEBUG HandlerThread:2093159 [system_info.py:probe():152] Probing system |
| 2025-01-06 17:39:06,622 DEBUG HandlerThread:2093159 [system_info.py:_probe_git():137] Probing git |
| 2025-01-06 17:39:06,627 DEBUG HandlerThread:2093159 [system_info.py:_probe_git():145] Probing git done |
| 2025-01-06 17:39:06,627 DEBUG HandlerThread:2093159 [system_info.py:probe():200] Probing system done |
| 2025-01-06 17:39:06,627 DEBUG HandlerThread:2093159 [system_monitor.py:probe():223] {'os': 'Linux-5.15.0-125-generic-x86_64-with-glibc2.31', 'python': '3.10.14', 'heartbeatAt': '2025-01-06T22:39:06.621054', 'startedAt': '2025-01-06T22:39:06.220965', 'docker': None, 'cuda': None, 'args': ('agent=diffusion', 'suite=frankagym', 'suite/frankagym_task@_global_=insertion'), 'state': 'running', 'program': '/home/leonmkim/fish_leon/FISH/eval_robot.py', 'codePathLocal': None, 'codePath': 'FISH/eval_robot.py', 'git': {'remote': 'https://github.com/leonmkim/fish_leon.git', 'commit': '22931875031f38fc79957f52ac8e49b1fb3d9fdc'}, 'email': 'leonmkim@seas.upenn.edu', 'root': '/home/leonmkim/fish_leon', 'host': 'leonmkim-ROG-Strix-G15CS-G15CS', 'username': 'leonmkim', 'executable': '/home/leonmkim/.pyenv/versions/lerobot/bin/python', 'cpu_count': 8, 'cpu_count_logical': 8, 'cpu_freq': {'current': 3374.93825, 'min': 800.0, 'max': 4700.0}, 'cpu_freq_per_core': [{'current': 3000.0, 'min': 800.0, 'max': 4700.0}, {'current': 3000.0, 'min': 800.0, 'max': 4700.0}, {'current': 3000.0, 'min': 800.0, 'max': 4700.0}, {'current': 3000.0, 'min': 800.0, 'max': 4700.0}, {'current': 4499.946, 'min': 800.0, 'max': 4700.0}, {'current': 3000.0, 'min': 800.0, 'max': 4700.0}, {'current': 3000.0, 'min': 800.0, 'max': 4700.0}, {'current': 3000.0, 'min': 800.0, 'max': 4700.0}], 'disk': {'/': {'total': 915.3232879638672, 'used': 353.0933418273926}}, 'gpu': 'NVIDIA GeForce RTX 2070 SUPER', 'gpu_count': 1, 'gpu_devices': [{'name': 'NVIDIA GeForce RTX 2070 SUPER', 'memory_total': 8589934592}], 'memory': {'total': 62.71595764160156}} |
| 2025-01-06 17:39:06,627 INFO HandlerThread:2093159 [system_monitor.py:probe():224] Finished collecting system info |
| 2025-01-06 17:39:06,627 INFO HandlerThread:2093159 [system_monitor.py:probe():227] Publishing system info |
| 2025-01-06 17:39:06,627 DEBUG HandlerThread:2093159 [system_info.py:_save_code():44] Saving code |
| 2025-01-06 17:39:06,629 DEBUG HandlerThread:2093159 [system_info.py:_save_code():67] Saving code done |
| 2025-01-06 17:39:06,629 DEBUG HandlerThread:2093159 [system_info.py:_save_patches():84] Saving git patches |
| 2025-01-06 17:39:06,730 DEBUG HandlerThread:2093159 [system_info.py:_save_patches():126] Saving git patches done |
| 2025-01-06 17:39:06,730 INFO HandlerThread:2093159 [system_monitor.py:probe():229] Finished publishing system info |
| 2025-01-06 17:39:06,733 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:39:06,733 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-metadata.json with policy now |
| 2025-01-06 17:39:06,733 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file code/FISH/eval_robot.py with policy now |
| 2025-01-06 17:39:06,734 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file diff.patch with policy now |
| 2025-01-06 17:39:06,969 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: python_packages |
| 2025-01-06 17:39:06,969 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: python_packages |
| 2025-01-06 17:39:06,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:39:06,970 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:39:06,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:06,972 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/40nlmnbq-wandb-metadata.json |
| 2025-01-06 17:39:06,972 INFO HandlerThread:2093159 [handler.py:handle_tbrecord():783] handling tbrecord: tbrecord { |
| log_dir: "/home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/tb" |
| save: true |
| } |
| control { |
| relay_id: "469e8a1cb67b419d92c8e71f617843fd" |
| } |
| _info { |
| stream_id: "1jous172" |
| } |
|
|
| 2025-01-06 17:39:06,973 DEBUG HandlerThread:2093159 [config_util.py:dict_from_config_file():70] no default config file found in config-defaults.yaml |
| 2025-01-06 17:39:07,025 INFO wandb-upload_1:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/il8kn86k-code/FISH/eval_robot.py |
| 2025-01-06 17:39:07,033 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/tz3pb0wm-diff.patch |
| 2025-01-06 17:39:07,042 DEBUG SenderThread:2093159 [sender.py:send():379] send: telemetry |
| 2025-01-06 17:39:07,042 DEBUG SenderThread:2093159 [sender.py:send():379] send: tbrecord |
| 2025-01-06 17:39:07,042 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:39:07,042 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 with policy live |
| 2025-01-06 17:39:07,226 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/gdyxm7b1-events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:39:07,513 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/code/FISH/eval_robot.py |
| 2025-01-06 17:39:07,513 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/diff.patch |
| 2025-01-06 17:39:07,513 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:39:07,513 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-metadata.json |
| 2025-01-06 17:39:07,514 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:39:07,514 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/requirements.txt |
| 2025-01-06 17:39:07,514 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/code/FISH |
| 2025-01-06 17:39:07,514 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/code |
| 2025-01-06 17:39:07,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:08,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:09,512 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:39:09,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:10,618 DEBUG SenderThread:2093159 [sender.py:send():379] send: config |
| 2025-01-06 17:39:10,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:11,627 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:11,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:12,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:13,514 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:39:13,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:14,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:15,514 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:39:15,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:16,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:16,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:17,515 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:39:17,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:18,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:19,515 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:39:19,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:20,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:21,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:39:21,970 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:39:22,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:22,055 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:22,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:23,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:24,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:25,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:26,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:27,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:27,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:28,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:29,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:30,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:31,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:32,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:32,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:33,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:34,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:35,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:36,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:39:36,970 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:39:37,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:37,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:38,085 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:38,520 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/config.yaml |
| 2025-01-06 17:39:38,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:39,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:40,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:41,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:42,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:43,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:43,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:44,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:45,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:46,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:47,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:48,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:49,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:49,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:50,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:51,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:39:51,970 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:39:52,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:52,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:53,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:54,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:55,055 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:39:55,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:56,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:57,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:58,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:39:59,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:00,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:00,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:01,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:02,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:03,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:04,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:05,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:05,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:06,611 DEBUG SystemMonitor:2093159 [system_monitor.py:_start():172] Starting system metrics aggregation loop |
| 2025-01-06 17:40:06,611 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:40:06,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:40:06,971 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:40:07,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:07,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:08,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:09,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:10,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:11,092 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:11,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:12,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:13,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:14,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:15,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:16,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:16,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:17,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:18,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:19,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:20,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:21,970 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:40:21,971 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:40:22,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:22,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:22,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:23,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:24,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:25,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:26,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:27,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:27,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:28,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:29,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:30,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:31,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:32,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:32,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:33,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:34,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:35,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:36,612 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:40:36,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:40:36,971 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:40:37,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:37,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:38,049 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:38,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:39,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:40,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:41,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:42,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:43,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:43,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:44,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:45,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:46,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:47,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:48,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:48,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:49,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:50,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:51,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:40:51,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:40:52,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:52,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:53,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:54,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:54,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:55,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:56,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:57,985 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:58,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:40:59,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:40:59,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:00,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:01,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:02,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:03,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:04,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:04,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:05,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:06,612 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:41:06,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:41:06,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:41:07,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:07,986 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:08,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:09,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:10,094 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:10,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:11,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:12,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:13,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:14,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:15,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:15,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:16,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:17,988 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:18,988 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:19,988 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:20,988 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:21,971 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:41:21,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:41:22,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:22,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:22,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:23,989 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:24,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:25,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:26,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:27,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:27,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:28,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:29,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:30,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:31,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:32,990 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:33,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:33,991 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:34,991 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:35,991 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:36,614 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:41:36,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:41:36,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:41:37,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:37,991 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:38,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:39,110 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:39,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:40,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:41,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:42,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:43,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:44,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:44,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:45,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:46,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:47,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:48,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:49,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:50,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:50,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:51,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:41:51,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:41:52,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:52,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:53,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:54,993 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:55,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:56,051 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:41:56,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:57,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:58,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:41:59,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:00,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:01,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:01,994 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:02,995 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:03,995 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:04,995 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:05,995 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:06,614 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:42:06,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:42:06,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:42:07,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:07,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:07,995 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:08,995 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:09,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:10,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:11,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:12,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:12,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:13,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:14,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:15,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:16,996 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:17,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:17,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:18,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:19,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:20,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:21,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:42:21,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:42:22,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:22,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:23,043 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:23,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:24,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:25,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:26,997 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:27,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:28,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:28,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:29,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:30,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:31,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:32,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:33,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:33,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:34,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:35,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:36,615 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:42:36,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:42:36,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:42:37,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:37,998 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:38,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:39,043 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:39,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:40,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:41,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:42,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:43,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:44,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:44,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:45,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:47,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:48,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:49,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:49,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:50,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:51,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:51,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:42:51,972 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:42:52,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:53,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:54,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:55,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:55,097 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:42:56,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:57,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:58,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:42:59,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:00,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:00,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:01,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:02,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:03,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:04,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:05,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:05,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:06,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:06,616 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:43:06,972 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:43:06,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:43:07,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:08,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:09,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:10,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:11,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:11,031 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:12,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:13,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:14,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:15,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:16,003 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:16,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:17,003 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:18,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:19,003 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:20,003 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:21,003 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:21,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:43:21,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:43:22,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:22,107 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:23,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:24,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:25,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:26,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:27,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:27,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:28,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:29,004 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:30,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:31,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:32,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:32,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:33,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:34,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:35,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:36,005 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:36,617 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:43:36,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:43:36,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:43:37,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:38,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:38,085 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:39,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:40,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:41,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:42,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:43,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:43,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:44,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:45,006 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:46,008 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:47,008 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:48,008 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:49,008 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:49,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:50,008 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:51,008 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:51,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:43:51,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:43:52,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:53,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:54,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:55,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:55,092 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:43:56,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:57,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:58,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:43:59,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:00,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:00,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:01,009 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:02,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:03,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:04,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:05,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:06,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:06,617 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:44:06,618 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:06,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:44:06,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:44:07,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:08,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:09,010 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:10,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:11,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:11,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:12,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:13,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:14,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:15,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:16,011 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:16,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:17,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:18,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:19,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:20,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:21,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:21,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:44:21,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:44:22,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:22,036 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:23,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:24,012 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:25,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:25,605 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:44:26,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:27,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:27,606 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:44:27,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:28,013 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:29,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:29,607 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:44:30,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:31,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:32,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:32,979 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:33,015 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:34,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:35,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:36,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:36,618 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:44:36,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:44:36,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:44:37,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:38,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:38,577 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:39,014 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:39,609 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:44:40,015 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:41,015 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:42,015 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:43,015 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:43,611 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:44:43,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:44,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:45,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:46,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:47,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:48,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:49,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:49,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:50,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:51,016 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:51,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:44:51,973 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:44:52,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:53,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:54,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:55,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:55,144 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:44:56,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:57,017 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:58,018 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:44:59,018 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:00,018 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:00,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:01,019 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:02,019 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:03,019 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:04,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:04,564 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:45:04,564 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:45:04,564 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/videos/eval/0_eval_0_1f9e236adbaeed571af3.mp4 with policy now |
| 2025-01-06 17:45:04,565 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:45:04,565 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:45:04,566 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:45:04,616 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:45:04,616 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/0_eval_0_1f9e236adbaeed571af3.mp4 |
| 2025-01-06 17:45:04,616 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos |
| 2025-01-06 17:45:04,617 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media |
| 2025-01-06 17:45:04,617 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval |
| 2025-01-06 17:45:04,865 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/sqafg0jm-media/videos/eval/0_eval_0_1f9e236adbaeed571af3.mp4 |
| 2025-01-06 17:45:05,019 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:05,151 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/l2dx4xh3-media/videos/eval/0_eval_0_1f9e236adbaeed571af3.mp4 |
| 2025-01-06 17:45:05,617 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:06,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:06,579 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:06,618 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:45:06,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:45:06,974 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:45:07,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:08,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:09,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:09,618 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:10,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:11,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:11,619 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:11,937 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:12,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:13,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:13,619 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:14,020 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:15,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:16,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:16,980 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:17,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:18,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:19,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:20,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:21,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:21,973 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:45:21,974 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:45:22,021 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:22,090 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:23,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:24,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:25,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:26,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:26,624 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:45:26,870 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/tp0klld9-events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:45:27,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:27,417 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:27,624 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:28,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:29,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:29,625 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:30,022 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:31,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:31,625 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:32,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:32,981 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:33,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:34,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:35,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:36,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:36,619 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:45:36,974 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:45:36,974 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:45:37,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:37,627 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:38,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:38,054 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:39,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:40,023 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:41,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:41,629 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:45:42,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:43,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:43,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:44,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:45,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:46,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:47,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:48,024 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:49,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:49,982 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:50,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:51,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:51,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:45:51,975 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:45:52,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:53,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:54,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:55,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:55,103 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:45:56,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:57,025 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:58,026 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:45:59,026 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:00,027 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:00,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:01,026 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:02,026 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:03,026 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:04,027 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:05,027 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:06,027 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:06,167 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:06,218 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:46:06,218 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/videos/eval/1_eval_1_04037a9f5a7dfcbf716d.mp4 with policy now |
| 2025-01-06 17:46:06,220 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:46:06,221 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:46:06,221 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:46:06,240 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:46:06,485 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/6h1a6y54-media/videos/eval/1_eval_1_04037a9f5a7dfcbf716d.mp4 |
| 2025-01-06 17:46:06,619 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:46:06,637 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:46:06,637 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/1_eval_1_04037a9f5a7dfcbf716d.mp4 |
| 2025-01-06 17:46:06,637 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval |
| 2025-01-06 17:46:06,920 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/j3pwg2pg-media/videos/eval/1_eval_1_04037a9f5a7dfcbf716d.mp4 |
| 2025-01-06 17:46:06,977 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:46:06,977 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:46:07,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:07,637 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:08,036 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:09,027 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:10,036 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:11,044 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:11,638 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:11,889 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:12,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:13,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:13,639 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:14,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:15,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:16,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:16,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:17,029 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:18,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:19,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:20,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:21,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:21,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:46:21,975 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:46:22,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:22,124 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:23,031 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:24,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:25,030 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:26,031 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:27,031 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:27,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:28,033 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:29,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:30,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:31,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:32,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:33,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:33,649 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:46:33,651 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:33,790 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:46:33,790 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:46:33,791 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:33,791 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:46:33,924 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/5027jja0-events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:46:34,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:34,649 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:46:35,033 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:35,650 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:36,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:36,619 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:46:36,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:46:36,975 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:46:37,032 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:37,650 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:38,033 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:39,033 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:39,088 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:40,034 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:41,034 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:42,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:43,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:44,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:44,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:45,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:46,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:47,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:48,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:49,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:50,035 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:50,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:51,036 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:51,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:46:51,976 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:46:52,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:53,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:54,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:55,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:55,656 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:46:56,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:56,854 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:46:57,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:58,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:59,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:46:59,658 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:00,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:01,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:01,987 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:02,037 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:03,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:04,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:05,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:06,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:06,620 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:47:06,975 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:47:06,976 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:47:07,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:07,098 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:08,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:09,038 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:10,039 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:11,039 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:12,042 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:12,179 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:12,515 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:47:12,515 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/videos/eval/2_eval_2_c482b456ff56ddc8a1e1.mp4 with policy now |
| 2025-01-06 17:47:12,515 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:47:12,516 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:47:12,516 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:47:12,525 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:47:12,662 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:47:12,662 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/2_eval_2_c482b456ff56ddc8a1e1.mp4 |
| 2025-01-06 17:47:12,662 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval |
| 2025-01-06 17:47:12,791 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/st0fd33d-media/videos/eval/2_eval_2_c482b456ff56ddc8a1e1.mp4 |
| 2025-01-06 17:47:13,042 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:13,059 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/y6u3pzub-media/videos/eval/2_eval_2_c482b456ff56ddc8a1e1.mp4 |
| 2025-01-06 17:47:13,662 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:14,043 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:15,045 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:16,047 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:17,047 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:17,664 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:17,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:18,046 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:19,043 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:19,664 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:20,044 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:21,044 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:21,988 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:47:21,988 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:47:22,045 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:23,046 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:23,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:23,615 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:47:23,615 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:47:23,615 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:47:23,666 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:47:23,666 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:47:23,666 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:24,047 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:25,048 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:25,666 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:26,051 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:27,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:27,667 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:28,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:28,117 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:29,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:29,668 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:30,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:31,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:32,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:33,057 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:33,991 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:34,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:35,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:36,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:36,621 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:47:36,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:47:36,976 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:47:37,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:38,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:39,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:39,138 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:40,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:41,058 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:41,672 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:42,059 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:43,059 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:44,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:44,185 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:45,059 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:45,674 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:46,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:47,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:47,675 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:47:48,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:49,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:49,991 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:50,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:51,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:51,976 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:47:51,977 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:47:52,060 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:53,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:54,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:55,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:55,117 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:47:56,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:57,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:58,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:47:59,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:00,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:00,992 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:01,061 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:02,070 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:02,815 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:48:02,815 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/videos/eval/3_eval_3_77698069b89e74b81013.mp4 with policy now |
| 2025-01-06 17:48:02,819 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:48:02,844 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:48:02,844 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:48:02,869 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:48:03,064 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:03,076 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/y7qlbzk4-media/videos/eval/3_eval_3_77698069b89e74b81013.mp4 |
| 2025-01-06 17:48:03,680 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:48:03,680 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:48:03,680 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/3_eval_3_77698069b89e74b81013.mp4 |
| 2025-01-06 17:48:03,680 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval |
| 2025-01-06 17:48:03,976 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/5jr7o643-media/videos/eval/3_eval_3_77698069b89e74b81013.mp4 |
| 2025-01-06 17:48:04,076 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:05,077 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:06,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:06,471 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:06,622 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:48:06,983 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:48:06,983 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:48:07,079 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:07,681 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:48:08,081 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:09,080 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:10,078 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:11,079 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:11,683 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:48:12,001 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:12,083 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:13,080 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:14,085 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:15,081 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:16,088 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:17,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:17,082 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:18,083 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:19,086 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:20,086 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:21,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:21,686 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:48:21,692 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:48:21,692 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:48:21,693 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:48:21,965 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/vxbri0xf-events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:48:21,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:48:21,985 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:48:22,090 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:22,140 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:22,686 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:48:23,096 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:23,686 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:48:24,090 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:25,088 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:25,687 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:48:26,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:27,088 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:27,688 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:48:28,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:28,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:29,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:30,090 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:31,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:32,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:33,089 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:33,999 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:34,090 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:35,090 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:36,092 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:36,622 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:48:36,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:48:36,978 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:48:37,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:38,092 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:39,077 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:39,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:40,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:41,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:42,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:43,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:44,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:45,000 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:45,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:46,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:47,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:48,093 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:49,094 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:50,002 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:50,094 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:51,094 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:51,978 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:48:51,979 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:48:52,095 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:53,095 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:54,096 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:55,048 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:48:55,097 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:56,097 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:57,097 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:58,097 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:59,097 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:48:59,699 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:00,102 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:00,567 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:49:01,120 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:01,177 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:49:01,177 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/videos/eval/4_eval_4_254f22dd2caf14ce117c.mp4 with policy now |
| 2025-01-06 17:49:01,191 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:49:01,191 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:49:01,191 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:01,212 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:01,441 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/uhz293fi-media/videos/eval/4_eval_4_254f22dd2caf14ce117c.mp4 |
| 2025-01-06 17:49:01,700 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:49:01,700 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:01,700 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/4_eval_4_254f22dd2caf14ce117c.mp4 |
| 2025-01-06 17:49:01,701 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval |
| 2025-01-06 17:49:01,992 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/nkocmcjf-media/videos/eval/4_eval_4_254f22dd2caf14ce117c.mp4 |
| 2025-01-06 17:49:02,109 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:03,115 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:03,701 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:04,116 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:05,103 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:05,701 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:05,897 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:49:06,115 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:06,623 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:49:06,984 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: stop_status |
| 2025-01-06 17:49:06,984 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: stop_status |
| 2025-01-06 17:49:07,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:08,111 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:09,111 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:09,578 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: log_artifact |
| 2025-01-06 17:49:09,578 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: log_artifact |
| 2025-01-06 17:49:09,703 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:49:09,703 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:09,703 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table/eval/success_rate_ci_5_1b420c32327aa0ba7f75.table.json |
| 2025-01-06 17:49:09,703 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media |
| 2025-01-06 17:49:09,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table |
| 2025-01-06 17:49:09,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table/eval |
| 2025-01-06 17:49:09,787 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:49:09,890 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/mc9dnymz-media/table/eval/success_rate_ci_5_1b420c32327aa0ba7f75.table.json |
| 2025-01-06 17:49:10,042 INFO wandb-upload_2:2093159 [upload_job.py:push():88] Uploaded file /home/leonmkim/.local/share/wandb/artifacts/staging/tmptzqc3sdw |
| 2025-01-06 17:49:10,104 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:10,137 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: log_artifact |
| 2025-01-06 17:49:10,177 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: partial_history |
| 2025-01-06 17:49:10,480 INFO SenderThread:2093159 [sender.py:send_request_log_artifact():1525] logged artifact run-1jous172-evalsuccess_rate_ci - {'id': 'QXJ0aWZhY3Q6MTQyMTk1OTE2Mw==', 'state': 'PENDING', 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjUzMDUxNDc1Nw==', 'latestArtifact': None}} |
| 2025-01-06 17:49:10,481 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:49:10,481 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/table/eval/success_rate_ci_5_1b420c32327aa0ba7f75.table.json with policy now |
| 2025-01-06 17:49:10,481 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:49:10,482 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,482 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,482 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:49:10,483 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,486 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,487 DEBUG SenderThread:2093159 [sender.py:send():379] send: telemetry |
| 2025-01-06 17:49:10,487 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,487 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,487 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,487 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,488 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,488 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,488 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,488 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,489 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,489 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,489 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,489 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,489 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,490 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,490 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,490 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,490 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,491 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,491 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,491 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,491 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:10,492 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:10,492 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: log_artifact |
| 2025-01-06 17:49:10,703 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:49:10,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:49:10,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_created():271] file/dir created: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table/eval/total_success_rate_ci_6_e32f206babc2825ce217.table.json |
| 2025-01-06 17:49:10,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table/eval |
| 2025-01-06 17:49:10,728 INFO wandb-upload_1:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/4kkpa2oc-media/table/eval/success_rate_ci_5_1b420c32327aa0ba7f75.table.json |
| 2025-01-06 17:49:10,854 INFO wandb-upload_2:2093159 [upload_job.py:push():88] Uploaded file /home/leonmkim/.local/share/wandb/artifacts/staging/tmpmrxdjnzp |
| 2025-01-06 17:49:10,930 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/pnjp4gzq-media/table/eval/total_success_rate_ci_6_e32f206babc2825ce217.table.json |
| 2025-01-06 17:49:11,104 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:11,247 INFO SenderThread:2093159 [sender.py:send_request_log_artifact():1525] logged artifact run-1jous172-evaltotal_success_rate_ci - {'id': 'QXJ0aWZhY3Q6MTQyMTk1OTE5Ng==', 'state': 'PENDING', 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjUzMDUxNDc2MQ==', 'latestArtifact': None}} |
| 2025-01-06 17:49:11,247 DEBUG SenderThread:2093159 [sender.py:send():379] send: files |
| 2025-01-06 17:49:11,247 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file media/table/eval/total_success_rate_ci_6_e32f206babc2825ce217.table.json with policy now |
| 2025-01-06 17:49:11,248 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:49:11,248 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:11,248 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:49:11,248 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:11,497 INFO wandb-upload_1:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/wyeihir3-media/table/eval/total_success_rate_ci_6_e32f206babc2825ce217.table.json |
| 2025-01-06 17:49:11,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:49:11,704 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:12,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:13,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:14,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:15,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:16,105 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:16,578 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:49:16,621 DEBUG SenderThread:2093159 [sender.py:send():379] send: exit |
| 2025-01-06 17:49:16,622 INFO SenderThread:2093159 [sender.py:send_exit():586] handling exit code: 0 |
| 2025-01-06 17:49:16,622 INFO SenderThread:2093159 [sender.py:send_exit():588] handling runtime: 610 |
| 2025-01-06 17:49:16,622 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:16,622 INFO SenderThread:2093159 [sender.py:send_exit():594] send defer |
| 2025-01-06 17:49:16,622 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:16,622 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 0 |
| 2025-01-06 17:49:16,622 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:16,623 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 0 |
| 2025-01-06 17:49:16,623 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 1 |
| 2025-01-06 17:49:16,623 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:16,623 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 1 |
| 2025-01-06 17:49:16,623 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:16,623 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 1 |
| 2025-01-06 17:49:16,623 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 2 |
| 2025-01-06 17:49:16,623 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:16,623 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 2 |
| 2025-01-06 17:49:16,623 INFO HandlerThread:2093159 [system_monitor.py:finish():203] Stopping system monitor |
| 2025-01-06 17:49:16,623 DEBUG SystemMonitor:2093159 [system_monitor.py:_start():179] Finished system metrics aggregation loop |
| 2025-01-06 17:49:16,624 DEBUG SystemMonitor:2093159 [system_monitor.py:_start():183] Publishing last batch of metrics |
| 2025-01-06 17:49:16,624 INFO HandlerThread:2093159 [interfaces.py:finish():200] Joined cpu monitor |
| 2025-01-06 17:49:16,624 INFO HandlerThread:2093159 [interfaces.py:finish():200] Joined disk monitor |
| 2025-01-06 17:49:16,694 INFO HandlerThread:2093159 [interfaces.py:finish():200] Joined gpu monitor |
| 2025-01-06 17:49:16,694 INFO HandlerThread:2093159 [interfaces.py:finish():200] Joined memory monitor |
| 2025-01-06 17:49:16,695 INFO HandlerThread:2093159 [interfaces.py:finish():200] Joined network monitor |
| 2025-01-06 17:49:16,695 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:16,695 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 2 |
| 2025-01-06 17:49:16,695 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 3 |
| 2025-01-06 17:49:16,695 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:16,695 DEBUG SenderThread:2093159 [sender.py:send():379] send: stats |
| 2025-01-06 17:49:16,695 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 3 |
| 2025-01-06 17:49:16,696 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:16,696 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 3 |
| 2025-01-06 17:49:16,696 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 4 |
| 2025-01-06 17:49:16,696 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:16,696 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 4 |
| 2025-01-06 17:49:16,706 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:49:17,706 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:22,758 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: poll_exit |
| 2025-01-06 17:49:22,758 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:49:22,758 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: keepalive |
| 2025-01-06 17:49:22,759 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:22,759 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 4 |
| 2025-01-06 17:49:22,759 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 5 |
| 2025-01-06 17:49:22,760 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: poll_exit |
| 2025-01-06 17:49:22,760 DEBUG SenderThread:2093159 [sender.py:send():379] send: history |
| 2025-01-06 17:49:22,760 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: summary_record |
| 2025-01-06 17:49:22,760 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:22,760 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:22,760 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 5 |
| 2025-01-06 17:49:22,761 DEBUG SenderThread:2093159 [sender.py:send():379] send: summary |
| 2025-01-06 17:49:22,764 INFO SenderThread:2093159 [sender.py:_save_file():1454] saving file wandb-summary.json with policy end |
| 2025-01-06 17:49:22,764 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:22,765 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 5 |
| 2025-01-06 17:49:22,765 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 6 |
| 2025-01-06 17:49:22,765 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:22,765 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 6 |
| 2025-01-06 17:49:22,765 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:22,765 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 6 |
| 2025-01-06 17:49:22,768 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: status_report |
| 2025-01-06 17:49:22,906 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 7 |
| 2025-01-06 17:49:22,906 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:22,906 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 7 |
| 2025-01-06 17:49:22,906 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:22,906 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 7 |
| 2025-01-06 17:49:23,623 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: poll_exit |
| 2025-01-06 17:49:23,709 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:49:23,709 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/config.yaml |
| 2025-01-06 17:49:25,145 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 8 |
| 2025-01-06 17:49:25,145 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: poll_exit |
| 2025-01-06 17:49:25,145 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:25,145 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 8 |
| 2025-01-06 17:49:25,146 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:25,146 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 8 |
| 2025-01-06 17:49:25,146 INFO SenderThread:2093159 [job_builder.py:build():440] Attempting to build job artifact |
| 2025-01-06 17:49:25,148 INFO SenderThread:2093159 [job_builder.py:_get_source_type():569] is repo sourced job |
| 2025-01-06 17:49:25,150 INFO SenderThread:2093159 [job_builder.py:build():545] adding wandb-job metadata file |
| 2025-01-06 17:49:25,160 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 9 |
| 2025-01-06 17:49:25,161 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:25,161 DEBUG SenderThread:2093159 [sender.py:send():379] send: artifact |
| 2025-01-06 17:49:25,161 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 9 |
| 2025-01-06 17:49:25,620 INFO wandb-upload_2:2093159 [upload_job.py:push():85] Skipped uploading /home/leonmkim/.local/share/wandb/artifacts/staging/tmpwscw5le0 |
| 2025-01-06 17:49:25,620 INFO wandb-upload_1:2093159 [upload_job.py:push():85] Skipped uploading /tmp/tmp3ojk991x/wandb-job.json |
| 2025-01-06 17:49:25,624 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: poll_exit |
| 2025-01-06 17:49:25,710 INFO Thread-12 :2093159 [dir_watcher.py:_on_file_modified():288] file/dir modified: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:25,726 INFO wandb-upload_0:2093159 [upload_job.py:push():88] Uploaded file /home/leonmkim/.local/share/wandb/artifacts/staging/tmpnh0s_z63 |
| 2025-01-06 17:49:26,354 INFO SenderThread:2093159 [sender.py:send_artifact():1537] sent artifact job-https___github.com_leonmkim_fish_leon.git_FISH_eval_robot.py - {'id': 'QXJ0aWZhY3Q6MTQyMTk1OTYxMQ==', 'state': 'PENDING', 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjMxNzk5NjkyNQ==', 'latestArtifact': {'id': 'QXJ0aWZhY3Q6MTQyMTkzNjUwMA==', 'versionIndex': 424}}} |
| 2025-01-06 17:49:26,354 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:26,354 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 9 |
| 2025-01-06 17:49:26,354 INFO SenderThread:2093159 [dir_watcher.py:finish():358] shutting down directory watcher |
| 2025-01-06 17:49:26,711 INFO SenderThread:2093159 [dir_watcher.py:finish():388] scan: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files |
| 2025-01-06 17:49:26,711 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/config.yaml config.yaml |
| 2025-01-06 17:49:26,711 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json wandb-summary.json |
| 2025-01-06 17:49:26,712 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/requirements.txt requirements.txt |
| 2025-01-06 17:49:26,712 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-metadata.json wandb-metadata.json |
| 2025-01-06 17:49:26,712 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:49:26,712 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log output.log |
| 2025-01-06 17:49:26,715 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/diff.patch diff.patch |
| 2025-01-06 17:49:26,716 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/code/FISH/eval_robot.py code/FISH/eval_robot.py |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/3_eval_3_77698069b89e74b81013.mp4 media/videos/eval/3_eval_3_77698069b89e74b81013.mp4 |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/2_eval_2_c482b456ff56ddc8a1e1.mp4 media/videos/eval/2_eval_2_c482b456ff56ddc8a1e1.mp4 |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/0_eval_0_1f9e236adbaeed571af3.mp4 media/videos/eval/0_eval_0_1f9e236adbaeed571af3.mp4 |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/4_eval_4_254f22dd2caf14ce117c.mp4 media/videos/eval/4_eval_4_254f22dd2caf14ce117c.mp4 |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/videos/eval/1_eval_1_04037a9f5a7dfcbf716d.mp4 media/videos/eval/1_eval_1_04037a9f5a7dfcbf716d.mp4 |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table/eval/total_success_rate_ci_6_e32f206babc2825ce217.table.json media/table/eval/total_success_rate_ci_6_e32f206babc2825ce217.table.json |
| 2025-01-06 17:49:26,719 INFO SenderThread:2093159 [dir_watcher.py:finish():402] scan save: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/media/table/eval/success_rate_ci_5_1b420c32327aa0ba7f75.table.json media/table/eval/success_rate_ci_5_1b420c32327aa0ba7f75.table.json |
| 2025-01-06 17:49:26,720 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 10 |
| 2025-01-06 17:49:26,720 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: poll_exit |
| 2025-01-06 17:49:26,722 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:26,722 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 10 |
| 2025-01-06 17:49:26,722 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:26,722 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 10 |
| 2025-01-06 17:49:26,722 INFO SenderThread:2093159 [file_pusher.py:finish():169] shutting down file pusher |
| 2025-01-06 17:49:26,897 INFO wandb-upload_1:2093159 [upload_job.py:push():130] Uploaded file /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/wandb-summary.json |
| 2025-01-06 17:49:26,912 INFO wandb-upload_2:2093159 [upload_job.py:push():130] Uploaded file /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/config.yaml |
| 2025-01-06 17:49:26,946 INFO wandb-upload_0:2093159 [upload_job.py:push():130] Uploaded file /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/requirements.txt |
| 2025-01-06 17:49:26,974 INFO wandb-upload_4:2093159 [upload_job.py:push():130] Uploaded file /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/files/output.log |
| 2025-01-06 17:49:26,995 INFO wandb-upload_3:2093159 [upload_job.py:push():130] Uploaded file /tmp/tmpdbd0smeswandb/ie0stzsg-events.out.tfevents.1736203146.leonmkim-ROG-Strix-G15CS-G15CS.2092961.0 |
| 2025-01-06 17:49:27,195 INFO Thread-11 (_thread_body):2093159 [sender.py:transition_state():614] send defer: 11 |
| 2025-01-06 17:49:27,195 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:27,195 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 11 |
| 2025-01-06 17:49:27,196 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:27,196 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 11 |
| 2025-01-06 17:49:27,196 INFO SenderThread:2093159 [file_pusher.py:join():175] waiting for file pusher |
| 2025-01-06 17:49:27,197 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 12 |
| 2025-01-06 17:49:27,197 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:27,197 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 12 |
| 2025-01-06 17:49:27,197 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:27,197 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 12 |
| 2025-01-06 17:49:27,197 INFO SenderThread:2093159 [file_stream.py:finish():601] file stream finish called |
| 2025-01-06 17:49:27,407 INFO SenderThread:2093159 [file_stream.py:finish():605] file stream finish is done |
| 2025-01-06 17:49:27,407 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 13 |
| 2025-01-06 17:49:27,407 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:27,407 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 13 |
| 2025-01-06 17:49:27,407 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:27,407 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 13 |
| 2025-01-06 17:49:27,407 INFO SenderThread:2093159 [sender.py:transition_state():614] send defer: 14 |
| 2025-01-06 17:49:27,407 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: defer |
| 2025-01-06 17:49:27,407 INFO HandlerThread:2093159 [handler.py:handle_request_defer():184] handle defer: 14 |
| 2025-01-06 17:49:27,408 DEBUG SenderThread:2093159 [sender.py:send():379] send: final |
| 2025-01-06 17:49:27,408 DEBUG SenderThread:2093159 [sender.py:send():379] send: footer |
| 2025-01-06 17:49:27,408 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: defer |
| 2025-01-06 17:49:27,408 INFO SenderThread:2093159 [sender.py:send_request_defer():610] handle sender defer: 14 |
| 2025-01-06 17:49:27,408 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: poll_exit |
| 2025-01-06 17:49:27,409 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: poll_exit |
| 2025-01-06 17:49:27,409 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: server_info |
| 2025-01-06 17:49:27,409 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: poll_exit |
| 2025-01-06 17:49:27,409 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: get_summary |
| 2025-01-06 17:49:27,410 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: server_info |
| 2025-01-06 17:49:27,411 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: sampled_history |
| 2025-01-06 17:49:27,413 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: internal_messages |
| 2025-01-06 17:49:27,469 DEBUG SenderThread:2093159 [sender.py:send_request():406] send_request: poll_exit |
| 2025-01-06 17:49:27,470 INFO MainThread:2093159 [wandb_run.py:_footer_history_summary_info():4016] rendering history |
| 2025-01-06 17:49:27,470 INFO MainThread:2093159 [wandb_run.py:_footer_history_summary_info():4048] rendering summary |
| 2025-01-06 17:49:27,470 INFO MainThread:2093159 [wandb_run.py:_footer_sync_info():3975] logging synced files |
| 2025-01-06 17:49:27,471 DEBUG HandlerThread:2093159 [handler.py:handle_request():158] handle_request: shutdown |
| 2025-01-06 17:49:27,471 INFO HandlerThread:2093159 [handler.py:finish():882] shutting down handler |
| 2025-01-06 17:49:28,409 INFO WriterThread:2093159 [datastore.py:close():296] close: /home/leonmkim/fish_leon/FISH/exp_local/frankagym_pixels/FrankaInsertion-v1/1003_0/1jous172/wandb/run-20250106_173906-1jous172/run-1jous172.wandb |
| 2025-01-06 17:49:28,470 INFO SenderThread:2093159 [sender.py:finish():1615] shutting down sender |
| 2025-01-06 17:49:28,470 INFO SenderThread:2093159 [file_pusher.py:finish():169] shutting down file pusher |
| 2025-01-06 17:49:28,470 INFO SenderThread:2093159 [file_pusher.py:join():175] waiting for file pusher |
|
|
