2024-06-25 17:08:22,683 INFO    StreamThr :149 [internal.py:wandb_internal():86] W&B internal server running at pid: 149, started at: 2024-06-25 17:08:22.683035
2024-06-25 17:08:22,685 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status
2024-06-25 17:08:23,448 INFO    WriterThread:149 [datastore.py:open_for_write():87] open: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/run-n5fw7h0h.wandb
2024-06-25 17:08:23,449 DEBUG   SenderThread:149 [sender.py:send():382] send: header
2024-06-25 17:08:23,452 DEBUG   SenderThread:149 [sender.py:send():382] send: run
2024-06-25 17:08:23,921 INFO    SenderThread:149 [dir_watcher.py:__init__():211] watching files in: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files
2024-06-25 17:08:23,921 INFO    SenderThread:149 [sender.py:_start_run_threads():1136] run started: n5fw7h0h with start time 1719335302.684404
2024-06-25 17:08:23,933 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: check_version
2024-06-25 17:08:23,933 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: check_version
2024-06-25 17:08:23,989 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: run_start
2024-06-25 17:08:24,003 DEBUG   HandlerThread:149 [system_info.py:__init__():27] System info init
2024-06-25 17:08:24,003 DEBUG   HandlerThread:149 [system_info.py:__init__():42] System info init done
2024-06-25 17:08:24,003 INFO    HandlerThread:149 [system_monitor.py:start():194] Starting system monitor
2024-06-25 17:08:24,004 INFO    SystemMonitor:149 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-06-25 17:08:24,004 INFO    HandlerThread:149 [system_monitor.py:probe():214] Collecting system info
2024-06-25 17:08:24,005 INFO    SystemMonitor:149 [interfaces.py:start():190] Started cpu monitoring
2024-06-25 17:08:24,005 INFO    SystemMonitor:149 [interfaces.py:start():190] Started disk monitoring
2024-06-25 17:08:24,006 INFO    SystemMonitor:149 [interfaces.py:start():190] Started gpu monitoring
2024-06-25 17:08:24,008 INFO    SystemMonitor:149 [interfaces.py:start():190] Started memory monitoring
2024-06-25 17:08:24,008 INFO    SystemMonitor:149 [interfaces.py:start():190] Started network monitoring
2024-06-25 17:08:24,016 DEBUG   HandlerThread:149 [system_info.py:probe():151] Probing system
2024-06-25 17:08:24,018 DEBUG   HandlerThread:149 [gitlib.py:_init_repo():56] git repository is invalid
2024-06-25 17:08:24,018 DEBUG   HandlerThread:149 [system_info.py:probe():199] Probing system done
2024-06-25 17:08:24,018 DEBUG   HandlerThread:149 [system_monitor.py:probe():223] {'os': 'Linux-5.15.133+-x86_64-with-glibc2.35', 'python': '3.10.12', 'heartbeatAt': '2024-06-25T17:08:24.016433', 'startedAt': '2024-06-25T17:08:22.676465', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'kaggle.ipynb', 'codePathLocal': None, 'root': '/kaggle/working', 'host': 'b10c572b8059', 'username': 'root', 'executable': '/opt/conda/bin/python3.10', 'cpu_count': 2, 'cpu_count_logical': 4, 'cpu_freq': {'current': 2000.17, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.17, 'min': 0.0, 'max': 0.0}, {'current': 2000.17, 'min': 0.0, 'max': 0.0}, {'current': 2000.17, 'min': 0.0, 'max': 0.0}, {'current': 2000.17, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 8062.387607574463, 'used': 5689.849388122559}}, 'gpu': 'Tesla P100-PCIE-16GB', 'gpu_count': 1, 'gpu_devices': [{'name': 'Tesla P100-PCIE-16GB', 'memory_total': 17179869184}], 'memory': {'total': 31.357563018798828}}
2024-06-25 17:08:24,018 INFO    HandlerThread:149 [system_monitor.py:probe():224] Finished collecting system info
2024-06-25 17:08:24,018 INFO    HandlerThread:149 [system_monitor.py:probe():227] Publishing system info
2024-06-25 17:08:24,019 DEBUG   HandlerThread:149 [system_info.py:_save_conda():208] Saving list of conda packages installed into the current environment
2024-06-25 17:08:24,923 INFO    Thread-12 :149 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/conda-environment.yaml
2024-06-25 17:09:14,938 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/conda-environment.yaml
2024-06-25 17:09:15,254 DEBUG   HandlerThread:149 [system_info.py:_save_conda():220] Saving conda packages done
2024-06-25 17:09:15,255 INFO    HandlerThread:149 [system_monitor.py:probe():229] Finished publishing system info
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,264 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: python_packages
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: internal_messages
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: stop_status
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:15,265 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:15,267 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: python_packages
2024-06-25 17:09:15,269 DEBUG   SenderThread:149 [sender.py:send():382] send: telemetry
2024-06-25 17:09:15,269 DEBUG   SenderThread:149 [sender.py:send():382] send: config
2024-06-25 17:09:15,270 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:09:15,270 DEBUG   SenderThread:149 [sender.py:send():382] send: telemetry
2024-06-25 17:09:15,270 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:09:15,270 WARNING SenderThread:149 [sender.py:send_metric():1354] Seen metric with glob (shouldn't happen)
2024-06-25 17:09:15,270 DEBUG   SenderThread:149 [sender.py:send():382] send: telemetry
2024-06-25 17:09:15,271 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: stop_status
2024-06-25 17:09:15,523 DEBUG   SenderThread:149 [sender.py:send():382] send: files
2024-06-25 17:09:15,524 INFO    SenderThread:149 [sender.py:_save_file():1403] saving file wandb-metadata.json with policy now
2024-06-25 17:09:15,938 INFO    Thread-12 :149 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/output.log
2024-06-25 17:09:15,939 INFO    Thread-12 :149 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/wandb-metadata.json
2024-06-25 17:09:15,939 INFO    Thread-12 :149 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/requirements.txt
2024-06-25 17:09:16,295 INFO    wandb-upload_0:149 [upload_job.py:push():131] Uploaded file /tmp/tmp_1pzo1aiwandb/85y8vdqe-wandb-metadata.json
2024-06-25 17:09:17,939 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/output.log
2024-06-25 17:09:18,525 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:19,042 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:23,530 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:23,942 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/config.yaml
2024-06-25 17:09:24,009 DEBUG   SystemMonitor:149 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-06-25 17:09:24,010 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:09:24,060 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:29,011 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:29,061 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:34,012 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:34,062 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:39,013 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:39,064 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:44,014 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:44,065 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:49,015 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:49,066 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:54,010 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:09:54,067 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:09:55,011 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:09:59,068 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:00,012 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:04,070 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:05,013 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:09,072 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:10,014 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:14,090 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:15,015 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:19,092 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:20,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:24,011 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:10:24,093 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:26,012 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:29,094 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:31,013 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:34,095 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:36,014 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:39,096 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:41,015 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:44,097 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:46,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:49,107 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:51,017 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:54,012 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:10:54,109 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:10:57,013 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:10:59,109 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:02,014 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:04,111 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:07,015 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:09,112 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:12,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:14,113 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:17,017 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:19,114 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:22,018 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:24,013 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:11:24,115 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:28,014 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:29,116 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:33,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:34,117 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:38,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:39,118 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:43,017 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:44,119 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:48,018 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:49,120 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:53,019 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:54,014 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:11:54,122 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:11:59,015 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:11:59,123 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:04,019 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:04,124 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:09,019 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:09,125 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:14,020 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:14,126 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:19,021 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:19,127 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:24,014 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:12:24,129 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:25,015 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:29,130 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:30,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:34,141 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:35,017 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:39,142 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:40,018 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:44,143 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:45,019 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:49,161 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:50,020 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:54,015 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:12:54,162 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:12:56,016 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:12:59,164 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:01,017 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:04,165 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:06,018 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:09,075 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: partial_history
2024-06-25 17:13:09,078 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:13:09,078 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:13:09,078 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:13:09,078 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:13:09,078 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:13:09,078 DEBUG   SenderThread:149 [sender.py:send():382] send: history
2024-06-25 17:13:09,079 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: summary_record
2024-06-25 17:13:09,080 INFO    SenderThread:149 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
2024-06-25 17:13:09,175 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:10,034 INFO    Thread-12 :149 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/wandb-summary.json
2024-06-25 17:13:11,081 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:14,176 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:16,083 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:19,177 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:21,083 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:24,016 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:13:24,178 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:27,018 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:29,179 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:32,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:33,046 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/config.yaml
2024-06-25 17:13:34,192 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:37,285 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:39,193 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:42,286 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:44,194 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:47,287 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:49,196 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:52,288 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:54,018 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:13:54,197 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:13:58,020 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:13:59,198 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:03,021 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:04,199 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:08,022 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:09,200 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:13,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:14,201 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:18,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:19,203 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:23,025 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:24,018 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:14:24,204 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:29,019 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:29,205 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:34,020 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:34,206 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:39,021 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:39,217 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:44,022 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:44,227 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:49,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:49,228 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:54,019 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:14:54,229 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:14:55,019 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:14:59,230 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:00,020 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:04,232 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:05,021 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:09,233 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:10,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:14,234 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:15,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:19,235 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:20,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:24,019 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:15:24,237 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:26,020 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:29,238 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:31,021 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:34,239 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:36,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:39,240 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:41,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:44,242 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:46,025 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:49,243 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:51,025 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:54,020 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:15:54,244 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:15:57,021 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:15:59,245 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:02,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:04,250 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:07,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:09,251 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:12,025 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:14,252 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:17,026 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:19,253 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:22,027 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:24,021 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:16:24,254 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:28,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:29,256 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:33,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:34,257 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:34,488 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: partial_history
2024-06-25 17:16:34,490 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:16:34,490 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:16:34,490 DEBUG   SenderThread:149 [sender.py:send():382] send: history
2024-06-25 17:16:34,491 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: summary_record
2024-06-25 17:16:34,491 INFO    SenderThread:149 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
2024-06-25 17:16:35,121 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/wandb-summary.json
2024-06-25 17:16:38,497 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:39,123 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/config.yaml
2024-06-25 17:16:39,578 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:43,767 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:44,579 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:48,772 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:49,594 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:53,773 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:54,022 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:16:54,595 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:16:59,023 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:16:59,606 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:04,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:04,607 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:09,025 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:09,619 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:14,026 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:14,632 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:19,027 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:19,633 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:24,023 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:17:24,635 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:25,024 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:27,056 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: partial_history
2024-06-25 17:17:27,057 DEBUG   SenderThread:149 [sender.py:send():382] send: history
2024-06-25 17:17:27,057 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: summary_record
2024-06-25 17:17:27,058 INFO    SenderThread:149 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
2024-06-25 17:17:27,141 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/wandb-summary.json
2024-06-25 17:17:29,649 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:30,059 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:34,650 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:35,060 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:39,652 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:40,061 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:44,653 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:45,062 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:49,654 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:50,063 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:54,024 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:17:54,655 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:17:56,025 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:17:59,657 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:01,026 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:04,658 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:06,027 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:09,659 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:11,028 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:14,660 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:16,029 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:19,661 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:21,030 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:24,025 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:18:24,670 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:27,026 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:29,671 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:32,027 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:34,672 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:37,028 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:39,673 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:42,030 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:44,674 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:47,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:49,675 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:52,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:54,026 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:18:54,676 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:18:58,027 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:18:59,678 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:03,028 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:04,679 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:08,029 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:09,680 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:13,030 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:14,699 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:18,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:19,700 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:23,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:24,027 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:19:24,701 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:29,028 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:29,702 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:34,029 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:34,703 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:39,030 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:39,704 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:44,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:44,705 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:49,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:49,707 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:54,028 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:19:54,720 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:19:55,028 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:19:59,721 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:00,029 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:04,722 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:05,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:09,723 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:10,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:14,724 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:15,033 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:19,725 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:20,034 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:24,028 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:20:24,726 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:26,029 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:29,729 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:31,030 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:34,730 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:36,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:39,731 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:41,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:44,732 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:46,033 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:49,733 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:51,034 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:54,029 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:20:54,735 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:20:57,030 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:20:59,736 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:02,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:04,736 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:07,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:09,738 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:12,033 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:14,739 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:17,035 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:19,740 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:22,037 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:24,030 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:21:24,741 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:28,031 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:29,742 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:33,032 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:34,743 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:38,033 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:39,744 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:42,144 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: partial_history
2024-06-25 17:21:42,146 DEBUG   SenderThread:149 [sender.py:send():382] send: history
2024-06-25 17:21:42,146 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: summary_record
2024-06-25 17:21:42,146 INFO    SenderThread:149 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
2024-06-25 17:21:42,148 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: partial_history
2024-06-25 17:21:42,150 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:21:42,150 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:21:42,151 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:21:42,151 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:21:42,151 DEBUG   SenderThread:149 [sender.py:send():382] send: metric
2024-06-25 17:21:42,151 DEBUG   SenderThread:149 [sender.py:send():382] send: history
2024-06-25 17:21:42,151 DEBUG   SenderThread:149 [sender.py:send_request():409] send_request: summary_record
2024-06-25 17:21:42,152 INFO    SenderThread:149 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
2024-06-25 17:21:42,154 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: pause
2024-06-25 17:21:42,155 INFO    HandlerThread:149 [handler.py:handle_request_pause():708] stopping system metrics thread
2024-06-25 17:21:42,155 INFO    HandlerThread:149 [system_monitor.py:finish():203] Stopping system monitor
2024-06-25 17:21:42,155 DEBUG   SystemMonitor:149 [system_monitor.py:_start():179] Finished system metrics aggregation loop
2024-06-25 17:21:42,155 DEBUG   SystemMonitor:149 [system_monitor.py:_start():183] Publishing last batch of metrics
2024-06-25 17:21:42,156 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined cpu monitor
2024-06-25 17:21:42,157 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined disk monitor
2024-06-25 17:21:42,162 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined gpu monitor
2024-06-25 17:21:42,162 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined memory monitor
2024-06-25 17:21:42,162 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined network monitor
2024-06-25 17:21:42,163 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: resume
2024-06-25 17:21:42,163 INFO    HandlerThread:149 [handler.py:handle_request_resume():699] starting system metrics thread
2024-06-25 17:21:42,163 INFO    HandlerThread:149 [system_monitor.py:start():194] Starting system monitor
2024-06-25 17:21:42,163 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:21:42,164 INFO    SystemMonitor:149 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-06-25 17:21:42,164 INFO    SystemMonitor:149 [interfaces.py:start():190] Started cpu monitoring
2024-06-25 17:21:42,165 INFO    SystemMonitor:149 [interfaces.py:start():190] Started disk monitoring
2024-06-25 17:21:42,166 INFO    SystemMonitor:149 [interfaces.py:start():190] Started gpu monitoring
2024-06-25 17:21:42,166 INFO    SystemMonitor:149 [interfaces.py:start():190] Started memory monitoring
2024-06-25 17:21:42,167 INFO    SystemMonitor:149 [interfaces.py:start():190] Started network monitoring
2024-06-25 17:21:42,248 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/wandb-summary.json
2024-06-25 17:21:43,164 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:43,345 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: pause
2024-06-25 17:21:43,345 INFO    HandlerThread:149 [handler.py:handle_request_pause():708] stopping system metrics thread
2024-06-25 17:21:43,345 INFO    HandlerThread:149 [system_monitor.py:finish():203] Stopping system monitor
2024-06-25 17:21:43,346 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined cpu monitor
2024-06-25 17:21:43,346 DEBUG   SystemMonitor:149 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-06-25 17:21:43,347 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined disk monitor
2024-06-25 17:21:43,347 DEBUG   SystemMonitor:149 [system_monitor.py:_start():179] Finished system metrics aggregation loop
2024-06-25 17:21:43,347 DEBUG   SystemMonitor:149 [system_monitor.py:_start():183] Publishing last batch of metrics
2024-06-25 17:21:43,353 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined gpu monitor
2024-06-25 17:21:43,353 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined memory monitor
2024-06-25 17:21:43,354 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined network monitor
2024-06-25 17:21:43,354 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:21:44,755 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:48,360 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:49,251 INFO    Thread-12 :149 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240625_170822-n5fw7h0h/files/config.yaml
2024-06-25 17:21:49,756 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:53,646 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:54,757 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:21:58,647 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:21:59,758 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:03,648 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:04,759 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:08,650 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:09,760 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:13,651 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:14,761 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:18,652 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:19,762 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:23,653 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:24,763 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:28,654 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:29,765 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:33,656 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:34,766 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:38,657 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:39,767 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:43,658 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:44,769 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:48,660 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:49,770 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:53,661 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:54,771 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:22:58,662 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:22:59,772 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:03,664 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:04,773 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:08,665 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:09,775 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:13,666 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:14,776 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:18,667 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:19,777 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:23,668 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:24,778 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:28,670 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:29,779 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:33,671 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:34,781 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:38,672 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:39,782 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:43,673 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:44,783 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:48,674 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:49,784 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:53,675 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:54,785 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:23:58,676 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:23:59,786 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:03,677 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:04,787 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:08,678 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:09,789 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:13,679 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:14,790 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:18,680 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:19,791 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:23,681 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:24,792 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:28,683 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:29,793 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:33,684 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:34,794 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:38,685 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:39,795 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:43,686 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:44,796 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:48,687 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:49,797 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:53,688 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:54,798 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:24:58,689 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:24:59,799 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:03,690 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:04,800 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:08,691 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:09,802 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:13,693 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:14,803 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:18,694 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:19,804 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:23,696 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:24,805 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:28,697 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:29,816 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:32,340 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: resume
2024-06-25 17:25:32,340 INFO    HandlerThread:149 [handler.py:handle_request_resume():699] starting system metrics thread
2024-06-25 17:25:32,341 INFO    HandlerThread:149 [system_monitor.py:start():194] Starting system monitor
2024-06-25 17:25:32,341 INFO    SystemMonitor:149 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-06-25 17:25:32,341 INFO    SystemMonitor:149 [interfaces.py:start():190] Started cpu monitoring
2024-06-25 17:25:32,342 INFO    SystemMonitor:149 [interfaces.py:start():190] Started disk monitoring
2024-06-25 17:25:32,343 INFO    SystemMonitor:149 [interfaces.py:start():190] Started gpu monitoring
2024-06-25 17:25:32,344 INFO    SystemMonitor:149 [interfaces.py:start():190] Started memory monitoring
2024-06-25 17:25:32,345 INFO    SystemMonitor:149 [interfaces.py:start():190] Started network monitoring
2024-06-25 17:25:33,276 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: pause
2024-06-25 17:25:33,276 INFO    HandlerThread:149 [handler.py:handle_request_pause():708] stopping system metrics thread
2024-06-25 17:25:33,276 INFO    HandlerThread:149 [system_monitor.py:finish():203] Stopping system monitor
2024-06-25 17:25:33,277 DEBUG   SystemMonitor:149 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-06-25 17:25:33,277 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined cpu monitor
2024-06-25 17:25:33,277 DEBUG   SystemMonitor:149 [system_monitor.py:_start():179] Finished system metrics aggregation loop
2024-06-25 17:25:33,278 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined disk monitor
2024-06-25 17:25:33,278 DEBUG   SystemMonitor:149 [system_monitor.py:_start():183] Publishing last batch of metrics
2024-06-25 17:25:33,284 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined gpu monitor
2024-06-25 17:25:33,284 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined memory monitor
2024-06-25 17:25:33,284 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined network monitor
2024-06-25 17:25:33,285 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:25:34,285 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:34,822 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:39,287 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:39,823 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:41,640 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: resume
2024-06-25 17:25:41,640 INFO    HandlerThread:149 [handler.py:handle_request_resume():699] starting system metrics thread
2024-06-25 17:25:41,640 INFO    HandlerThread:149 [system_monitor.py:start():194] Starting system monitor
2024-06-25 17:25:41,640 INFO    SystemMonitor:149 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-06-25 17:25:41,641 INFO    SystemMonitor:149 [interfaces.py:start():190] Started cpu monitoring
2024-06-25 17:25:41,642 INFO    SystemMonitor:149 [interfaces.py:start():190] Started disk monitoring
2024-06-25 17:25:41,642 INFO    SystemMonitor:149 [interfaces.py:start():190] Started gpu monitoring
2024-06-25 17:25:41,643 INFO    SystemMonitor:149 [interfaces.py:start():190] Started memory monitoring
2024-06-25 17:25:41,645 INFO    SystemMonitor:149 [interfaces.py:start():190] Started network monitoring
2024-06-25 17:25:41,698 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: pause
2024-06-25 17:25:41,698 INFO    HandlerThread:149 [handler.py:handle_request_pause():708] stopping system metrics thread
2024-06-25 17:25:41,698 INFO    HandlerThread:149 [system_monitor.py:finish():203] Stopping system monitor
2024-06-25 17:25:41,698 DEBUG   SystemMonitor:149 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-06-25 17:25:41,700 DEBUG   SystemMonitor:149 [system_monitor.py:_start():179] Finished system metrics aggregation loop
2024-06-25 17:25:41,701 DEBUG   SystemMonitor:149 [system_monitor.py:_start():183] Publishing last batch of metrics
2024-06-25 17:25:41,702 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined cpu monitor
2024-06-25 17:25:41,702 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined disk monitor
2024-06-25 17:25:41,708 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined gpu monitor
2024-06-25 17:25:41,708 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined memory monitor
2024-06-25 17:25:41,709 INFO    HandlerThread:149 [interfaces.py:finish():202] Joined network monitor
2024-06-25 17:25:41,709 DEBUG   SenderThread:149 [sender.py:send():382] send: stats
2024-06-25 17:25:44,710 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:44,824 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:49,711 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:49,825 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:54,712 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:54,826 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:25:59,712 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:25:59,827 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:04,713 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:04,828 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:09,714 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:09,829 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:14,715 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:14,830 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:19,715 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:19,832 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:24,716 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:24,833 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:29,717 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:29,834 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:34,718 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:34,836 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:35,742 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: resume
2024-06-25 17:26:35,743 INFO    HandlerThread:149 [handler.py:handle_request_resume():699] starting system metrics thread
2024-06-25 17:26:35,743 INFO    HandlerThread:149 [system_monitor.py:start():194] Starting system monitor
2024-06-25 17:26:35,743 INFO    SystemMonitor:149 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-06-25 17:26:35,744 INFO    SystemMonitor:149 [interfaces.py:start():190] Started cpu monitoring
2024-06-25 17:26:35,744 INFO    SystemMonitor:149 [interfaces.py:start():190] Started disk monitoring
2024-06-25 17:26:35,746 INFO    SystemMonitor:149 [interfaces.py:start():190] Started gpu monitoring
2024-06-25 17:26:35,747 INFO    SystemMonitor:149 [interfaces.py:start():190] Started memory monitoring
2024-06-25 17:26:35,748 INFO    SystemMonitor:149 [interfaces.py:start():190] Started network monitoring
2024-06-25 17:26:39,719 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:39,837 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:44,720 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:44,838 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:49,720 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:49,839 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:54,721 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:54,840 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:26:59,722 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:26:59,842 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:27:04,723 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:27:04,843 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:27:09,723 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:27:09,844 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:27:14,724 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:27:14,845 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:27:19,725 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:27:19,847 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive
2024-06-25 17:27:24,725 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: status_report
2024-06-25 17:27:24,849 DEBUG   HandlerThread:149 [handler.py:handle_request():146] handle_request: keepalive