Training in progress, step 550
Browse files
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 485755120
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9378a0e34cf44e5370a7eda16c22a9f57f806fde847b668e6efe88e6125235ba
|
| 3 |
size 485755120
|
wandb/debug-internal.log
CHANGED
|
@@ -559,3 +559,53 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
|
|
| 559 |
2024-06-02 21:42:31,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 560 |
2024-06-02 21:42:32,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/config.yaml
|
| 561 |
2024-06-02 21:42:37,356 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 559 |
2024-06-02 21:42:31,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 560 |
2024-06-02 21:42:32,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/config.yaml
|
| 561 |
2024-06-02 21:42:37,356 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 562 |
+
2024-06-02 21:42:42,357 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 563 |
+
2024-06-02 21:42:43,554 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 564 |
+
2024-06-02 21:42:44,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 565 |
+
2024-06-02 21:42:45,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 566 |
+
2024-06-02 21:42:45,107 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 567 |
+
2024-06-02 21:42:48,145 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 568 |
+
2024-06-02 21:42:53,146 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 569 |
+
2024-06-02 21:42:58,147 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 570 |
+
2024-06-02 21:42:59,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 571 |
+
2024-06-02 21:43:00,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 572 |
+
2024-06-02 21:43:00,107 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 573 |
+
2024-06-02 21:43:03,176 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 574 |
+
2024-06-02 21:43:08,177 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 575 |
+
2024-06-02 21:43:13,178 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 576 |
+
2024-06-02 21:43:13,555 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 577 |
+
2024-06-02 21:43:14,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 578 |
+
2024-06-02 21:43:15,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 579 |
+
2024-06-02 21:43:15,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 580 |
+
2024-06-02 21:43:19,173 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 581 |
+
2024-06-02 21:43:24,174 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 582 |
+
2024-06-02 21:43:29,175 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 583 |
+
2024-06-02 21:43:29,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 584 |
+
2024-06-02 21:43:30,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 585 |
+
2024-06-02 21:43:30,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 586 |
+
2024-06-02 21:43:35,146 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 587 |
+
2024-06-02 21:43:40,147 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 588 |
+
2024-06-02 21:43:43,555 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 589 |
+
2024-06-02 21:43:44,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 590 |
+
2024-06-02 21:43:45,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 591 |
+
2024-06-02 21:43:45,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 592 |
+
2024-06-02 21:43:45,188 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 593 |
+
2024-06-02 21:43:50,190 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 594 |
+
2024-06-02 21:43:55,190 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 595 |
+
2024-06-02 21:43:59,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 596 |
+
2024-06-02 21:44:00,108 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 597 |
+
2024-06-02 21:44:00,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 598 |
+
2024-06-02 21:44:01,137 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 599 |
+
2024-06-02 21:44:06,138 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 600 |
+
2024-06-02 21:44:11,139 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 601 |
+
2024-06-02 21:44:13,557 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 602 |
+
2024-06-02 21:44:15,416 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 603 |
+
2024-06-02 21:44:15,417 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 604 |
+
2024-06-02 21:44:15,457 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 605 |
+
2024-06-02 21:44:16,501 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 606 |
+
2024-06-02 21:44:19,949 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 607 |
+
2024-06-02 21:44:22,365 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 608 |
+
2024-06-02 21:44:27,366 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 609 |
+
2024-06-02 21:44:29,904 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 610 |
+
2024-06-02 21:44:30,232 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 611 |
+
2024-06-02 21:44:30,232 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
wandb/run-20240602_212342-ypv2rcj7/files/output.log
CHANGED
|
@@ -20,4 +20,6 @@
|
|
| 20 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 21 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
| 22 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
|
|
|
|
|
|
| 23 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
|
|
|
| 20 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 21 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
| 22 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 23 |
+
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
| 24 |
+
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 25 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
wandb/run-20240602_212342-ypv2rcj7/logs/debug-internal.log
CHANGED
|
@@ -559,3 +559,53 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
|
|
| 559 |
2024-06-02 21:42:31,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 560 |
2024-06-02 21:42:32,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/config.yaml
|
| 561 |
2024-06-02 21:42:37,356 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 559 |
2024-06-02 21:42:31,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 560 |
2024-06-02 21:42:32,908 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/config.yaml
|
| 561 |
2024-06-02 21:42:37,356 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 562 |
+
2024-06-02 21:42:42,357 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 563 |
+
2024-06-02 21:42:43,554 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 564 |
+
2024-06-02 21:42:44,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 565 |
+
2024-06-02 21:42:45,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 566 |
+
2024-06-02 21:42:45,107 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 567 |
+
2024-06-02 21:42:48,145 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 568 |
+
2024-06-02 21:42:53,146 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 569 |
+
2024-06-02 21:42:58,147 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 570 |
+
2024-06-02 21:42:59,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 571 |
+
2024-06-02 21:43:00,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 572 |
+
2024-06-02 21:43:00,107 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 573 |
+
2024-06-02 21:43:03,176 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 574 |
+
2024-06-02 21:43:08,177 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 575 |
+
2024-06-02 21:43:13,178 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 576 |
+
2024-06-02 21:43:13,555 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 577 |
+
2024-06-02 21:43:14,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 578 |
+
2024-06-02 21:43:15,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 579 |
+
2024-06-02 21:43:15,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 580 |
+
2024-06-02 21:43:19,173 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 581 |
+
2024-06-02 21:43:24,174 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 582 |
+
2024-06-02 21:43:29,175 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 583 |
+
2024-06-02 21:43:29,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 584 |
+
2024-06-02 21:43:30,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 585 |
+
2024-06-02 21:43:30,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 586 |
+
2024-06-02 21:43:35,146 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 587 |
+
2024-06-02 21:43:40,147 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 588 |
+
2024-06-02 21:43:43,555 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 589 |
+
2024-06-02 21:43:44,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 590 |
+
2024-06-02 21:43:45,107 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 591 |
+
2024-06-02 21:43:45,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 592 |
+
2024-06-02 21:43:45,188 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 593 |
+
2024-06-02 21:43:50,190 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 594 |
+
2024-06-02 21:43:55,190 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 595 |
+
2024-06-02 21:43:59,897 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 596 |
+
2024-06-02 21:44:00,108 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 597 |
+
2024-06-02 21:44:00,108 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 598 |
+
2024-06-02 21:44:01,137 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 599 |
+
2024-06-02 21:44:06,138 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 600 |
+
2024-06-02 21:44:11,139 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 601 |
+
2024-06-02 21:44:13,557 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 602 |
+
2024-06-02 21:44:15,416 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 603 |
+
2024-06-02 21:44:15,417 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 604 |
+
2024-06-02 21:44:15,457 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 605 |
+
2024-06-02 21:44:16,501 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 606 |
+
2024-06-02 21:44:19,949 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 607 |
+
2024-06-02 21:44:22,365 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 608 |
+
2024-06-02 21:44:27,366 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 609 |
+
2024-06-02 21:44:29,904 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 610 |
+
2024-06-02 21:44:30,232 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 611 |
+
2024-06-02 21:44:30,232 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb
CHANGED
|
Binary files a/wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb and b/wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb differ
|
|
|