Training in progress, step 850
Browse files
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 485755120
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ecf4095b5991265767d7e399e9fd44d66dab5cbacca344af84450a78084ffbb3
|
| 3 |
size 485755120
|
wandb/debug-internal.log
CHANGED
|
@@ -847,3 +847,49 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
|
|
| 847 |
2024-06-02 21:53:32,374 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 848 |
2024-06-02 21:53:37,375 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 849 |
2024-06-02 21:53:42,376 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 847 |
2024-06-02 21:53:32,374 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 848 |
2024-06-02 21:53:37,375 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 849 |
2024-06-02 21:53:42,376 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 850 |
+
2024-06-02 21:53:43,581 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 851 |
+
2024-06-02 21:53:44,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 852 |
+
2024-06-02 21:53:45,242 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 853 |
+
2024-06-02 21:53:45,242 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 854 |
+
2024-06-02 21:53:48,302 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 855 |
+
2024-06-02 21:53:53,303 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 856 |
+
2024-06-02 21:53:58,304 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 857 |
+
2024-06-02 21:53:59,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 858 |
+
2024-06-02 21:54:00,242 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 859 |
+
2024-06-02 21:54:00,243 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 860 |
+
2024-06-02 21:54:04,287 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 861 |
+
2024-06-02 21:54:09,288 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 862 |
+
2024-06-02 21:54:13,582 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 863 |
+
2024-06-02 21:54:14,583 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 864 |
+
2024-06-02 21:54:14,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 865 |
+
2024-06-02 21:54:15,242 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 866 |
+
2024-06-02 21:54:15,243 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 867 |
+
2024-06-02 21:54:20,266 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 868 |
+
2024-06-02 21:54:25,267 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 869 |
+
2024-06-02 21:54:29,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 870 |
+
2024-06-02 21:54:30,244 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 871 |
+
2024-06-02 21:54:30,245 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 872 |
+
2024-06-02 21:54:30,334 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 873 |
+
2024-06-02 21:54:35,335 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 874 |
+
2024-06-02 21:54:40,336 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 875 |
+
2024-06-02 21:54:43,583 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 876 |
+
2024-06-02 21:54:44,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 877 |
+
2024-06-02 21:54:45,244 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 878 |
+
2024-06-02 21:54:45,245 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 879 |
+
2024-06-02 21:54:45,374 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 880 |
+
2024-06-02 21:54:50,375 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 881 |
+
2024-06-02 21:54:55,376 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 882 |
+
2024-06-02 21:54:59,912 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 883 |
+
2024-06-02 21:55:00,244 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 884 |
+
2024-06-02 21:55:00,245 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 885 |
+
2024-06-02 21:55:01,351 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 886 |
+
2024-06-02 21:55:06,352 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 887 |
+
2024-06-02 21:55:11,353 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 888 |
+
2024-06-02 21:55:13,584 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 889 |
+
2024-06-02 21:55:14,915 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 890 |
+
2024-06-02 21:55:15,449 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 891 |
+
2024-06-02 21:55:15,450 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 892 |
+
2024-06-02 21:55:16,500 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 893 |
+
2024-06-02 21:55:21,526 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 894 |
+
2024-06-02 21:55:22,208 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 895 |
+
2024-06-02 21:55:26,527 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
wandb/run-20240602_212342-ypv2rcj7/files/output.log
CHANGED
|
@@ -32,4 +32,6 @@
|
|
| 32 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 33 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
| 34 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
|
|
|
|
|
|
| 35 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
|
|
|
| 32 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 33 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
| 34 |
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 35 |
+
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
| 36 |
+
warnings.warn('Was asked to gather along dimension 0, but all '
|
| 37 |
/opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
|
wandb/run-20240602_212342-ypv2rcj7/logs/debug-internal.log
CHANGED
|
@@ -847,3 +847,49 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
|
|
| 847 |
2024-06-02 21:53:32,374 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 848 |
2024-06-02 21:53:37,375 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 849 |
2024-06-02 21:53:42,376 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 847 |
2024-06-02 21:53:32,374 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 848 |
2024-06-02 21:53:37,375 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 849 |
2024-06-02 21:53:42,376 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 850 |
+
2024-06-02 21:53:43,581 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 851 |
+
2024-06-02 21:53:44,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 852 |
+
2024-06-02 21:53:45,242 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 853 |
+
2024-06-02 21:53:45,242 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 854 |
+
2024-06-02 21:53:48,302 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 855 |
+
2024-06-02 21:53:53,303 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 856 |
+
2024-06-02 21:53:58,304 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 857 |
+
2024-06-02 21:53:59,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 858 |
+
2024-06-02 21:54:00,242 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 859 |
+
2024-06-02 21:54:00,243 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 860 |
+
2024-06-02 21:54:04,287 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 861 |
+
2024-06-02 21:54:09,288 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 862 |
+
2024-06-02 21:54:13,582 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 863 |
+
2024-06-02 21:54:14,583 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 864 |
+
2024-06-02 21:54:14,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 865 |
+
2024-06-02 21:54:15,242 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 866 |
+
2024-06-02 21:54:15,243 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 867 |
+
2024-06-02 21:54:20,266 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 868 |
+
2024-06-02 21:54:25,267 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 869 |
+
2024-06-02 21:54:29,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 870 |
+
2024-06-02 21:54:30,244 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 871 |
+
2024-06-02 21:54:30,245 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 872 |
+
2024-06-02 21:54:30,334 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 873 |
+
2024-06-02 21:54:35,335 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 874 |
+
2024-06-02 21:54:40,336 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 875 |
+
2024-06-02 21:54:43,583 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 876 |
+
2024-06-02 21:54:44,911 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 877 |
+
2024-06-02 21:54:45,244 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 878 |
+
2024-06-02 21:54:45,245 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 879 |
+
2024-06-02 21:54:45,374 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 880 |
+
2024-06-02 21:54:50,375 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 881 |
+
2024-06-02 21:54:55,376 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 882 |
+
2024-06-02 21:54:59,912 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 883 |
+
2024-06-02 21:55:00,244 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 884 |
+
2024-06-02 21:55:00,245 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 885 |
+
2024-06-02 21:55:01,351 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 886 |
+
2024-06-02 21:55:06,352 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 887 |
+
2024-06-02 21:55:11,353 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 888 |
+
2024-06-02 21:55:13,584 DEBUG SenderThread:82 [sender.py:send():379] send: stats
|
| 889 |
+
2024-06-02 21:55:14,915 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: internal_messages
|
| 890 |
+
2024-06-02 21:55:15,449 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: stop_status
|
| 891 |
+
2024-06-02 21:55:15,450 DEBUG SenderThread:82 [sender.py:send_request():406] send_request: stop_status
|
| 892 |
+
2024-06-02 21:55:16,500 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 893 |
+
2024-06-02 21:55:21,526 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
| 894 |
+
2024-06-02 21:55:22,208 INFO Thread-12 :82 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240602_212342-ypv2rcj7/files/output.log
|
| 895 |
+
2024-06-02 21:55:26,527 DEBUG HandlerThread:82 [handler.py:handle_request():146] handle_request: status_report
|
wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb
CHANGED
|
Binary files a/wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb and b/wandb/run-20240602_212342-ypv2rcj7/run-ypv2rcj7.wandb differ
|
|
|