nornor02 commited on
Commit
8d8ffc2
·
verified ·
1 Parent(s): b880bcb

Training in progress, step 5000

Browse files
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4b6c29fbfce895408d6996c09ce561e957c5cb99ce1218f1ff1ab122ce7b6255
3
  size 328693404
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88935bf475e67d3135fec89b5e656d0e8a87445bcf72f8a428c36e6599229dc2
3
  size 328693404
runs/Jan12_02-28-34_ea19ce76d6cc/events.out.tfevents.1705026515.ea19ce76d6cc.10822.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5ba8ef601ea4a898435fee2dd11f0c9d9f63325fdd161c49f5d9669c1ca578e0
3
- size 5778
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0706f461f7cdfc8e6d0ec72985f42b8df67fbe5a9ff222d2c403ae7df55d9551
3
+ size 5935
wandb/debug-internal.log CHANGED
@@ -455,3 +455,44 @@
455
  2024-01-12 02:41:50,453 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
456
  2024-01-12 02:41:52,581 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
457
  2024-01-12 02:41:55,454 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
455
  2024-01-12 02:41:50,453 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
456
  2024-01-12 02:41:52,581 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
457
  2024-01-12 02:41:55,454 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
458
+ 2024-01-12 02:41:57,583 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
459
+ 2024-01-12 02:42:00,455 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
460
+ 2024-01-12 02:42:02,584 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
461
+ 2024-01-12 02:42:05,455 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
462
+ 2024-01-12 02:42:07,585 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
463
+ 2024-01-12 02:42:08,310 DEBUG SenderThread:10874 [sender.py:send():382] send: stats
464
+ 2024-01-12 02:42:11,311 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
465
+ 2024-01-12 02:42:12,586 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
466
+ 2024-01-12 02:42:16,312 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
467
+ 2024-01-12 02:42:17,587 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
468
+ 2024-01-12 02:42:21,313 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
469
+ 2024-01-12 02:42:22,588 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
470
+ 2024-01-12 02:42:26,314 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
471
+ 2024-01-12 02:42:27,589 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
472
+ 2024-01-12 02:42:31,315 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
473
+ 2024-01-12 02:42:32,590 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
474
+ 2024-01-12 02:42:36,315 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
475
+ 2024-01-12 02:42:37,592 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
476
+ 2024-01-12 02:42:38,311 DEBUG SenderThread:10874 [sender.py:send():382] send: stats
477
+ 2024-01-12 02:42:42,312 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
478
+ 2024-01-12 02:42:42,592 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
479
+ 2024-01-12 02:42:47,313 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
480
+ 2024-01-12 02:42:47,593 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
481
+ 2024-01-12 02:42:52,314 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
482
+ 2024-01-12 02:42:52,594 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
483
+ 2024-01-12 02:42:57,315 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
484
+ 2024-01-12 02:42:57,595 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
485
+ 2024-01-12 02:43:02,316 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
486
+ 2024-01-12 02:43:02,575 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: partial_history
487
+ 2024-01-12 02:43:02,576 DEBUG SenderThread:10874 [sender.py:send():382] send: history
488
+ 2024-01-12 02:43:02,577 DEBUG SenderThread:10874 [sender.py:send_request():409] send_request: summary_record
489
+ 2024-01-12 02:43:02,577 INFO SenderThread:10874 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
490
+ 2024-01-12 02:43:02,596 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
491
+ 2024-01-12 02:43:03,503 INFO Thread-12 :10874 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240112_022837-twhkj0lf/files/wandb-summary.json
492
+ 2024-01-12 02:43:04,504 INFO Thread-12 :10874 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240112_022837-twhkj0lf/files/output.log
493
+ 2024-01-12 02:43:06,504 INFO Thread-12 :10874 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240112_022837-twhkj0lf/files/output.log
494
+ 2024-01-12 02:43:07,317 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
495
+ 2024-01-12 02:43:07,879 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
496
+ 2024-01-12 02:43:08,313 DEBUG SenderThread:10874 [sender.py:send():382] send: stats
497
+ 2024-01-12 02:43:12,880 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
498
+ 2024-01-12 02:43:13,314 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240112_022837-twhkj0lf/files/output.log CHANGED
@@ -26,4 +26,7 @@ Checkpoint destination directory /kaggle/working/checkpoint-4000 already exists
26
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
27
  warnings.warn('Was asked to gather along dimension 0, but all '
28
  Checkpoint destination directory /kaggle/working/checkpoint-4500 already exists and is non-empty.Saving will proceed but saved results may be invalid.
 
 
 
29
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
 
26
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
27
  warnings.warn('Was asked to gather along dimension 0, but all '
28
  Checkpoint destination directory /kaggle/working/checkpoint-4500 already exists and is non-empty.Saving will proceed but saved results may be invalid.
29
+ /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
30
+ warnings.warn('Was asked to gather along dimension 0, but all '
31
+ Checkpoint destination directory /kaggle/working/checkpoint-5000 already exists and is non-empty.Saving will proceed but saved results may be invalid.
32
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
wandb/run-20240112_022837-twhkj0lf/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.4679, "train/learning_rate": 2.6923076923076923e-06, "train/epoch": 346.15, "train/global_step": 4500, "_timestamp": 1705027298.665949, "_runtime": 781.0069971084595, "_step": 8}
 
1
+ {"train/loss": 0.4462, "train/learning_rate": 7.692307692307694e-07, "train/epoch": 384.62, "train/global_step": 5000, "_timestamp": 1705027382.574807, "_runtime": 864.9158549308777, "_step": 9}
wandb/run-20240112_022837-twhkj0lf/logs/debug-internal.log CHANGED
@@ -455,3 +455,44 @@
455
  2024-01-12 02:41:50,453 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
456
  2024-01-12 02:41:52,581 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
457
  2024-01-12 02:41:55,454 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
455
  2024-01-12 02:41:50,453 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
456
  2024-01-12 02:41:52,581 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
457
  2024-01-12 02:41:55,454 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
458
+ 2024-01-12 02:41:57,583 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
459
+ 2024-01-12 02:42:00,455 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
460
+ 2024-01-12 02:42:02,584 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
461
+ 2024-01-12 02:42:05,455 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
462
+ 2024-01-12 02:42:07,585 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
463
+ 2024-01-12 02:42:08,310 DEBUG SenderThread:10874 [sender.py:send():382] send: stats
464
+ 2024-01-12 02:42:11,311 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
465
+ 2024-01-12 02:42:12,586 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
466
+ 2024-01-12 02:42:16,312 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
467
+ 2024-01-12 02:42:17,587 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
468
+ 2024-01-12 02:42:21,313 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
469
+ 2024-01-12 02:42:22,588 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
470
+ 2024-01-12 02:42:26,314 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
471
+ 2024-01-12 02:42:27,589 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
472
+ 2024-01-12 02:42:31,315 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
473
+ 2024-01-12 02:42:32,590 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
474
+ 2024-01-12 02:42:36,315 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
475
+ 2024-01-12 02:42:37,592 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
476
+ 2024-01-12 02:42:38,311 DEBUG SenderThread:10874 [sender.py:send():382] send: stats
477
+ 2024-01-12 02:42:42,312 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
478
+ 2024-01-12 02:42:42,592 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
479
+ 2024-01-12 02:42:47,313 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
480
+ 2024-01-12 02:42:47,593 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
481
+ 2024-01-12 02:42:52,314 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
482
+ 2024-01-12 02:42:52,594 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
483
+ 2024-01-12 02:42:57,315 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
484
+ 2024-01-12 02:42:57,595 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
485
+ 2024-01-12 02:43:02,316 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
486
+ 2024-01-12 02:43:02,575 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: partial_history
487
+ 2024-01-12 02:43:02,576 DEBUG SenderThread:10874 [sender.py:send():382] send: history
488
+ 2024-01-12 02:43:02,577 DEBUG SenderThread:10874 [sender.py:send_request():409] send_request: summary_record
489
+ 2024-01-12 02:43:02,577 INFO SenderThread:10874 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
490
+ 2024-01-12 02:43:02,596 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
491
+ 2024-01-12 02:43:03,503 INFO Thread-12 :10874 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240112_022837-twhkj0lf/files/wandb-summary.json
492
+ 2024-01-12 02:43:04,504 INFO Thread-12 :10874 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240112_022837-twhkj0lf/files/output.log
493
+ 2024-01-12 02:43:06,504 INFO Thread-12 :10874 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240112_022837-twhkj0lf/files/output.log
494
+ 2024-01-12 02:43:07,317 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
495
+ 2024-01-12 02:43:07,879 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
496
+ 2024-01-12 02:43:08,313 DEBUG SenderThread:10874 [sender.py:send():382] send: stats
497
+ 2024-01-12 02:43:12,880 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: keepalive
498
+ 2024-01-12 02:43:13,314 DEBUG HandlerThread:10874 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240112_022837-twhkj0lf/run-twhkj0lf.wandb CHANGED
Binary files a/wandb/run-20240112_022837-twhkj0lf/run-twhkj0lf.wandb and b/wandb/run-20240112_022837-twhkj0lf/run-twhkj0lf.wandb differ