sharukat commited on
Commit
d68917d
·
verified ·
1 Parent(s): b7f3d2b

Training in progress, epoch 3

Browse files
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ef7bced2bc60385c21f03fcae9691a5b296c12a7e761e636ae2518088d067e8c
3
  size 502675828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:595387e04ed4c89ad2261cabc84ada41dc85f9c625a8fe84e799366424331531
3
  size 502675828
runs/Mar06_14-59-58_41759fa8e6ad/events.out.tfevents.1709737199.41759fa8e6ad.34.3 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:494669d6f22754997b246b9b888db23db9911c97b050782fb231af3e436332d6
3
- size 6508
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1a7c761ef214f4dc02a43fc9c266f7bdb50e1f21385d23682b2bdd1f69d5db9
3
+ size 7191
wandb/debug-internal.log CHANGED
@@ -398,3 +398,34 @@
398
  2024-03-06 15:01:16,490 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
399
  2024-03-06 15:01:20,271 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
400
  2024-03-06 15:01:20,636 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
398
  2024-03-06 15:01:16,490 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
399
  2024-03-06 15:01:20,271 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
400
  2024-03-06 15:01:20,636 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
401
+ 2024-03-06 15:01:21,491 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
402
+ 2024-03-06 15:01:25,272 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
403
+ 2024-03-06 15:01:25,637 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
404
+ 2024-03-06 15:01:26,492 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
405
+ 2024-03-06 15:01:26,789 DEBUG SenderThread:137 [sender.py:send():382] send: stats
406
+ 2024-03-06 15:01:30,272 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
407
+ 2024-03-06 15:01:30,638 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
408
+ 2024-03-06 15:01:31,790 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
409
+ 2024-03-06 15:01:35,273 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
410
+ 2024-03-06 15:01:35,559 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: partial_history
411
+ 2024-03-06 15:01:35,561 DEBUG SenderThread:137 [sender.py:send():382] send: history
412
+ 2024-03-06 15:01:35,561 DEBUG SenderThread:137 [sender.py:send_request():409] send_request: summary_record
413
+ 2024-03-06 15:01:35,561 INFO SenderThread:137 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
414
+ 2024-03-06 15:01:35,638 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
415
+ 2024-03-06 15:01:35,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json
416
+ 2024-03-06 15:01:36,671 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: partial_history
417
+ 2024-03-06 15:01:36,672 DEBUG SenderThread:137 [sender.py:send():382] send: history
418
+ 2024-03-06 15:01:36,672 DEBUG SenderThread:137 [sender.py:send_request():409] send_request: summary_record
419
+ 2024-03-06 15:01:36,673 INFO SenderThread:137 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
420
+ 2024-03-06 15:01:36,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json
421
+ 2024-03-06 15:01:37,674 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
422
+ 2024-03-06 15:01:37,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/output.log
423
+ 2024-03-06 15:01:40,274 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
424
+ 2024-03-06 15:01:40,910 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
425
+ 2024-03-06 15:01:42,675 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
426
+ 2024-03-06 15:01:45,275 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
427
+ 2024-03-06 15:01:45,911 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
428
+ 2024-03-06 15:01:47,676 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
429
+ 2024-03-06 15:01:50,276 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
430
+ 2024-03-06 15:01:50,912 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
431
+ 2024-03-06 15:01:52,677 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240306_145424-trm7fvg4/logs/debug-internal.log CHANGED
@@ -449,3 +449,34 @@ wandb.errors.AuthenticationError: The API key you provided is either invalid or
449
  2024-03-06 15:01:16,490 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
450
  2024-03-06 15:01:20,271 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
451
  2024-03-06 15:01:20,636 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
449
  2024-03-06 15:01:16,490 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
450
  2024-03-06 15:01:20,271 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
451
  2024-03-06 15:01:20,636 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
452
+ 2024-03-06 15:01:21,491 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
453
+ 2024-03-06 15:01:25,272 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
454
+ 2024-03-06 15:01:25,637 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
455
+ 2024-03-06 15:01:26,492 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
456
+ 2024-03-06 15:01:26,789 DEBUG SenderThread:137 [sender.py:send():382] send: stats
457
+ 2024-03-06 15:01:30,272 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
458
+ 2024-03-06 15:01:30,638 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
459
+ 2024-03-06 15:01:31,790 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
460
+ 2024-03-06 15:01:35,273 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
461
+ 2024-03-06 15:01:35,559 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: partial_history
462
+ 2024-03-06 15:01:35,561 DEBUG SenderThread:137 [sender.py:send():382] send: history
463
+ 2024-03-06 15:01:35,561 DEBUG SenderThread:137 [sender.py:send_request():409] send_request: summary_record
464
+ 2024-03-06 15:01:35,561 INFO SenderThread:137 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
465
+ 2024-03-06 15:01:35,638 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
466
+ 2024-03-06 15:01:35,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json
467
+ 2024-03-06 15:01:36,671 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: partial_history
468
+ 2024-03-06 15:01:36,672 DEBUG SenderThread:137 [sender.py:send():382] send: history
469
+ 2024-03-06 15:01:36,672 DEBUG SenderThread:137 [sender.py:send_request():409] send_request: summary_record
470
+ 2024-03-06 15:01:36,673 INFO SenderThread:137 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
471
+ 2024-03-06 15:01:36,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json
472
+ 2024-03-06 15:01:37,674 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
473
+ 2024-03-06 15:01:37,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/output.log
474
+ 2024-03-06 15:01:40,274 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
475
+ 2024-03-06 15:01:40,910 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
476
+ 2024-03-06 15:01:42,675 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
477
+ 2024-03-06 15:01:45,275 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
478
+ 2024-03-06 15:01:45,911 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
479
+ 2024-03-06 15:01:47,676 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
480
+ 2024-03-06 15:01:50,276 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
481
+ 2024-03-06 15:01:50,912 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
482
+ 2024-03-06 15:01:52,677 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240306_145455-h1uv5tyi/files/output.log CHANGED
@@ -17,3 +17,6 @@ You should probably TRAIN this model on a down-stream task to be able to use it
17
  Checkpoint destination directory /kaggle/working/checkpoint-62 already exists and is non-empty. Saving will proceed but saved results may be invalid.
18
  /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
19
  _warn_prf(average, modifier, msg_start, len(result))
 
 
 
 
17
  Checkpoint destination directory /kaggle/working/checkpoint-62 already exists and is non-empty. Saving will proceed but saved results may be invalid.
18
  /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
19
  _warn_prf(average, modifier, msg_start, len(result))
20
+ Checkpoint destination directory /kaggle/working/checkpoint-124 already exists and is non-empty. Saving will proceed but saved results may be invalid.
21
+ /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
22
+ _warn_prf(average, modifier, msg_start, len(result))
wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 1.7375, "train/grad_norm": 11.856295585632324, "train/learning_rate": 8.016129032258066e-06, "train/epoch": 2.0, "train/global_step": 124, "_timestamp": 1709737262.4837022, "_runtime": 366.764945268631, "_step": 15, "eval/loss": 1.7562878131866455, "eval/accuracy": 0.34545454545454546, "eval/precision": 0.11933884297520661, "eval/recall": 0.34545454545454546, "eval/f1": 0.17739557739557738, "eval/runtime": 1.1126, "eval/samples_per_second": 49.433, "eval/steps_per_second": 6.292, "train/train_runtime": 237.4526, "train/train_samples_per_second": 10.339, "train/train_steps_per_second": 1.306, "train/total_flos": 645966638976000.0, "train/train_loss": 1.7031736066264491}
 
1
+ {"train/loss": 1.7235, "train/grad_norm": 8.239474296569824, "train/learning_rate": 7.0161290322580654e-06, "train/epoch": 3.0, "train/global_step": 186, "_timestamp": 1709737296.670676, "_runtime": 400.9519190788269, "_step": 17, "eval/loss": 1.7306572198867798, "eval/accuracy": 0.36363636363636365, "eval/precision": 0.26503496503496504, "eval/recall": 0.36363636363636365, "eval/f1": 0.23334186939820742, "eval/runtime": 1.1077, "eval/samples_per_second": 49.653, "eval/steps_per_second": 6.319, "train/train_runtime": 237.4526, "train/train_samples_per_second": 10.339, "train/train_steps_per_second": 1.306, "train/total_flos": 645966638976000.0, "train/train_loss": 1.7031736066264491}
wandb/run-20240306_145455-h1uv5tyi/logs/debug-internal.log CHANGED
@@ -398,3 +398,34 @@
398
  2024-03-06 15:01:16,490 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
399
  2024-03-06 15:01:20,271 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
400
  2024-03-06 15:01:20,636 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
398
  2024-03-06 15:01:16,490 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
399
  2024-03-06 15:01:20,271 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
400
  2024-03-06 15:01:20,636 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
401
+ 2024-03-06 15:01:21,491 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
402
+ 2024-03-06 15:01:25,272 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
403
+ 2024-03-06 15:01:25,637 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
404
+ 2024-03-06 15:01:26,492 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
405
+ 2024-03-06 15:01:26,789 DEBUG SenderThread:137 [sender.py:send():382] send: stats
406
+ 2024-03-06 15:01:30,272 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
407
+ 2024-03-06 15:01:30,638 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
408
+ 2024-03-06 15:01:31,790 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
409
+ 2024-03-06 15:01:35,273 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
410
+ 2024-03-06 15:01:35,559 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: partial_history
411
+ 2024-03-06 15:01:35,561 DEBUG SenderThread:137 [sender.py:send():382] send: history
412
+ 2024-03-06 15:01:35,561 DEBUG SenderThread:137 [sender.py:send_request():409] send_request: summary_record
413
+ 2024-03-06 15:01:35,561 INFO SenderThread:137 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
414
+ 2024-03-06 15:01:35,638 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
415
+ 2024-03-06 15:01:35,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json
416
+ 2024-03-06 15:01:36,671 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: partial_history
417
+ 2024-03-06 15:01:36,672 DEBUG SenderThread:137 [sender.py:send():382] send: history
418
+ 2024-03-06 15:01:36,672 DEBUG SenderThread:137 [sender.py:send_request():409] send_request: summary_record
419
+ 2024-03-06 15:01:36,673 INFO SenderThread:137 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
420
+ 2024-03-06 15:01:36,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/wandb-summary.json
421
+ 2024-03-06 15:01:37,674 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
422
+ 2024-03-06 15:01:37,893 INFO Thread-18 :137 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_145455-h1uv5tyi/files/output.log
423
+ 2024-03-06 15:01:40,274 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
424
+ 2024-03-06 15:01:40,910 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
425
+ 2024-03-06 15:01:42,675 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
426
+ 2024-03-06 15:01:45,275 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
427
+ 2024-03-06 15:01:45,911 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
428
+ 2024-03-06 15:01:47,676 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
429
+ 2024-03-06 15:01:50,276 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
430
+ 2024-03-06 15:01:50,912 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: keepalive
431
+ 2024-03-06 15:01:52,677 DEBUG HandlerThread:137 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240306_145455-h1uv5tyi/run-h1uv5tyi.wandb CHANGED
Binary files a/wandb/run-20240306_145455-h1uv5tyi/run-h1uv5tyi.wandb and b/wandb/run-20240306_145455-h1uv5tyi/run-h1uv5tyi.wandb differ