sharukat commited on
Commit
32a6cff
·
verified ·
1 Parent(s): e7babcd

Training in progress, epoch 5

Browse files
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:825444113828aa3126056bf7424ebc69982c0fbbb99254dd9490e178121cc37c
3
  size 502675828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:807acdaab6f08df8df2340ce1a5291d0d0c3827304fb291d9ef3ce64596255bc
3
  size 502675828
runs/Mar06_16-01-10_30b2b3b8a538/events.out.tfevents.1709740872.30b2b3b8a538.186.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3e73eadcd2403bda53179cd86c46446bab5b4fc5c0bff5cd84d427e2ee1d426f
3
- size 7899
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7a3b7301de362ef3d682e852507178254d1f5b677dc889bfb7ffd6f711a2d87
3
+ size 8582
wandb/debug-internal.log CHANGED
@@ -242,3 +242,55 @@
242
  2024-03-06 16:06:37,802 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
243
  2024-03-06 16:06:39,877 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
244
  2024-03-06 16:06:42,803 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
242
  2024-03-06 16:06:37,802 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
243
  2024-03-06 16:06:39,877 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
244
  2024-03-06 16:06:42,803 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
245
+ 2024-03-06 16:06:44,878 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
246
+ 2024-03-06 16:06:46,348 DEBUG SenderThread:249 [sender.py:send():382] send: stats
247
+ 2024-03-06 16:06:47,804 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
248
+ 2024-03-06 16:06:50,350 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
249
+ 2024-03-06 16:06:52,805 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
250
+ 2024-03-06 16:06:55,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
251
+ 2024-03-06 16:06:57,807 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
252
+ 2024-03-06 16:07:00,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
253
+ 2024-03-06 16:07:02,808 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
254
+ 2024-03-06 16:07:05,352 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
255
+ 2024-03-06 16:07:07,809 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
256
+ 2024-03-06 16:07:10,353 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
257
+ 2024-03-06 16:07:12,819 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
258
+ 2024-03-06 16:07:15,354 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
259
+ 2024-03-06 16:07:16,349 DEBUG SenderThread:249 [sender.py:send():382] send: stats
260
+ 2024-03-06 16:07:17,820 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
261
+ 2024-03-06 16:07:21,350 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
262
+ 2024-03-06 16:07:22,821 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
263
+ 2024-03-06 16:07:26,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
264
+ 2024-03-06 16:07:27,822 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
265
+ 2024-03-06 16:07:31,336 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: partial_history
266
+ 2024-03-06 16:07:31,337 DEBUG SenderThread:249 [sender.py:send():382] send: history
267
+ 2024-03-06 16:07:31,337 DEBUG SenderThread:249 [sender.py:send_request():409] send_request: summary_record
268
+ 2024-03-06 16:07:31,338 INFO SenderThread:249 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
269
+ 2024-03-06 16:07:31,369 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json
270
+ 2024-03-06 16:07:32,339 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
271
+ 2024-03-06 16:07:32,832 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
272
+ 2024-03-06 16:07:33,817 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: partial_history
273
+ 2024-03-06 16:07:33,819 DEBUG SenderThread:249 [sender.py:send():382] send: history
274
+ 2024-03-06 16:07:33,819 DEBUG SenderThread:249 [sender.py:send_request():409] send_request: summary_record
275
+ 2024-03-06 16:07:33,820 INFO SenderThread:249 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
276
+ 2024-03-06 16:07:34,371 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json
277
+ 2024-03-06 16:07:35,371 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/output.log
278
+ 2024-03-06 16:07:37,821 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
279
+ 2024-03-06 16:07:37,902 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
280
+ 2024-03-06 16:07:39,215 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: partial_history
281
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
282
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
283
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
284
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
285
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
286
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: history
287
+ 2024-03-06 16:07:39,219 DEBUG SenderThread:249 [sender.py:send_request():409] send_request: summary_record
288
+ 2024-03-06 16:07:39,219 INFO SenderThread:249 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
289
+ 2024-03-06 16:07:39,373 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json
290
+ 2024-03-06 16:07:42,903 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
291
+ 2024-03-06 16:07:43,220 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
292
+ 2024-03-06 16:07:46,350 DEBUG SenderThread:249 [sender.py:send():382] send: stats
293
+ 2024-03-06 16:07:47,905 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
294
+ 2024-03-06 16:07:48,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
295
+ 2024-03-06 16:07:52,906 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
296
+ 2024-03-06 16:07:53,352 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240306_160115-aj5k3nji/files/output.log CHANGED
@@ -4,4 +4,6 @@
4
  _warn_prf(average, modifier, msg_start, len(result))
5
  /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
6
  _warn_prf(average, modifier, msg_start, len(result))
 
 
7
  /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
 
4
  _warn_prf(average, modifier, msg_start, len(result))
5
  /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
6
  _warn_prf(average, modifier, msg_start, len(result))
7
+ /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
8
+ _warn_prf(average, modifier, msg_start, len(result))
9
  /opt/conda/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 1.4479, "train/grad_norm": 11.683911323547363, "train/learning_rate": 4.086956521739131e-06, "train/epoch": 4.0, "train/global_step": 552, "_timestamp": 1709741182.8713813, "_runtime": 307.8046941757202, "_step": 7, "eval/loss": 1.6020480394363403, "eval/accuracy": 0.43902439024390244, "eval/precision": 0.3063372941421722, "eval/recall": 0.43902439024390244, "eval/f1": 0.35384128067054893, "eval/runtime": 2.468, "eval/samples_per_second": 49.837, "eval/steps_per_second": 6.483}
 
1
+ {"train/loss": 1.4019, "train/grad_norm": 12.517467498779297, "train/learning_rate": 8.695652173913044e-08, "train/epoch": 5.0, "train/global_step": 690, "_timestamp": 1709741259.2145789, "_runtime": 384.14789175987244, "_step": 10, "eval/loss": 1.6076624393463135, "eval/accuracy": 0.44715447154471544, "eval/precision": 0.3170395646722763, "eval/recall": 0.44715447154471544, "eval/f1": 0.3620794803367881, "eval/runtime": 2.4768, "eval/samples_per_second": 49.661, "eval/steps_per_second": 6.46, "train/train_runtime": 386.9109, "train/train_samples_per_second": 14.189, "train/train_steps_per_second": 1.783, "train/total_flos": 1444544540928000.0, "train/train_loss": 1.5175944452700407}
wandb/run-20240306_160115-aj5k3nji/logs/debug-internal.log CHANGED
@@ -242,3 +242,55 @@
242
  2024-03-06 16:06:37,802 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
243
  2024-03-06 16:06:39,877 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
244
  2024-03-06 16:06:42,803 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
242
  2024-03-06 16:06:37,802 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
243
  2024-03-06 16:06:39,877 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
244
  2024-03-06 16:06:42,803 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
245
+ 2024-03-06 16:06:44,878 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
246
+ 2024-03-06 16:06:46,348 DEBUG SenderThread:249 [sender.py:send():382] send: stats
247
+ 2024-03-06 16:06:47,804 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
248
+ 2024-03-06 16:06:50,350 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
249
+ 2024-03-06 16:06:52,805 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
250
+ 2024-03-06 16:06:55,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
251
+ 2024-03-06 16:06:57,807 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
252
+ 2024-03-06 16:07:00,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
253
+ 2024-03-06 16:07:02,808 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
254
+ 2024-03-06 16:07:05,352 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
255
+ 2024-03-06 16:07:07,809 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
256
+ 2024-03-06 16:07:10,353 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
257
+ 2024-03-06 16:07:12,819 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
258
+ 2024-03-06 16:07:15,354 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
259
+ 2024-03-06 16:07:16,349 DEBUG SenderThread:249 [sender.py:send():382] send: stats
260
+ 2024-03-06 16:07:17,820 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
261
+ 2024-03-06 16:07:21,350 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
262
+ 2024-03-06 16:07:22,821 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
263
+ 2024-03-06 16:07:26,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
264
+ 2024-03-06 16:07:27,822 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
265
+ 2024-03-06 16:07:31,336 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: partial_history
266
+ 2024-03-06 16:07:31,337 DEBUG SenderThread:249 [sender.py:send():382] send: history
267
+ 2024-03-06 16:07:31,337 DEBUG SenderThread:249 [sender.py:send_request():409] send_request: summary_record
268
+ 2024-03-06 16:07:31,338 INFO SenderThread:249 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
269
+ 2024-03-06 16:07:31,369 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json
270
+ 2024-03-06 16:07:32,339 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
271
+ 2024-03-06 16:07:32,832 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
272
+ 2024-03-06 16:07:33,817 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: partial_history
273
+ 2024-03-06 16:07:33,819 DEBUG SenderThread:249 [sender.py:send():382] send: history
274
+ 2024-03-06 16:07:33,819 DEBUG SenderThread:249 [sender.py:send_request():409] send_request: summary_record
275
+ 2024-03-06 16:07:33,820 INFO SenderThread:249 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
276
+ 2024-03-06 16:07:34,371 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json
277
+ 2024-03-06 16:07:35,371 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/output.log
278
+ 2024-03-06 16:07:37,821 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
279
+ 2024-03-06 16:07:37,902 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
280
+ 2024-03-06 16:07:39,215 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: partial_history
281
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
282
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
283
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
284
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
285
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: metric
286
+ 2024-03-06 16:07:39,218 DEBUG SenderThread:249 [sender.py:send():382] send: history
287
+ 2024-03-06 16:07:39,219 DEBUG SenderThread:249 [sender.py:send_request():409] send_request: summary_record
288
+ 2024-03-06 16:07:39,219 INFO SenderThread:249 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
289
+ 2024-03-06 16:07:39,373 INFO Thread-12 :249 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240306_160115-aj5k3nji/files/wandb-summary.json
290
+ 2024-03-06 16:07:42,903 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
291
+ 2024-03-06 16:07:43,220 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
292
+ 2024-03-06 16:07:46,350 DEBUG SenderThread:249 [sender.py:send():382] send: stats
293
+ 2024-03-06 16:07:47,905 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
294
+ 2024-03-06 16:07:48,351 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
295
+ 2024-03-06 16:07:52,906 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: keepalive
296
+ 2024-03-06 16:07:53,352 DEBUG HandlerThread:249 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240306_160115-aj5k3nji/run-aj5k3nji.wandb CHANGED
Binary files a/wandb/run-20240306_160115-aj5k3nji/run-aj5k3nji.wandb and b/wandb/run-20240306_160115-aj5k3nji/run-aj5k3nji.wandb differ