nornor02 commited on
Commit
c1a89e6
·
verified ·
1 Parent(s): e999094

Training in progress, step 1500

Browse files
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2e6a3ffbaa2952f56c30c472405e211c525ad4d42b4fa93ac03253e1cb3b38ea
3
  size 328693404
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee32357e79d3b677aa66452e0e0d7eb519c263fc834d49316f16c67a1abce823
3
  size 328693404
runs/Jan18_17-18-07_bddbbc3e305c/events.out.tfevents.1705598291.bddbbc3e305c.76.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:75ad2cb1cd5aa530c36d5b80d07a228d59e1f4d14499e58b260e1e7abb3a767f
3
- size 9298
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01f4f562290015d907639a1c10370dea005bb133acf88a2a3b9a645232d90125
3
+ size 9455
wandb/debug-internal.log CHANGED
@@ -267,3 +267,41 @@
267
  2024-01-18 17:25:04,932 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
268
  2024-01-18 17:25:05,036 DEBUG SenderThread:127 [sender.py:send():382] send: stats
269
  2024-01-18 17:25:06,957 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
267
  2024-01-18 17:25:04,932 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
268
  2024-01-18 17:25:05,036 DEBUG SenderThread:127 [sender.py:send():382] send: stats
269
  2024-01-18 17:25:06,957 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
270
+ 2024-01-18 17:25:10,037 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
271
+ 2024-01-18 17:25:11,963 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
272
+ 2024-01-18 17:25:15,038 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
273
+ 2024-01-18 17:25:16,964 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
274
+ 2024-01-18 17:25:20,038 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
275
+ 2024-01-18 17:25:21,970 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
276
+ 2024-01-18 17:25:25,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
277
+ 2024-01-18 17:25:26,971 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
278
+ 2024-01-18 17:25:30,040 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
279
+ 2024-01-18 17:25:31,972 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
280
+ 2024-01-18 17:25:35,037 DEBUG SenderThread:127 [sender.py:send():382] send: stats
281
+ 2024-01-18 17:25:36,037 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
282
+ 2024-01-18 17:25:36,973 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
283
+ 2024-01-18 17:25:41,038 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
284
+ 2024-01-18 17:25:41,974 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
285
+ 2024-01-18 17:25:46,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
286
+ 2024-01-18 17:25:46,975 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
287
+ 2024-01-18 17:25:51,040 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
288
+ 2024-01-18 17:25:51,976 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
289
+ 2024-01-18 17:25:56,041 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
290
+ 2024-01-18 17:25:56,977 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
291
+ 2024-01-18 17:26:01,041 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
292
+ 2024-01-18 17:26:01,979 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
293
+ 2024-01-18 17:26:05,038 DEBUG SenderThread:127 [sender.py:send():382] send: stats
294
+ 2024-01-18 17:26:06,979 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
295
+ 2024-01-18 17:26:07,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
296
+ 2024-01-18 17:26:11,980 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
297
+ 2024-01-18 17:26:12,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
298
+ 2024-01-18 17:26:14,696 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: partial_history
299
+ 2024-01-18 17:26:14,697 DEBUG SenderThread:127 [sender.py:send():382] send: history
300
+ 2024-01-18 17:26:14,697 DEBUG SenderThread:127 [sender.py:send_request():409] send_request: summary_record
301
+ 2024-01-18 17:26:14,697 INFO SenderThread:127 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
302
+ 2024-01-18 17:26:15,564 INFO Thread-12 :127 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240118_171829-uvan9htf/files/wandb-summary.json
303
+ 2024-01-18 17:26:17,113 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
304
+ 2024-01-18 17:26:17,879 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
305
+ 2024-01-18 17:26:18,566 INFO Thread-12 :127 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240118_171829-uvan9htf/files/output.log
306
+ 2024-01-18 17:26:22,115 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
307
+ 2024-01-18 17:26:22,880 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240118_171829-uvan9htf/files/output.log CHANGED
@@ -9,4 +9,6 @@ Checkpoint destination directory /kaggle/working/checkpoint-500 already exists a
9
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
10
  warnings.warn('Was asked to gather along dimension 0, but all '
11
  Checkpoint destination directory /kaggle/working/checkpoint-1000 already exists and is non-empty.Saving will proceed but saved results may be invalid.
 
 
12
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
 
9
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
10
  warnings.warn('Was asked to gather along dimension 0, but all '
11
  Checkpoint destination directory /kaggle/working/checkpoint-1000 already exists and is non-empty.Saving will proceed but saved results may be invalid.
12
+ /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
13
+ warnings.warn('Was asked to gather along dimension 0, but all '
14
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
wandb/run-20240118_171829-uvan9htf/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 1.1434, "train/learning_rate": 1.230769230769231e-05, "train/epoch": 76.92, "train/global_step": 1000, "_timestamp": 1705598690.869199, "_runtime": 381.53432512283325, "_step": 3}
 
1
+ {"train/loss": 0.8873, "train/learning_rate": 8.461538461538462e-06, "train/epoch": 115.38, "train/global_step": 1500, "_timestamp": 1705598774.695556, "_runtime": 465.36068201065063, "_step": 4}
wandb/run-20240118_171829-uvan9htf/logs/debug-internal.log CHANGED
@@ -267,3 +267,41 @@
267
  2024-01-18 17:25:04,932 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
268
  2024-01-18 17:25:05,036 DEBUG SenderThread:127 [sender.py:send():382] send: stats
269
  2024-01-18 17:25:06,957 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
267
  2024-01-18 17:25:04,932 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
268
  2024-01-18 17:25:05,036 DEBUG SenderThread:127 [sender.py:send():382] send: stats
269
  2024-01-18 17:25:06,957 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
270
+ 2024-01-18 17:25:10,037 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
271
+ 2024-01-18 17:25:11,963 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
272
+ 2024-01-18 17:25:15,038 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
273
+ 2024-01-18 17:25:16,964 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
274
+ 2024-01-18 17:25:20,038 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
275
+ 2024-01-18 17:25:21,970 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
276
+ 2024-01-18 17:25:25,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
277
+ 2024-01-18 17:25:26,971 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
278
+ 2024-01-18 17:25:30,040 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
279
+ 2024-01-18 17:25:31,972 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
280
+ 2024-01-18 17:25:35,037 DEBUG SenderThread:127 [sender.py:send():382] send: stats
281
+ 2024-01-18 17:25:36,037 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
282
+ 2024-01-18 17:25:36,973 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
283
+ 2024-01-18 17:25:41,038 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
284
+ 2024-01-18 17:25:41,974 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
285
+ 2024-01-18 17:25:46,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
286
+ 2024-01-18 17:25:46,975 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
287
+ 2024-01-18 17:25:51,040 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
288
+ 2024-01-18 17:25:51,976 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
289
+ 2024-01-18 17:25:56,041 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
290
+ 2024-01-18 17:25:56,977 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
291
+ 2024-01-18 17:26:01,041 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
292
+ 2024-01-18 17:26:01,979 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
293
+ 2024-01-18 17:26:05,038 DEBUG SenderThread:127 [sender.py:send():382] send: stats
294
+ 2024-01-18 17:26:06,979 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
295
+ 2024-01-18 17:26:07,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
296
+ 2024-01-18 17:26:11,980 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
297
+ 2024-01-18 17:26:12,039 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
298
+ 2024-01-18 17:26:14,696 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: partial_history
299
+ 2024-01-18 17:26:14,697 DEBUG SenderThread:127 [sender.py:send():382] send: history
300
+ 2024-01-18 17:26:14,697 DEBUG SenderThread:127 [sender.py:send_request():409] send_request: summary_record
301
+ 2024-01-18 17:26:14,697 INFO SenderThread:127 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
302
+ 2024-01-18 17:26:15,564 INFO Thread-12 :127 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240118_171829-uvan9htf/files/wandb-summary.json
303
+ 2024-01-18 17:26:17,113 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
304
+ 2024-01-18 17:26:17,879 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
305
+ 2024-01-18 17:26:18,566 INFO Thread-12 :127 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240118_171829-uvan9htf/files/output.log
306
+ 2024-01-18 17:26:22,115 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: keepalive
307
+ 2024-01-18 17:26:22,880 DEBUG HandlerThread:127 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240118_171829-uvan9htf/run-uvan9htf.wandb CHANGED
Binary files a/wandb/run-20240118_171829-uvan9htf/run-uvan9htf.wandb and b/wandb/run-20240118_171829-uvan9htf/run-uvan9htf.wandb differ