nornor02 commited on
Commit
276a90b
·
1 Parent(s): 8464381

Training in progress, step 1500

Browse files
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b6568526d6f510a17641b444d415b05c4f0af34d7532d14de56d3b524900ebff
3
  size 328693404
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52cedebb0d870bbc8bae4c746d81d95eb2a2f08b5c91660f3befad7d95968b3b
3
  size 328693404
runs/Dec21_15-40-36_25a3f4873e1e/events.out.tfevents.1703173236.25a3f4873e1e.42.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0a444c7cb79b5b8b40dcb71c468cbddbdd810039cf195f2c4e5073a2933d18ca
3
- size 4684
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd80c92cb81ff666f1a652f6218160dc890ec8f0015e34ab8dfd59e2498ee3bc
3
+ size 4841
wandb/debug-internal.log CHANGED
@@ -200,3 +200,70 @@
200
  2023-12-21 15:46:07,368 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
201
  2023-12-21 15:46:10,050 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
202
  2023-12-21 15:46:12,376 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  2023-12-21 15:46:07,368 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
201
  2023-12-21 15:46:10,050 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
202
  2023-12-21 15:46:12,376 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
203
+ 2023-12-21 15:46:15,051 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
204
+ 2023-12-21 15:46:17,379 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
205
+ 2023-12-21 15:46:20,052 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
206
+ 2023-12-21 15:46:20,883 DEBUG SenderThread:833 [sender.py:send():382] send: stats
207
+ 2023-12-21 15:46:22,380 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
208
+ 2023-12-21 15:46:25,884 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
209
+ 2023-12-21 15:46:27,381 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
210
+ 2023-12-21 15:46:30,885 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
211
+ 2023-12-21 15:46:32,382 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
212
+ 2023-12-21 15:46:35,886 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
213
+ 2023-12-21 15:46:37,384 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
214
+ 2023-12-21 15:46:40,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
215
+ 2023-12-21 15:46:42,385 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
216
+ 2023-12-21 15:46:45,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
217
+ 2023-12-21 15:46:47,386 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
218
+ 2023-12-21 15:46:50,884 DEBUG SenderThread:833 [sender.py:send():382] send: stats
219
+ 2023-12-21 15:46:51,885 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
220
+ 2023-12-21 15:46:52,387 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
221
+ 2023-12-21 15:46:56,886 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
222
+ 2023-12-21 15:46:57,391 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
223
+ 2023-12-21 15:47:01,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
224
+ 2023-12-21 15:47:02,397 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
225
+ 2023-12-21 15:47:06,888 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
226
+ 2023-12-21 15:47:07,398 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
227
+ 2023-12-21 15:47:11,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
228
+ 2023-12-21 15:47:12,399 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
229
+ 2023-12-21 15:47:16,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
230
+ 2023-12-21 15:47:17,399 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
231
+ 2023-12-21 15:47:20,885 DEBUG SenderThread:833 [sender.py:send():382] send: stats
232
+ 2023-12-21 15:47:22,400 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
233
+ 2023-12-21 15:47:22,886 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
234
+ 2023-12-21 15:47:27,410 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
235
+ 2023-12-21 15:47:27,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
236
+ 2023-12-21 15:47:32,411 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
237
+ 2023-12-21 15:47:32,888 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
238
+ 2023-12-21 15:47:37,412 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
239
+ 2023-12-21 15:47:37,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
240
+ 2023-12-21 15:47:42,413 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
241
+ 2023-12-21 15:47:42,890 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
242
+ 2023-12-21 15:47:47,414 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
243
+ 2023-12-21 15:47:47,891 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
244
+ 2023-12-21 15:47:50,886 DEBUG SenderThread:833 [sender.py:send():382] send: stats
245
+ 2023-12-21 15:47:52,415 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
246
+ 2023-12-21 15:47:53,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
247
+ 2023-12-21 15:47:57,416 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
248
+ 2023-12-21 15:47:58,888 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
249
+ 2023-12-21 15:48:02,418 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
250
+ 2023-12-21 15:48:03,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
251
+ 2023-12-21 15:48:07,420 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
252
+ 2023-12-21 15:48:08,890 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
253
+ 2023-12-21 15:48:12,421 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
254
+ 2023-12-21 15:48:13,891 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
255
+ 2023-12-21 15:48:17,424 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
256
+ 2023-12-21 15:48:18,892 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
257
+ 2023-12-21 15:48:20,195 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: partial_history
258
+ 2023-12-21 15:48:20,196 DEBUG SenderThread:833 [sender.py:send():382] send: history
259
+ 2023-12-21 15:48:20,196 DEBUG SenderThread:833 [sender.py:send_request():409] send_request: summary_record
260
+ 2023-12-21 15:48:20,197 INFO SenderThread:833 [sender.py:_save_file():1392] saving file wandb-summary.json with policy end
261
+ 2023-12-21 15:48:20,887 DEBUG SenderThread:833 [sender.py:send():382] send: stats
262
+ 2023-12-21 15:48:20,930 INFO Thread-12 :833 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20231221_154050-m3289itb/files/wandb-summary.json
263
+ 2023-12-21 15:48:22,477 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
264
+ 2023-12-21 15:48:24,020 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
265
+ 2023-12-21 15:48:25,933 INFO Thread-12 :833 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20231221_154050-m3289itb/files/output.log
266
+ 2023-12-21 15:48:27,478 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
267
+ 2023-12-21 15:48:29,020 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
268
+ 2023-12-21 15:48:32,479 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
269
+ 2023-12-21 15:48:34,021 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20231221_154050-m3289itb/files/output.log CHANGED
@@ -3,4 +3,6 @@ You're using a RobertaTokenizerFast tokenizer. Please note that with a fast toke
3
  warnings.warn('Was asked to gather along dimension 0, but all '
4
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
5
  warnings.warn('Was asked to gather along dimension 0, but all '
 
 
6
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
 
3
  warnings.warn('Was asked to gather along dimension 0, but all '
4
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
5
  warnings.warn('Was asked to gather along dimension 0, but all '
6
+ /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
7
+ warnings.warn('Was asked to gather along dimension 0, but all '
8
  /opt/conda/lib/python3.10/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
wandb/run-20231221_154050-m3289itb/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 2.1894, "train/learning_rate": 8.324576765907765e-06, "train/epoch": 1.75, "train/global_step": 1000, "_timestamp": 1703173561.633721, "_runtime": 311.5181519985199, "_step": 1}
 
1
+ {"train/loss": 2.1452, "train/learning_rate": 2.4868651488616464e-06, "train/epoch": 2.63, "train/global_step": 1500, "_timestamp": 1703173700.194624, "_runtime": 450.0790548324585, "_step": 2}
wandb/run-20231221_154050-m3289itb/logs/debug-internal.log CHANGED
@@ -200,3 +200,70 @@
200
  2023-12-21 15:46:07,368 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
201
  2023-12-21 15:46:10,050 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
202
  2023-12-21 15:46:12,376 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  2023-12-21 15:46:07,368 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
201
  2023-12-21 15:46:10,050 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
202
  2023-12-21 15:46:12,376 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
203
+ 2023-12-21 15:46:15,051 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
204
+ 2023-12-21 15:46:17,379 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
205
+ 2023-12-21 15:46:20,052 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
206
+ 2023-12-21 15:46:20,883 DEBUG SenderThread:833 [sender.py:send():382] send: stats
207
+ 2023-12-21 15:46:22,380 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
208
+ 2023-12-21 15:46:25,884 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
209
+ 2023-12-21 15:46:27,381 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
210
+ 2023-12-21 15:46:30,885 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
211
+ 2023-12-21 15:46:32,382 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
212
+ 2023-12-21 15:46:35,886 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
213
+ 2023-12-21 15:46:37,384 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
214
+ 2023-12-21 15:46:40,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
215
+ 2023-12-21 15:46:42,385 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
216
+ 2023-12-21 15:46:45,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
217
+ 2023-12-21 15:46:47,386 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
218
+ 2023-12-21 15:46:50,884 DEBUG SenderThread:833 [sender.py:send():382] send: stats
219
+ 2023-12-21 15:46:51,885 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
220
+ 2023-12-21 15:46:52,387 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
221
+ 2023-12-21 15:46:56,886 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
222
+ 2023-12-21 15:46:57,391 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
223
+ 2023-12-21 15:47:01,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
224
+ 2023-12-21 15:47:02,397 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
225
+ 2023-12-21 15:47:06,888 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
226
+ 2023-12-21 15:47:07,398 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
227
+ 2023-12-21 15:47:11,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
228
+ 2023-12-21 15:47:12,399 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
229
+ 2023-12-21 15:47:16,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
230
+ 2023-12-21 15:47:17,399 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
231
+ 2023-12-21 15:47:20,885 DEBUG SenderThread:833 [sender.py:send():382] send: stats
232
+ 2023-12-21 15:47:22,400 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
233
+ 2023-12-21 15:47:22,886 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
234
+ 2023-12-21 15:47:27,410 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
235
+ 2023-12-21 15:47:27,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
236
+ 2023-12-21 15:47:32,411 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
237
+ 2023-12-21 15:47:32,888 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
238
+ 2023-12-21 15:47:37,412 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
239
+ 2023-12-21 15:47:37,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
240
+ 2023-12-21 15:47:42,413 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
241
+ 2023-12-21 15:47:42,890 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
242
+ 2023-12-21 15:47:47,414 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
243
+ 2023-12-21 15:47:47,891 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
244
+ 2023-12-21 15:47:50,886 DEBUG SenderThread:833 [sender.py:send():382] send: stats
245
+ 2023-12-21 15:47:52,415 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
246
+ 2023-12-21 15:47:53,887 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
247
+ 2023-12-21 15:47:57,416 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
248
+ 2023-12-21 15:47:58,888 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
249
+ 2023-12-21 15:48:02,418 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
250
+ 2023-12-21 15:48:03,889 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
251
+ 2023-12-21 15:48:07,420 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
252
+ 2023-12-21 15:48:08,890 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
253
+ 2023-12-21 15:48:12,421 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
254
+ 2023-12-21 15:48:13,891 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
255
+ 2023-12-21 15:48:17,424 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
256
+ 2023-12-21 15:48:18,892 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
257
+ 2023-12-21 15:48:20,195 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: partial_history
258
+ 2023-12-21 15:48:20,196 DEBUG SenderThread:833 [sender.py:send():382] send: history
259
+ 2023-12-21 15:48:20,196 DEBUG SenderThread:833 [sender.py:send_request():409] send_request: summary_record
260
+ 2023-12-21 15:48:20,197 INFO SenderThread:833 [sender.py:_save_file():1392] saving file wandb-summary.json with policy end
261
+ 2023-12-21 15:48:20,887 DEBUG SenderThread:833 [sender.py:send():382] send: stats
262
+ 2023-12-21 15:48:20,930 INFO Thread-12 :833 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20231221_154050-m3289itb/files/wandb-summary.json
263
+ 2023-12-21 15:48:22,477 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
264
+ 2023-12-21 15:48:24,020 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
265
+ 2023-12-21 15:48:25,933 INFO Thread-12 :833 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20231221_154050-m3289itb/files/output.log
266
+ 2023-12-21 15:48:27,478 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
267
+ 2023-12-21 15:48:29,020 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
268
+ 2023-12-21 15:48:32,479 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: keepalive
269
+ 2023-12-21 15:48:34,021 DEBUG HandlerThread:833 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20231221_154050-m3289itb/run-m3289itb.wandb CHANGED
Binary files a/wandb/run-20231221_154050-m3289itb/run-m3289itb.wandb and b/wandb/run-20231221_154050-m3289itb/run-m3289itb.wandb differ