*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
train: 27839 of 27839 rows fit in 3072 tokens
val: 300 of 300 rows fit in 3072 tokens

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 130.07it/s]
train: 27839 of 27839 rows fit in 3072 tokens
trainable params: 40,370,176 || all params: 7,655,986,688 || trainable%: 0.5273
val: 300 of 300 rows fit in 3072 tokens

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 128.48it/s]
trainable params: 40,370,176 || all params: 7,655,986,688 || trainable%: 0.5273

  0%|          | 0/870 [00:00<?, ?it/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.

  0%|          | 1/870 [00:12<2:54:06, 12.02s/it]
  0%|          | 2/870 [00:21<2:35:52, 10.78s/it]
  0%|          | 3/870 [00:31<2:25:50, 10.09s/it]
  0%|          | 4/870 [00:40<2:20:18,  9.72s/it]
  1%|          | 5/870 [00:49<2:15:37,  9.41s/it]
  1%|          | 6/870 [00:57<2:10:20,  9.05s/it]
  1%|          | 7/870 [01:05<2:06:14,  8.78s/it]
  1%|          | 8/870 [01:13<2:02:41,  8.54s/it]
  1%|          | 9/870 [01:22<2:02:41,  8.55s/it]
  1%|          | 10/870 [01:29<1:57:18,  8.18s/it]
                                                  
{'loss': 0.2436, 'grad_norm': 0.23789972066879272, 'learning_rate': 6.666666666666667e-05, 'epoch': 0.01}

  1%|          | 10/870 [01:29<1:57:18,  8.18s/it]
  1%|▏         | 11/870 [01:37<1:54:25,  7.99s/it]
  1%|▏         | 12/870 [01:44<1:51:11,  7.78s/it]
  1%|▏         | 13/870 [01:51<1:47:37,  7.53s/it]
  2%|▏         | 14/870 [01:58<1:46:39,  7.48s/it]
  2%|▏         | 15/870 [02:05<1:43:51,  7.29s/it]
  2%|▏         | 16/870 [02:12<1:40:38,  7.07s/it]
  2%|▏         | 17/870 [02:18<1:36:44,  6.80s/it]
  2%|▏         | 18/870 [02:24<1:34:07,  6.63s/it]
  2%|▏         | 19/870 [02:31<1:32:42,  6.54s/it]
  2%|▏         | 20/870 [02:36<1:29:24,  6.31s/it]
                                                  
{'loss': 0.0853, 'grad_norm': 0.12851466238498688, 'learning_rate': 0.00014074074074074076, 'epoch': 0.02}

  2%|▏         | 20/870 [02:36<1:29:24,  6.31s/it]
  2%|▏         | 21/870 [02:42<1:26:33,  6.12s/it]
  3%|▎         | 22/870 [02:47<1:23:38,  5.92s/it]
  3%|▎         | 23/870 [02:53<1:20:05,  5.67s/it]
  3%|▎         | 24/870 [02:57<1:15:37,  5.36s/it]
  3%|▎         | 25/870 [03:00<1:05:59,  4.69s/it]
  3%|▎         | 26/870 [03:12<1:35:34,  6.79s/it]
  3%|▎         | 27/870 [03:22<1:47:13,  7.63s/it]
  3%|▎         | 28/870 [03:31<1:55:34,  8.24s/it]
  3%|▎         | 29/870 [03:40<1:59:03,  8.49s/it]
  3%|▎         | 30/870 [03:49<1:59:05,  8.51s/it]
                                                  
{'loss': 0.0627, 'grad_norm': 0.16485165059566498, 'learning_rate': 0.0001999972223815946, 'epoch': 0.03}

  3%|▎         | 30/870 [03:49<1:59:05,  8.51s/it]
  4%|▎         | 31/870 [03:57<1:58:25,  8.47s/it]
  4%|▎         | 32/870 [04:05<1:56:34,  8.35s/it]
  4%|▍         | 33/870 [04:13<1:55:11,  8.26s/it]
  4%|▍         | 34/870 [04:21<1:53:30,  8.15s/it]
  4%|▍         | 35/870 [04:29<1:50:43,  7.96s/it]
  4%|▍         | 36/870 [04:36<1:49:00,  7.84s/it]
  4%|▍         | 37/870 [04:44<1:46:33,  7.68s/it]
  4%|▍         | 38/870 [04:51<1:45:18,  7.59s/it]
  4%|▍         | 39/870 [04:58<1:42:40,  7.41s/it]
  5%|▍         | 40/870 [05:05<1:39:36,  7.20s/it]
                                                  
{'loss': 0.048, 'grad_norm': 0.08059470355510712, 'learning_rate': 0.00019990002193828922, 'epoch': 0.05}

  5%|▍         | 40/870 [05:05<1:39:36,  7.20s/it]
  5%|▍         | 41/870 [05:11<1:36:34,  6.99s/it]
  5%|▍         | 42/870 [05:17<1:33:16,  6.76s/it]
  5%|▍         | 43/870 [05:24<1:32:19,  6.70s/it]
  5%|▌         | 44/870 [05:30<1:29:42,  6.52s/it]
  5%|▌         | 45/870 [05:36<1:27:24,  6.36s/it]
  5%|▌         | 46/870 [05:42<1:24:36,  6.16s/it]
  5%|▌         | 47/870 [05:47<1:22:03,  5.98s/it]
  6%|▌         | 48/870 [05:53<1:20:04,  5.85s/it]
  6%|▌         | 49/870 [05:57<1:14:49,  5.47s/it]
  6%|▌         | 50/870 [06:01<1:05:50,  4.82s/it]
                                                  
{'loss': 0.0421, 'grad_norm': 0.10289835184812546, 'learning_rate': 0.0001996640948394666, 'epoch': 0.06}

  6%|▌         | 50/870 [06:01<1:05:50,  4.82s/it]
  6%|▌         | 51/870 [06:12<1:33:52,  6.88s/it]
  6%|▌         | 52/870 [06:23<1:47:31,  7.89s/it]
  6%|▌         | 53/870 [06:32<1:55:00,  8.45s/it]
  6%|▌         | 54/870 [06:42<1:58:20,  8.70s/it]
  6%|▋         | 55/870 [06:51<1:58:49,  8.75s/it]
  6%|▋         | 56/870 [06:59<1:57:23,  8.65s/it]
  7%|▋         | 57/870 [07:08<1:56:38,  8.61s/it]
  7%|▋         | 58/870 [07:15<1:53:18,  8.37s/it]
  7%|▋         | 59/870 [07:23<1:51:14,  8.23s/it]
  7%|▋         | 60/870 [07:31<1:48:41,  8.05s/it]
                                                  
{'loss': 0.05, 'grad_norm': 0.10482244938611984, 'learning_rate': 0.00019928976870644988, 'epoch': 0.07}

  7%|▋         | 60/870 [07:31<1:48:41,  8.05s/it]
  7%|▋         | 61/870 [07:38<1:45:49,  7.85s/it]
  7%|▋         | 62/870 [07:46<1:44:05,  7.73s/it]
  7%|▋         | 63/870 [07:53<1:41:06,  7.52s/it]
  7%|▋         | 64/870 [08:00<1:38:54,  7.36s/it]
  7%|▋         | 65/870 [08:07<1:36:39,  7.20s/it]
  8%|▊         | 66/870 [08:13<1:35:02,  7.09s/it]
  8%|▊         | 67/870 [08:20<1:32:27,  6.91s/it]
  8%|▊         | 68/870 [08:26<1:29:09,  6.67s/it]
  8%|▊         | 69/870 [08:32<1:26:27,  6.48s/it]
  8%|▊         | 70/870 [08:38<1:24:14,  6.32s/it]
                                                  
{'loss': 0.0377, 'grad_norm': 0.10887732356786728, 'learning_rate': 0.00019877756334905984, 'epoch': 0.08}

  8%|▊         | 70/870 [08:38<1:24:14,  6.32s/it]
  8%|▊         | 71/870 [08:44<1:21:37,  6.13s/it]
  8%|▊         | 72/870 [08:49<1:20:08,  6.03s/it]
  8%|▊         | 73/870 [08:55<1:16:25,  5.75s/it]
  9%|▊         | 74/870 [08:59<1:12:26,  5.46s/it]
  9%|▊         | 75/870 [09:03<1:03:50,  4.82s/it]
  9%|▊         | 76/870 [09:15<1:32:44,  7.01s/it]
  9%|▉         | 77/870 [09:25<1:47:17,  8.12s/it]
  9%|▉         | 78/870 [09:35<1:52:17,  8.51s/it]
  9%|▉         | 79/870 [09:44<1:55:33,  8.76s/it]
  9%|▉         | 80/870 [09:53<1:57:10,  8.90s/it]
                                                  
{'loss': 0.0427, 'grad_norm': 0.10280067473649979, 'learning_rate': 0.00019812819004377862, 'epoch': 0.09}

  9%|▉         | 80/870 [09:53<1:57:10,  8.90s/it]
  9%|▉         | 81/870 [10:02<1:55:32,  8.79s/it]
  9%|▉         | 82/870 [10:11<1:54:32,  8.72s/it]
 10%|▉         | 83/870 [10:19<1:52:27,  8.57s/it]
 10%|▉         | 84/870 [10:27<1:49:44,  8.38s/it]
 10%|▉         | 85/870 [10:35<1:47:23,  8.21s/it]
 10%|▉         | 86/870 [10:42<1:45:50,  8.10s/it]
 10%|█         | 87/870 [10:50<1:42:51,  7.88s/it]
 10%|█         | 88/870 [10:57<1:39:21,  7.62s/it]
 10%|█         | 89/870 [11:04<1:36:59,  7.45s/it]
 10%|█         | 90/870 [11:11<1:34:03,  7.23s/it]
                                                  
{'loss': 0.0375, 'grad_norm': 0.1345912218093872, 'learning_rate': 0.0001973425505460321, 'epoch': 0.1}

 10%|█         | 90/870 [11:11<1:34:03,  7.23s/it]
 10%|█         | 91/870 [11:18<1:33:21,  7.19s/it]
 11%|█         | 92/870 [11:24<1:29:23,  6.89s/it]
 11%|█         | 93/870 [11:30<1:26:27,  6.68s/it]
 11%|█         | 94/870 [11:36<1:24:23,  6.52s/it]
 11%|█         | 95/870 [11:42<1:21:49,  6.34s/it]
 11%|█         | 96/870 [11:48<1:20:51,  6.27s/it]
 11%|█         | 97/870 [11:54<1:17:53,  6.05s/it]
 11%|█▏        | 98/870 [11:59<1:14:14,  5.77s/it]
 11%|█▏        | 99/870 [12:03<1:08:35,  5.34s/it]
 11%|█▏        | 100/870 [12:07<1:00:45,  4.73s/it]
                                                   
{'loss': 0.0357, 'grad_norm': 0.3418385982513428, 'learning_rate': 0.00019642173583796267, 'epoch': 0.11}

 11%|█▏        | 100/870 [12:07<1:00:45,  4.73s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:13,  2.71it/s]

  8%|▊         | 3/38 [00:01<00:15,  2.22it/s]

 11%|█         | 4/38 [00:01<00:16,  2.09it/s]

 13%|█▎        | 5/38 [00:02<00:16,  2.01it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.00it/s]

 18%|█▊        | 7/38 [00:03<00:15,  2.05it/s]

 21%|██        | 8/38 [00:03<00:13,  2.17it/s]

 24%|██▎       | 9/38 [00:04<00:12,  2.34it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.05it/s]

 29%|██▉       | 11/38 [00:05<00:14,  1.82it/s]

 32%|███▏      | 12/38 [00:05<00:14,  1.80it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.87it/s]

 37%|███▋      | 14/38 [00:07<00:15,  1.54it/s]

 39%|███▉      | 15/38 [00:07<00:13,  1.68it/s]

 42%|████▏     | 16/38 [00:08<00:11,  1.85it/s]

 45%|████▍     | 17/38 [00:08<00:10,  2.05it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.37it/s]

 50%|█████     | 19/38 [00:09<00:08,  2.22it/s]

 53%|█████▎    | 20/38 [00:10<00:08,  2.01it/s]

 55%|█████▌    | 21/38 [00:10<00:08,  1.97it/s]

 58%|█████▊    | 22/38 [00:11<00:08,  1.93it/s]

 61%|██████    | 23/38 [00:11<00:07,  1.95it/s]

 63%|██████▎   | 24/38 [00:12<00:06,  2.02it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  2.12it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.22it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.36it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.05it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.83it/s]

 79%|███████▉  | 30/38 [00:15<00:04,  1.79it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.81it/s]

 84%|████████▍ | 32/38 [00:16<00:03,  1.86it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.90it/s]

 89%|████████▉ | 34/38 [00:17<00:02,  1.97it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.07it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.21it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.05it/s]

100%|██████████| 38/38 [00:19<00:00,  1.98it/s]
                                                   

                                               
{'eval_loss': 0.052477214485406876, 'eval_runtime': 20.6792, 'eval_samples_per_second': 14.507, 'eval_steps_per_second': 1.838, 'epoch': 0.11}

 11%|█▏        | 100/870 [12:27<1:00:45,  4.73s/it]

100%|██████████| 38/38 [00:19<00:00,  1.98it/s]

                                               
 12%|█▏        | 101/870 [12:40<2:49:23, 13.22s/it]
 12%|█▏        | 102/870 [12:50<2:37:13, 12.28s/it]
 12%|█▏        | 103/870 [12:59<2:26:46, 11.48s/it]
 12%|█▏        | 104/870 [13:09<2:19:54, 10.96s/it]
 12%|█▏        | 105/870 [13:18<2:12:26, 10.39s/it]
 12%|█▏        | 106/870 [13:27<2:05:02,  9.82s/it]
 12%|█▏        | 107/870 [13:35<1:59:00,  9.36s/it]
 12%|█▏        | 108/870 [13:43<1:54:14,  9.00s/it]
 13%|█▎        | 109/870 [13:51<1:50:00,  8.67s/it]
 13%|█▎        | 110/870 [13:59<1:47:06,  8.46s/it]
                                                   
{'loss': 0.047, 'grad_norm': 0.07799140363931656, 'learning_rate': 0.00019536702461343168, 'epoch': 0.13}

 13%|█▎        | 110/870 [13:59<1:47:06,  8.46s/it]
 13%|█▎        | 111/870 [14:06<1:43:48,  8.21s/it]
 13%|█▎        | 112/870 [14:14<1:40:26,  7.95s/it]
 13%|█▎        | 113/870 [14:21<1:37:19,  7.71s/it]
 13%|█▎        | 114/870 [14:28<1:34:16,  7.48s/it]
 13%|█▎        | 115/870 [14:35<1:32:24,  7.34s/it]
 13%|█▎        | 116/870 [14:42<1:29:43,  7.14s/it]
 13%|█▎        | 117/870 [14:48<1:26:33,  6.90s/it]
 14%|█▎        | 118/870 [14:54<1:23:41,  6.68s/it]
 14%|█▎        | 119/870 [15:01<1:22:48,  6.62s/it]
 14%|█▍        | 120/870 [15:06<1:19:57,  6.40s/it]
                                                   
{'loss': 0.0327, 'grad_norm': 0.1051594689488411, 'learning_rate': 0.00019417988150235494, 'epoch': 0.14}

 14%|█▍        | 120/870 [15:06<1:19:57,  6.40s/it]
 14%|█▍        | 121/870 [15:12<1:17:24,  6.20s/it]
 14%|█▍        | 122/870 [15:18<1:14:53,  6.01s/it]
 14%|█▍        | 123/870 [15:23<1:11:52,  5.77s/it]
 14%|█▍        | 124/870 [15:28<1:08:24,  5.50s/it]
 14%|█▍        | 125/870 [15:31<59:10,  4.77s/it]  
 14%|█▍        | 126/870 [15:43<1:24:39,  6.83s/it]
 15%|█▍        | 127/870 [15:53<1:36:16,  7.77s/it]
 15%|█▍        | 128/870 [16:02<1:42:15,  8.27s/it]
 15%|█▍        | 129/870 [16:11<1:46:32,  8.63s/it]
 15%|█▍        | 130/870 [16:20<1:46:36,  8.64s/it]
                                                   
{'loss': 0.0418, 'grad_norm': 0.08026126027107239, 'learning_rate': 0.00019286195503683708, 'epoch': 0.15}

 15%|█▍        | 130/870 [16:20<1:46:36,  8.64s/it]
 15%|█▌        | 131/870 [16:28<1:45:15,  8.55s/it]
 15%|█▌        | 132/870 [16:37<1:43:36,  8.42s/it]
 15%|█▌        | 133/870 [16:44<1:39:39,  8.11s/it]
 15%|█▌        | 134/870 [16:52<1:39:04,  8.08s/it]
 16%|█▌        | 135/870 [17:00<1:37:09,  7.93s/it]
 16%|█▌        | 136/870 [17:07<1:35:12,  7.78s/it]
 16%|█▌        | 137/870 [17:14<1:31:46,  7.51s/it]
 16%|█▌        | 138/870 [17:21<1:29:34,  7.34s/it]
 16%|█▌        | 139/870 [17:28<1:28:47,  7.29s/it]
 16%|█▌        | 140/870 [17:35<1:26:31,  7.11s/it]
                                                   
{'loss': 0.037, 'grad_norm': 0.07206742465496063, 'learning_rate': 0.0001914150753619294, 'epoch': 0.16}

 16%|█▌        | 140/870 [17:35<1:26:31,  7.11s/it]
 16%|█▌        | 141/870 [17:41<1:24:18,  6.94s/it]
 16%|█▋        | 142/870 [17:47<1:21:43,  6.74s/it]
 16%|█▋        | 143/870 [17:54<1:19:23,  6.55s/it]
 17%|█▋        | 144/870 [18:00<1:18:18,  6.47s/it]
 17%|█▋        | 145/870 [18:06<1:16:04,  6.30s/it]
 17%|█▋        | 146/870 [18:11<1:13:41,  6.11s/it]
 17%|█▋        | 147/870 [18:17<1:11:05,  5.90s/it]
 17%|█▋        | 148/870 [18:22<1:07:37,  5.62s/it]
 17%|█▋        | 149/870 [18:26<1:03:45,  5.31s/it]
 17%|█▋        | 150/870 [18:29<55:08,  4.60s/it]  
                                                 
{'loss': 0.033, 'grad_norm': 0.1900654435157776, 'learning_rate': 0.00018984125169419007, 'epoch': 0.17}

 17%|█▋        | 150/870 [18:29<55:08,  4.60s/it]
 17%|█▋        | 151/870 [18:41<1:21:17,  6.78s/it]
 17%|█▋        | 152/870 [18:51<1:33:13,  7.79s/it]
 18%|█▊        | 153/870 [19:01<1:40:06,  8.38s/it]
 18%|█▊        | 154/870 [19:10<1:42:27,  8.59s/it]
 18%|█▊        | 155/870 [19:19<1:41:57,  8.56s/it]
 18%|█▊        | 156/870 [19:27<1:40:58,  8.49s/it]
 18%|█▊        | 157/870 [19:35<1:39:13,  8.35s/it]
 18%|█▊        | 158/870 [19:43<1:38:27,  8.30s/it]
 18%|█▊        | 159/870 [19:51<1:36:54,  8.18s/it]
 18%|█▊        | 160/870 [19:59<1:34:58,  8.03s/it]
                                                   
{'loss': 0.0348, 'grad_norm': 0.11883541941642761, 'learning_rate': 0.00018814266953157556, 'epoch': 0.18}

 18%|█▊        | 160/870 [19:59<1:34:58,  8.03s/it]
 19%|█▊        | 161/870 [20:06<1:32:47,  7.85s/it]
 19%|█▊        | 162/870 [20:13<1:30:13,  7.65s/it]
 19%|█▊        | 163/870 [20:21<1:28:32,  7.51s/it]
 19%|█▉        | 164/870 [20:27<1:26:09,  7.32s/it]
 19%|█▉        | 165/870 [20:34<1:23:41,  7.12s/it]
 19%|█▉        | 166/870 [20:40<1:20:47,  6.88s/it]
 19%|█▉        | 167/870 [20:47<1:18:31,  6.70s/it]
 19%|█▉        | 168/870 [20:53<1:17:24,  6.62s/it]
 19%|█▉        | 169/870 [20:59<1:15:11,  6.44s/it]
 20%|█▉        | 170/870 [21:05<1:12:58,  6.25s/it]
                                                   
{'loss': 0.0296, 'grad_norm': 0.11700455099344254, 'learning_rate': 0.00018632168761853826, 'epoch': 0.2}

 20%|█▉        | 170/870 [21:05<1:12:58,  6.25s/it]
 20%|█▉        | 171/870 [21:11<1:10:43,  6.07s/it]
 20%|█▉        | 172/870 [21:16<1:09:16,  5.95s/it]
 20%|█▉        | 173/870 [21:21<1:06:06,  5.69s/it]
 20%|██        | 174/870 [21:26<1:01:11,  5.27s/it]
 20%|██        | 175/870 [21:29<53:16,  4.60s/it]  
 20%|██        | 176/870 [21:40<1:18:00,  6.74s/it]
 20%|██        | 177/870 [21:51<1:29:45,  7.77s/it]
 20%|██        | 178/870 [22:00<1:35:25,  8.27s/it]
 21%|██        | 179/870 [22:09<1:38:36,  8.56s/it]
 21%|██        | 180/870 [22:18<1:38:21,  8.55s/it]
                                                   
{'loss': 0.0305, 'grad_norm': 0.15726420283317566, 'learning_rate': 0.00018438083467054443, 'epoch': 0.21}

 21%|██        | 180/870 [22:18<1:38:21,  8.55s/it]
 21%|██        | 181/870 [22:26<1:37:35,  8.50s/it]
 21%|██        | 182/870 [22:35<1:37:03,  8.46s/it]
 21%|██        | 183/870 [22:42<1:34:58,  8.29s/it]
 21%|██        | 184/870 [22:50<1:33:56,  8.22s/it]
 21%|██▏       | 185/870 [22:58<1:31:03,  7.98s/it]
 21%|██▏       | 186/870 [23:06<1:29:59,  7.89s/it]
 21%|██▏       | 187/870 [23:13<1:27:01,  7.65s/it]
 22%|██▏       | 188/870 [23:20<1:24:43,  7.45s/it]
 22%|██▏       | 189/870 [23:27<1:23:00,  7.31s/it]
 22%|██▏       | 190/870 [23:33<1:20:43,  7.12s/it]
                                                   
{'loss': 0.0319, 'grad_norm': 0.18836188316345215, 'learning_rate': 0.00018232280586256102, 'epoch': 0.22}

 22%|██▏       | 190/870 [23:33<1:20:43,  7.12s/it]
 22%|██▏       | 191/870 [23:40<1:19:15,  7.00s/it]
 22%|██▏       | 192/870 [23:46<1:16:44,  6.79s/it]
 22%|██▏       | 193/870 [23:52<1:14:11,  6.57s/it]
 22%|██▏       | 194/870 [23:58<1:12:08,  6.40s/it]
 22%|██▏       | 195/870 [24:04<1:09:33,  6.18s/it]
 23%|██▎       | 196/870 [24:10<1:08:43,  6.12s/it]
 23%|██▎       | 197/870 [24:15<1:05:52,  5.87s/it]
 23%|██▎       | 198/870 [24:20<1:02:47,  5.61s/it]
 23%|██▎       | 199/870 [24:25<58:23,  5.22s/it]  
 23%|██▎       | 200/870 [24:28<50:26,  4.52s/it]
                                                 
{'loss': 0.0259, 'grad_norm': 0.08308509737253189, 'learning_rate': 0.0001801504590863877, 'epoch': 0.23}

 23%|██▎       | 200/870 [24:28<50:26,  4.52s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:12,  2.80it/s]

  8%|▊         | 3/38 [00:01<00:15,  2.25it/s]

 11%|█         | 4/38 [00:01<00:16,  2.09it/s]

 13%|█▎        | 5/38 [00:02<00:15,  2.10it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.11it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.12it/s]

 21%|██        | 8/38 [00:03<00:13,  2.23it/s]

 24%|██▎       | 9/38 [00:03<00:12,  2.39it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.12it/s]

 29%|██▉       | 11/38 [00:05<00:14,  1.90it/s]

 32%|███▏      | 12/38 [00:05<00:13,  1.88it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.90it/s]

 37%|███▋      | 14/38 [00:06<00:12,  1.91it/s]

 39%|███▉      | 15/38 [00:07<00:11,  1.99it/s]

 42%|████▏     | 16/38 [00:07<00:10,  2.11it/s]

 45%|████▍     | 17/38 [00:08<00:09,  2.27it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.44it/s]

 50%|█████     | 19/38 [00:08<00:08,  2.12it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.90it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.89it/s]

 58%|█████▊    | 22/38 [00:10<00:08,  1.84it/s]

 61%|██████    | 23/38 [00:11<00:08,  1.87it/s]

 63%|██████▎   | 24/38 [00:11<00:07,  1.90it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  1.98it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.08it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.25it/s]

 74%|███████▎  | 28/38 [00:13<00:05,  1.99it/s]

 76%|███████▋  | 29/38 [00:14<00:05,  1.78it/s]

 79%|███████▉  | 30/38 [00:14<00:04,  1.77it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.77it/s]

 84%|████████▍ | 32/38 [00:15<00:03,  1.82it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.91it/s]

 89%|████████▉ | 34/38 [00:16<00:01,  2.05it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.16it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.38it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.09it/s]

100%|██████████| 38/38 [00:18<00:00,  1.94it/s]
                                                 

                                               
{'eval_loss': 0.02992151863873005, 'eval_runtime': 20.3901, 'eval_samples_per_second': 14.713, 'eval_steps_per_second': 1.864, 'epoch': 0.23}

 23%|██▎       | 200/870 [24:48<50:26,  4.52s/it]

100%|██████████| 38/38 [00:19<00:00,  1.94it/s]

                                               
 23%|██▎       | 201/870 [25:02<2:28:54, 13.36s/it]
 23%|██▎       | 202/870 [25:12<2:18:46, 12.47s/it]
 23%|██▎       | 203/870 [25:21<2:08:46, 11.58s/it]
 23%|██▎       | 204/870 [25:31<2:00:47, 10.88s/it]
 24%|██▎       | 205/870 [25:40<1:54:07, 10.30s/it]
 24%|██▎       | 206/870 [25:48<1:47:43,  9.73s/it]
 24%|██▍       | 207/870 [25:56<1:41:57,  9.23s/it]
 24%|██▍       | 208/870 [26:04<1:37:56,  8.88s/it]
 24%|██▍       | 209/870 [26:12<1:34:34,  8.58s/it]
 24%|██▍       | 210/870 [26:20<1:31:57,  8.36s/it]
                                                   
{'loss': 0.0313, 'grad_norm': 0.05260265991091728, 'learning_rate': 0.00017786681098203158, 'epoch': 0.24}

 24%|██▍       | 210/870 [26:20<1:31:57,  8.36s/it]
 24%|██▍       | 211/870 [26:27<1:28:32,  8.06s/it]
 24%|██▍       | 212/870 [26:34<1:25:01,  7.75s/it]
 24%|██▍       | 213/870 [26:41<1:22:38,  7.55s/it]
 25%|██▍       | 214/870 [26:48<1:19:51,  7.30s/it]
 25%|██▍       | 215/870 [26:55<1:19:00,  7.24s/it]
 25%|██▍       | 216/870 [27:01<1:15:53,  6.96s/it]
 25%|██▍       | 217/870 [27:08<1:13:05,  6.72s/it]
 25%|██▌       | 218/870 [27:14<1:11:04,  6.54s/it]
 25%|██▌       | 219/870 [27:20<1:08:29,  6.31s/it]
 25%|██▌       | 220/870 [27:26<1:07:56,  6.27s/it]
                                                   
{'loss': 0.0229, 'grad_norm': 0.0633602887392044, 'learning_rate': 0.00017547503274863496, 'epoch': 0.25}

 25%|██▌       | 220/870 [27:26<1:07:56,  6.27s/it]
 25%|██▌       | 221/870 [27:31<1:05:34,  6.06s/it]
 26%|██▌       | 222/870 [27:37<1:03:39,  5.89s/it]
 26%|██▌       | 223/870 [27:42<1:01:25,  5.70s/it]
 26%|██▌       | 224/870 [27:47<58:18,  5.42s/it]  
 26%|██▌       | 225/870 [27:50<52:15,  4.86s/it]
 26%|██▌       | 226/870 [28:02<1:13:42,  6.87s/it]
 26%|██▌       | 227/870 [28:12<1:24:12,  7.86s/it]
 26%|██▌       | 228/870 [28:22<1:29:24,  8.36s/it]
 26%|██▋       | 229/870 [28:31<1:31:35,  8.57s/it]
 26%|██▋       | 230/870 [28:39<1:31:50,  8.61s/it]
                                                   
{'loss': 0.028, 'grad_norm': 0.11880059540271759, 'learning_rate': 0.00017297844574077446, 'epoch': 0.26}

 26%|██▋       | 230/870 [28:39<1:31:50,  8.61s/it]
 27%|██▋       | 231/870 [28:47<1:29:59,  8.45s/it]
 27%|██▋       | 232/870 [28:56<1:28:52,  8.36s/it]
 27%|██▋       | 233/870 [29:03<1:26:45,  8.17s/it]
 27%|██▋       | 234/870 [29:11<1:25:40,  8.08s/it]
 27%|██▋       | 235/870 [29:19<1:23:59,  7.94s/it]
 27%|██▋       | 236/870 [29:26<1:22:17,  7.79s/it]
 27%|██▋       | 237/870 [29:33<1:20:06,  7.59s/it]
 27%|██▋       | 238/870 [29:40<1:17:46,  7.38s/it]
 27%|██▋       | 239/870 [29:47<1:16:25,  7.27s/it]
 28%|██▊       | 240/870 [29:54<1:14:52,  7.13s/it]
                                                   
{'loss': 0.0264, 'grad_norm': 0.05497035011649132, 'learning_rate': 0.0001703805168562453, 'epoch': 0.28}

 28%|██▊       | 240/870 [29:54<1:14:52,  7.13s/it]
 28%|██▊       | 241/870 [30:01<1:12:49,  6.95s/it]
 28%|██▊       | 242/870 [30:07<1:10:42,  6.76s/it]
 28%|██▊       | 243/870 [30:13<1:08:38,  6.57s/it]
 28%|██▊       | 244/870 [30:19<1:07:52,  6.51s/it]
 28%|██▊       | 245/870 [30:25<1:05:45,  6.31s/it]
 28%|██▊       | 246/870 [30:31<1:03:40,  6.12s/it]
 28%|██▊       | 247/870 [30:36<1:01:43,  5.95s/it]
 29%|██▊       | 248/870 [30:42<1:00:15,  5.81s/it]
 29%|██▊       | 249/870 [30:47<56:18,  5.44s/it]  
 29%|██▊       | 250/870 [30:50<49:19,  4.77s/it]
                                                 
{'loss': 0.0213, 'grad_norm': 0.07999792695045471, 'learning_rate': 0.00016768485372173697, 'epoch': 0.29}

 29%|██▊       | 250/870 [30:50<49:19,  4.77s/it]
 29%|██▉       | 251/870 [31:02<1:10:55,  6.87s/it]
 29%|██▉       | 252/870 [31:11<1:20:12,  7.79s/it]
 29%|██▉       | 253/870 [31:21<1:25:52,  8.35s/it]
 29%|██▉       | 254/870 [31:30<1:28:35,  8.63s/it]
 29%|██▉       | 255/870 [31:39<1:28:40,  8.65s/it]
 29%|██▉       | 256/870 [31:48<1:27:49,  8.58s/it]
 30%|██▉       | 257/870 [31:56<1:26:38,  8.48s/it]
 30%|██▉       | 258/870 [32:04<1:25:45,  8.41s/it]
 30%|██▉       | 259/870 [32:12<1:24:03,  8.25s/it]
 30%|██▉       | 260/870 [32:20<1:22:20,  8.10s/it]
                                                   
{'loss': 0.0284, 'grad_norm': 0.0921878069639206, 'learning_rate': 0.00016489519968308457, 'epoch': 0.3}

 30%|██▉       | 260/870 [32:20<1:22:20,  8.10s/it]
 30%|███       | 261/870 [32:27<1:20:35,  7.94s/it]
 30%|███       | 262/870 [32:34<1:18:24,  7.74s/it]
 30%|███       | 263/870 [32:42<1:17:16,  7.64s/it]
 30%|███       | 264/870 [32:49<1:15:02,  7.43s/it]
 30%|███       | 265/870 [32:56<1:13:07,  7.25s/it]
 31%|███       | 266/870 [33:02<1:11:12,  7.07s/it]
 31%|███       | 267/870 [33:09<1:09:14,  6.89s/it]
 31%|███       | 268/870 [33:15<1:06:39,  6.64s/it]
 31%|███       | 269/870 [33:21<1:04:38,  6.45s/it]
 31%|███       | 270/870 [33:27<1:02:39,  6.27s/it]
                                                   
{'loss': 0.0216, 'grad_norm': 0.057495057582855225, 'learning_rate': 0.00016201542860705336, 'epoch': 0.31}

 31%|███       | 270/870 [33:27<1:02:39,  6.27s/it]
 31%|███       | 271/870 [33:32<1:00:25,  6.05s/it]
 31%|███▏      | 272/870 [33:38<59:24,  5.96s/it]  
 31%|███▏      | 273/870 [33:43<56:27,  5.67s/it]
 31%|███▏      | 274/870 [33:47<52:26,  5.28s/it]
 32%|███▏      | 275/870 [33:50<46:02,  4.64s/it]
 32%|███▏      | 276/870 [34:02<1:06:43,  6.74s/it]
 32%|███▏      | 277/870 [34:12<1:17:13,  7.81s/it]
 32%|███▏      | 278/870 [34:22<1:22:07,  8.32s/it]
 32%|███▏      | 279/870 [34:31<1:24:50,  8.61s/it]
 32%|███▏      | 280/870 [34:40<1:25:32,  8.70s/it]
                                                   
{'loss': 0.0235, 'grad_norm': 0.0719340518116951, 'learning_rate': 0.00015904953950187457, 'epoch': 0.32}

 32%|███▏      | 280/870 [34:40<1:25:32,  8.70s/it]
 32%|███▏      | 281/870 [34:49<1:24:48,  8.64s/it]
 32%|███▏      | 282/870 [34:57<1:24:20,  8.61s/it]
 33%|███▎      | 283/870 [35:05<1:23:02,  8.49s/it]
 33%|███▎      | 284/870 [35:13<1:21:13,  8.32s/it]
 33%|███▎      | 285/870 [35:21<1:19:05,  8.11s/it]
 33%|███▎      | 286/870 [35:29<1:17:25,  7.95s/it]
 33%|███▎      | 287/870 [35:36<1:16:30,  7.87s/it]
 33%|███▎      | 288/870 [35:43<1:14:17,  7.66s/it]
 33%|███▎      | 289/870 [35:50<1:12:23,  7.48s/it]
 33%|███▎      | 290/870 [35:57<1:10:05,  7.25s/it]
                                                   
{'loss': 0.0232, 'grad_norm': 0.11424289643764496, 'learning_rate': 0.00015600165096400307, 'epoch': 0.33}

 33%|███▎      | 290/870 [35:57<1:10:05,  7.25s/it]
 33%|███▎      | 291/870 [36:04<1:09:33,  7.21s/it]
 34%|███▎      | 292/870 [36:11<1:07:12,  6.98s/it]
 34%|███▎      | 293/870 [36:17<1:05:04,  6.77s/it]
 34%|███▍      | 294/870 [36:23<1:02:47,  6.54s/it]
 34%|███▍      | 295/870 [36:29<1:01:02,  6.37s/it]
 34%|███▍      | 296/870 [36:35<59:47,  6.25s/it]  
 34%|███▍      | 297/870 [36:40<57:44,  6.05s/it]
 34%|███▍      | 298/870 [36:46<55:17,  5.80s/it]
 34%|███▍      | 299/870 [36:50<51:47,  5.44s/it]
 34%|███▍      | 300/870 [36:54<45:18,  4.77s/it]
                                                 
{'loss': 0.0185, 'grad_norm': 0.11561417579650879, 'learning_rate': 0.0001528759954588082, 'epoch': 0.34}

 34%|███▍      | 300/870 [36:54<45:18,  4.77s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:13,  2.64it/s]

  8%|▊         | 3/38 [00:01<00:16,  2.17it/s]

 11%|█         | 4/38 [00:01<00:16,  2.10it/s]

 13%|█▎        | 5/38 [00:02<00:16,  2.06it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.11it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.19it/s]

 21%|██        | 8/38 [00:03<00:13,  2.28it/s]

 24%|██▎       | 9/38 [00:03<00:11,  2.43it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.15it/s]

 29%|██▉       | 11/38 [00:05<00:13,  1.93it/s]

 32%|███▏      | 12/38 [00:05<00:13,  1.91it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.90it/s]

 37%|███▋      | 14/38 [00:06<00:12,  1.91it/s]

 39%|███▉      | 15/38 [00:07<00:11,  1.94it/s]

 42%|████▏     | 16/38 [00:07<00:10,  2.01it/s]

 45%|████▍     | 17/38 [00:08<00:10,  2.09it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.27it/s]

 50%|█████     | 19/38 [00:09<00:09,  2.00it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.82it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.82it/s]

 58%|█████▊    | 22/38 [00:10<00:08,  1.79it/s]

 61%|██████    | 23/38 [00:11<00:08,  1.83it/s]

 63%|██████▎   | 24/38 [00:11<00:07,  1.93it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  2.02it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.15it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.36it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.08it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.86it/s]

 79%|███████▉  | 30/38 [00:14<00:04,  1.84it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.83it/s]

 84%|████████▍ | 32/38 [00:15<00:03,  1.89it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.95it/s]

 89%|████████▉ | 34/38 [00:16<00:01,  2.03it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.15it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.30it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.08it/s]

100%|██████████| 38/38 [00:18<00:00,  2.00it/s]
                                                 

                                               
{'eval_loss': 0.034754782915115356, 'eval_runtime': 20.413, 'eval_samples_per_second': 14.697, 'eval_steps_per_second': 1.862, 'epoch': 0.34}

 34%|███▍      | 300/870 [37:14<45:18,  4.77s/it]

100%|██████████| 38/38 [00:19<00:00,  2.00it/s]

                                               
 35%|███▍      | 301/870 [37:26<2:05:18, 13.21s/it]
 35%|███▍      | 302/870 [37:37<1:57:23, 12.40s/it]
 35%|███▍      | 303/870 [37:47<1:49:20, 11.57s/it]
 35%|███▍      | 304/870 [37:56<1:42:43, 10.89s/it]
 35%|███▌      | 305/870 [38:05<1:36:21, 10.23s/it]
 35%|███▌      | 306/870 [38:13<1:31:37,  9.75s/it]
 35%|███▌      | 307/870 [38:21<1:27:15,  9.30s/it]
 35%|███▌      | 308/870 [38:29<1:22:56,  8.86s/it]
 36%|███▌      | 309/870 [38:37<1:20:07,  8.57s/it]
 36%|███▌      | 310/870 [38:45<1:17:43,  8.33s/it]
                                                   
{'loss': 0.029, 'grad_norm': 0.08098062872886658, 'learning_rate': 0.00014967691344313996, 'epoch': 0.36}

 36%|███▌      | 310/870 [38:45<1:17:43,  8.33s/it]
 36%|███▌      | 311/870 [38:52<1:15:27,  8.10s/it]
 36%|███▌      | 312/870 [39:00<1:13:11,  7.87s/it]
 36%|███▌      | 313/870 [39:07<1:10:12,  7.56s/it]
 36%|███▌      | 314/870 [39:14<1:08:31,  7.39s/it]
 36%|███▌      | 315/870 [39:21<1:07:50,  7.33s/it]
 36%|███▋      | 316/870 [39:28<1:06:09,  7.16s/it]
 36%|███▋      | 317/870 [39:34<1:03:44,  6.92s/it]
 37%|███▋      | 318/870 [39:40<1:01:39,  6.70s/it]
 37%|███▋      | 319/870 [39:46<59:24,  6.47s/it]  
 37%|███▋      | 320/870 [39:52<58:45,  6.41s/it]
                                                 
{'loss': 0.021, 'grad_norm': 0.07463961839675903, 'learning_rate': 0.00014640884733793216, 'epoch': 0.37}

 37%|███▋      | 320/870 [39:52<58:45,  6.41s/it]
 37%|███▋      | 321/870 [39:58<56:38,  6.19s/it]
 37%|███▋      | 322/870 [40:03<54:26,  5.96s/it]
 37%|███▋      | 323/870 [40:09<51:47,  5.68s/it]
 37%|███▋      | 324/870 [40:13<49:31,  5.44s/it]
 37%|███▋      | 325/870 [40:17<43:28,  4.79s/it]
 37%|███▋      | 326/870 [40:29<1:02:56,  6.94s/it]
 38%|███▊      | 327/870 [40:39<1:11:20,  7.88s/it]
 38%|███▊      | 328/870 [40:48<1:15:38,  8.37s/it]
 38%|███▊      | 329/870 [40:58<1:18:06,  8.66s/it]
 38%|███▊      | 330/870 [41:06<1:17:32,  8.61s/it]
                                                   
{'loss': 0.0247, 'grad_norm': 0.055279821157455444, 'learning_rate': 0.00014307633535921317, 'epoch': 0.38}

 38%|███▊      | 330/870 [41:06<1:17:32,  8.61s/it]
 38%|███▊      | 331/870 [41:14<1:16:28,  8.51s/it]
 38%|███▊      | 332/870 [41:22<1:14:46,  8.34s/it]
 38%|███▊      | 333/870 [41:30<1:13:23,  8.20s/it]
 38%|███▊      | 334/870 [41:38<1:11:54,  8.05s/it]
 39%|███▊      | 335/870 [41:45<1:09:31,  7.80s/it]
 39%|███▊      | 336/870 [41:52<1:07:59,  7.64s/it]
 39%|███▊      | 337/870 [41:59<1:06:32,  7.49s/it]
 39%|███▉      | 338/870 [42:07<1:05:54,  7.43s/it]
 39%|███▉      | 339/870 [42:13<1:03:53,  7.22s/it]
 39%|███▉      | 340/870 [42:20<1:02:01,  7.02s/it]
                                                   
{'loss': 0.0203, 'grad_norm': 0.18279875814914703, 'learning_rate': 0.00013968400521608969, 'epoch': 0.39}

 39%|███▉      | 340/870 [42:20<1:02:01,  7.02s/it]
 39%|███▉      | 341/870 [42:26<1:00:02,  6.81s/it]
 39%|███▉      | 342/870 [42:33<58:24,  6.64s/it]  
 39%|███▉      | 343/870 [42:39<57:12,  6.51s/it]
 40%|███▉      | 344/870 [42:45<55:41,  6.35s/it]
 40%|███▉      | 345/870 [42:50<53:48,  6.15s/it]
 40%|███▉      | 346/870 [42:56<52:08,  5.97s/it]
 40%|███▉      | 347/870 [43:01<50:28,  5.79s/it]
 40%|████      | 348/870 [43:07<48:51,  5.62s/it]
 40%|████      | 349/870 [43:11<45:32,  5.25s/it]
 40%|████      | 350/870 [43:14<40:32,  4.68s/it]
                                                 
{'loss': 0.0151, 'grad_norm': 0.07882044464349747, 'learning_rate': 0.0001362365676844557, 'epoch': 0.4}

 40%|████      | 350/870 [43:14<40:32,  4.68s/it]
 40%|████      | 351/870 [43:26<59:04,  6.83s/it]
 40%|████      | 352/870 [43:36<1:07:36,  7.83s/it]
 41%|████      | 353/870 [43:46<1:12:04,  8.36s/it]
 41%|████      | 354/870 [43:55<1:14:07,  8.62s/it]
 41%|████      | 355/870 [44:04<1:13:45,  8.59s/it]
 41%|████      | 356/870 [44:12<1:12:59,  8.52s/it]
 41%|████      | 357/870 [44:20<1:12:12,  8.45s/it]
 41%|████      | 358/870 [44:28<1:10:45,  8.29s/it]
 41%|████▏     | 359/870 [44:36<1:09:07,  8.12s/it]
 41%|████▏     | 360/870 [44:44<1:07:42,  7.97s/it]
                                                   
{'loss': 0.0261, 'grad_norm': 0.07913503050804138, 'learning_rate': 0.00013273881006535074, 'epoch': 0.41}

 41%|████▏     | 360/870 [44:44<1:07:42,  7.97s/it]
 41%|████▏     | 361/870 [44:51<1:06:03,  7.79s/it]
 42%|████▏     | 362/870 [44:59<1:05:27,  7.73s/it]
 42%|████▏     | 363/870 [45:06<1:03:51,  7.56s/it]
 42%|████▏     | 364/870 [45:13<1:02:27,  7.41s/it]
 42%|████▏     | 365/870 [45:20<1:01:00,  7.25s/it]
 42%|████▏     | 366/870 [45:26<59:41,  7.11s/it]  
 42%|████▏     | 367/870 [45:33<58:21,  6.96s/it]
 42%|████▏     | 368/870 [45:39<56:28,  6.75s/it]
 42%|████▏     | 369/870 [45:45<54:47,  6.56s/it]
 43%|████▎     | 370/870 [45:51<53:11,  6.38s/it]
                                                 
{'loss': 0.0176, 'grad_norm': 0.06874211877584457, 'learning_rate': 0.00012919558953705055, 'epoch': 0.43}

 43%|████▎     | 370/870 [45:51<53:11,  6.38s/it]
 43%|████▎     | 371/870 [45:57<52:04,  6.26s/it]
 43%|████▎     | 372/870 [46:03<50:13,  6.05s/it]
 43%|████▎     | 373/870 [46:08<48:06,  5.81s/it]
 43%|████▎     | 374/870 [46:13<45:23,  5.49s/it]
 43%|████▎     | 375/870 [46:16<40:06,  4.86s/it]
 43%|████▎     | 376/870 [46:28<56:23,  6.85s/it]
 43%|████▎     | 377/870 [46:38<1:04:00,  7.79s/it]
 43%|████▎     | 378/870 [46:47<1:07:46,  8.26s/it]
 44%|████▎     | 379/870 [46:56<1:09:55,  8.54s/it]
 44%|████▎     | 380/870 [47:05<1:09:41,  8.53s/it]
                                                   
{'loss': 0.0187, 'grad_norm': 0.06528666615486145, 'learning_rate': 0.0001256118264101228, 'epoch': 0.44}

 44%|████▎     | 380/870 [47:05<1:09:41,  8.53s/it]
 44%|████▍     | 381/870 [47:13<1:09:07,  8.48s/it]
 44%|████▍     | 382/870 [47:21<1:07:39,  8.32s/it]
 44%|████▍     | 383/870 [47:29<1:05:39,  8.09s/it]
 44%|████▍     | 384/870 [47:36<1:04:41,  7.99s/it]
 44%|████▍     | 385/870 [47:44<1:03:34,  7.86s/it]
 44%|████▍     | 386/870 [47:51<1:02:18,  7.72s/it]
 44%|████▍     | 387/870 [47:58<1:00:28,  7.51s/it]
 45%|████▍     | 388/870 [48:05<58:38,  7.30s/it]  
 45%|████▍     | 389/870 [48:12<57:11,  7.13s/it]
 45%|████▍     | 390/870 [48:19<56:59,  7.12s/it]
                                                 
{'loss': 0.0158, 'grad_norm': 0.08248886466026306, 'learning_rate': 0.00012199249729481357, 'epoch': 0.45}

 45%|████▍     | 390/870 [48:19<56:59,  7.12s/it]
 45%|████▍     | 391/870 [48:26<55:25,  6.94s/it]
 45%|████▌     | 392/870 [48:32<53:45,  6.75s/it]
 45%|████▌     | 393/870 [48:38<52:17,  6.58s/it]
 45%|████▌     | 394/870 [48:44<50:33,  6.37s/it]
 45%|████▌     | 395/870 [48:50<50:11,  6.34s/it]
 46%|████▌     | 396/870 [48:56<48:29,  6.14s/it]
 46%|████▌     | 397/870 [49:01<46:42,  5.92s/it]
 46%|████▌     | 398/870 [49:06<44:36,  5.67s/it]
 46%|████▌     | 399/870 [49:11<41:11,  5.25s/it]
 46%|████▌     | 400/870 [49:14<36:47,  4.70s/it]
                                                 
{'loss': 0.0132, 'grad_norm': 0.24866768717765808, 'learning_rate': 0.00011834262819025326, 'epoch': 0.46}

 46%|████▌     | 400/870 [49:14<36:47,  4.70s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:12,  2.79it/s]

  8%|▊         | 3/38 [00:01<00:15,  2.28it/s]

 11%|█         | 4/38 [00:01<00:16,  2.03it/s]

 13%|█▎        | 5/38 [00:02<00:16,  2.00it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.05it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.10it/s]

 21%|██        | 8/38 [00:03<00:13,  2.22it/s]

 24%|██▎       | 9/38 [00:04<00:12,  2.40it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.09it/s]

 29%|██▉       | 11/38 [00:05<00:14,  1.86it/s]

 32%|███▏      | 12/38 [00:05<00:14,  1.82it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.82it/s]

 37%|███▋      | 14/38 [00:06<00:12,  1.87it/s]

 39%|███▉      | 15/38 [00:07<00:11,  1.94it/s]

 42%|████▏     | 16/38 [00:07<00:10,  2.02it/s]

 45%|████▍     | 17/38 [00:08<00:09,  2.15it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.25it/s]

 50%|█████     | 19/38 [00:09<00:09,  2.00it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.83it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.82it/s]

 58%|█████▊    | 22/38 [00:11<00:08,  1.80it/s]

 61%|██████    | 23/38 [00:11<00:08,  1.86it/s]

 63%|██████▎   | 24/38 [00:12<00:07,  1.90it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  2.02it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.14it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.29it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.02it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.84it/s]

 79%|███████▉  | 30/38 [00:15<00:04,  1.83it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.87it/s]

 84%|████████▍ | 32/38 [00:16<00:03,  1.94it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.99it/s]

 89%|████████▉ | 34/38 [00:16<00:01,  2.09it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.24it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.52it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.32it/s]

100%|██████████| 38/38 [00:18<00:00,  2.19it/s]
                                                 

                                               
{'eval_loss': 0.018751109018921852, 'eval_runtime': 20.2674, 'eval_samples_per_second': 14.802, 'eval_steps_per_second': 1.875, 'epoch': 0.46}

 46%|████▌     | 400/870 [49:34<36:47,  4.70s/it]

100%|██████████| 38/38 [00:19<00:00,  2.19it/s]

                                               
 46%|████▌     | 401/870 [49:48<1:45:01, 13.44s/it]
 46%|████▌     | 402/870 [49:58<1:37:21, 12.48s/it]
 46%|████▋     | 403/870 [50:08<1:30:29, 11.63s/it]
 46%|████▋     | 404/870 [50:17<1:25:28, 11.00s/it]
 47%|████▋     | 405/870 [50:26<1:20:38, 10.40s/it]
 47%|████▋     | 406/870 [50:35<1:15:28,  9.76s/it]
 47%|████▋     | 407/870 [50:43<1:11:51,  9.31s/it]
 47%|████▋     | 408/870 [50:51<1:08:52,  8.95s/it]
 47%|████▋     | 409/870 [50:59<1:06:56,  8.71s/it]
 47%|████▋     | 410/870 [51:07<1:04:35,  8.43s/it]
                                                   
{'loss': 0.0212, 'grad_norm': 0.11699023842811584, 'learning_rate': 0.00011466728750507805, 'epoch': 0.47}

 47%|████▋     | 410/870 [51:07<1:04:35,  8.43s/it]
 47%|████▋     | 411/870 [51:14<1:02:03,  8.11s/it]
 47%|████▋     | 412/870 [51:21<59:50,  7.84s/it]  
 47%|████▋     | 413/870 [51:29<58:02,  7.62s/it]
 48%|████▊     | 414/870 [51:36<56:29,  7.43s/it]
 48%|████▊     | 415/870 [51:42<54:59,  7.25s/it]
 48%|████▊     | 416/870 [51:49<53:30,  7.07s/it]
 48%|████▊     | 417/870 [51:55<51:06,  6.77s/it]
 48%|████▊     | 418/870 [52:01<49:42,  6.60s/it]
 48%|████▊     | 419/870 [52:08<49:15,  6.55s/it]
 48%|████▊     | 420/870 [52:14<47:50,  6.38s/it]
                                                 
{'loss': 0.0176, 'grad_norm': 0.08753076195716858, 'learning_rate': 0.00011097157901915953, 'epoch': 0.48}

 48%|████▊     | 420/870 [52:14<47:50,  6.38s/it]
 48%|████▊     | 421/870 [52:20<46:20,  6.19s/it]
 49%|████▊     | 422/870 [52:25<44:49,  6.00s/it]
 49%|████▊     | 423/870 [52:31<43:52,  5.89s/it]
 49%|████▊     | 424/870 [52:35<41:00,  5.52s/it]
 49%|████▉     | 425/870 [52:39<35:49,  4.83s/it]
 49%|████▉     | 426/870 [52:51<52:27,  7.09s/it]
 49%|████▉     | 427/870 [53:01<59:44,  8.09s/it]
 49%|████▉     | 428/870 [53:11<1:03:26,  8.61s/it]
 49%|████▉     | 429/870 [53:21<1:04:57,  8.84s/it]
 49%|████▉     | 430/870 [53:30<1:05:14,  8.90s/it]
                                                   
{'loss': 0.0226, 'grad_norm': 0.10365070402622223, 'learning_rate': 0.00010726063479621573, 'epoch': 0.49}

 49%|████▉     | 430/870 [53:30<1:05:14,  8.90s/it]
 50%|████▉     | 431/870 [53:38<1:04:13,  8.78s/it]
 50%|████▉     | 432/870 [53:46<1:03:11,  8.66s/it]
 50%|████▉     | 433/870 [53:54<1:01:32,  8.45s/it]
 50%|████▉     | 434/870 [54:02<59:50,  8.23s/it]  
 50%|█████     | 435/870 [54:10<57:53,  7.99s/it]
 50%|█████     | 436/870 [54:17<56:53,  7.87s/it]
 50%|█████     | 437/870 [54:25<56:21,  7.81s/it]
 50%|█████     | 438/870 [54:32<54:49,  7.61s/it]
 50%|█████     | 439/870 [54:39<53:13,  7.41s/it]
 51%|█████     | 440/870 [54:46<51:57,  7.25s/it]
                                                 
{'loss': 0.0189, 'grad_norm': 0.18819937109947205, 'learning_rate': 0.00010353960805714566, 'epoch': 0.51}

 51%|█████     | 440/870 [54:46<51:57,  7.25s/it]
 51%|█████     | 441/870 [54:53<50:52,  7.12s/it]
 51%|█████     | 442/870 [54:59<49:40,  6.96s/it]
 51%|█████     | 443/870 [55:05<47:53,  6.73s/it]
 51%|█████     | 444/870 [55:11<45:58,  6.48s/it]
 51%|█████     | 445/870 [55:17<44:17,  6.25s/it]
 51%|█████▏    | 446/870 [55:23<43:16,  6.12s/it]
 51%|█████▏    | 447/870 [55:28<41:25,  5.88s/it]
 51%|█████▏    | 448/870 [55:33<39:07,  5.56s/it]
 52%|█████▏    | 449/870 [55:37<36:22,  5.18s/it]
 52%|█████▏    | 450/870 [55:40<31:44,  4.53s/it]
                                                 
{'loss': 0.0129, 'grad_norm': 0.10421279072761536, 'learning_rate': 9.98136660239837e-05, 'epoch': 0.52}

 52%|█████▏    | 450/870 [55:40<31:44,  4.53s/it]
 52%|█████▏    | 451/870 [55:53<47:53,  6.86s/it]
 52%|█████▏    | 452/870 [56:03<54:37,  7.84s/it]
 52%|█████▏    | 453/870 [56:12<57:45,  8.31s/it]
 52%|█████▏    | 454/870 [56:21<59:09,  8.53s/it]
 52%|█████▏    | 455/870 [56:30<58:58,  8.53s/it]
 52%|█████▏    | 456/870 [56:38<58:56,  8.54s/it]
 53%|█████▎    | 457/870 [56:46<58:00,  8.43s/it]
 53%|█████▎    | 458/870 [56:54<56:47,  8.27s/it]
 53%|█████▎    | 459/870 [57:02<55:12,  8.06s/it]
 53%|█████▎    | 460/870 [57:09<54:01,  7.91s/it]
                                                 
{'loss': 0.0224, 'grad_norm': 0.07066013664007187, 'learning_rate': 9.60879827444116e-05, 'epoch': 0.53}

 53%|█████▎    | 460/870 [57:09<54:01,  7.91s/it]
 53%|█████▎    | 461/870 [57:17<52:45,  7.74s/it]
 53%|█████▎    | 462/870 [57:24<51:24,  7.56s/it]
 53%|█████▎    | 463/870 [57:31<50:13,  7.40s/it]
 53%|█████▎    | 464/870 [57:38<48:41,  7.20s/it]
 53%|█████▎    | 465/870 [57:45<48:17,  7.15s/it]
 54%|█████▎    | 466/870 [57:51<46:27,  6.90s/it]
 54%|█████▎    | 467/870 [57:57<44:48,  6.67s/it]
 54%|█████▍    | 468/870 [58:03<43:18,  6.46s/it]
 54%|█████▍    | 469/870 [58:09<42:14,  6.32s/it]
 54%|█████▍    | 470/870 [58:15<41:55,  6.29s/it]
                                                 
{'loss': 0.0158, 'grad_norm': 0.04448763281106949, 'learning_rate': 9.236773190679163e-05, 'epoch': 0.54}

 54%|█████▍    | 470/870 [58:15<41:55,  6.29s/it]
 54%|█████▍    | 471/870 [58:21<40:20,  6.07s/it]
 54%|█████▍    | 472/870 [58:26<38:49,  5.85s/it]
 54%|█████▍    | 473/870 [58:32<37:52,  5.72s/it]
 54%|█████▍    | 474/870 [58:36<35:57,  5.45s/it]
 55%|█████▍    | 475/870 [58:40<31:08,  4.73s/it]
 55%|█████▍    | 476/870 [58:52<46:56,  7.15s/it]
 55%|█████▍    | 477/870 [59:03<53:25,  8.16s/it]
 55%|█████▍    | 478/870 [59:13<56:14,  8.61s/it]
 55%|█████▌    | 479/870 [59:22<57:48,  8.87s/it]
 55%|█████▌    | 480/870 [59:31<57:23,  8.83s/it]
                                                 
{'loss': 0.0216, 'grad_norm': 0.06225728988647461, 'learning_rate': 8.865807965569879e-05, 'epoch': 0.55}

 55%|█████▌    | 480/870 [59:31<57:23,  8.83s/it]
 55%|█████▌    | 481/870 [59:39<56:35,  8.73s/it]
 55%|█████▌    | 482/870 [59:47<55:34,  8.59s/it]
 56%|█████▌    | 483/870 [59:56<54:26,  8.44s/it]
 56%|█████▌    | 484/870 [1:00:04<53:45,  8.36s/it]
 56%|█████▌    | 485/870 [1:00:11<52:25,  8.17s/it]
 56%|█████▌    | 486/870 [1:00:19<51:10,  8.00s/it]
 56%|█████▌    | 487/870 [1:00:26<49:58,  7.83s/it]
 56%|█████▌    | 488/870 [1:00:34<48:33,  7.63s/it]
 56%|█████▌    | 489/870 [1:00:41<48:09,  7.58s/it]
 56%|█████▋    | 490/870 [1:00:48<46:42,  7.38s/it]
                                                   
{'loss': 0.022, 'grad_norm': 0.12085612863302231, 'learning_rate': 8.496417741792913e-05, 'epoch': 0.56}

 56%|█████▋    | 490/870 [1:00:48<46:42,  7.38s/it]
 56%|█████▋    | 491/870 [1:00:55<45:13,  7.16s/it]
 57%|█████▋    | 492/870 [1:01:01<43:08,  6.85s/it]
 57%|█████▋    | 493/870 [1:01:07<42:28,  6.76s/it]
 57%|█████▋    | 494/870 [1:01:13<41:02,  6.55s/it]
 57%|█████▋    | 495/870 [1:01:19<39:42,  6.35s/it]
 57%|█████▋    | 496/870 [1:01:25<38:37,  6.20s/it]
 57%|█████▋    | 497/870 [1:01:31<37:28,  6.03s/it]
 57%|█████▋    | 498/870 [1:01:36<36:36,  5.90s/it]
 57%|█████▋    | 499/870 [1:01:41<34:25,  5.57s/it]
 57%|█████▋    | 500/870 [1:01:45<30:20,  4.92s/it]
                                                   
{'loss': 0.0152, 'grad_norm': 0.04715719074010849, 'learning_rate': 8.129115474894523e-05, 'epoch': 0.57}

 57%|█████▋    | 500/870 [1:01:45<30:20,  4.92s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:13,  2.68it/s]

  8%|▊         | 3/38 [00:01<00:16,  2.09it/s]

 11%|█         | 4/38 [00:01<00:17,  1.98it/s]

 13%|█▎        | 5/38 [00:02<00:16,  1.98it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.03it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.09it/s]

 21%|██        | 8/38 [00:03<00:13,  2.18it/s]

 24%|██▎       | 9/38 [00:04<00:12,  2.33it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.04it/s]

 29%|██▉       | 11/38 [00:05<00:14,  1.84it/s]

 32%|███▏      | 12/38 [00:05<00:14,  1.83it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.84it/s]

 37%|███▋      | 14/38 [00:07<00:12,  1.87it/s]

 39%|███▉      | 15/38 [00:07<00:11,  1.95it/s]

 42%|████▏     | 16/38 [00:07<00:10,  2.04it/s]

 45%|████▍     | 17/38 [00:08<00:09,  2.16it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.31it/s]

 50%|█████     | 19/38 [00:09<00:09,  2.03it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.85it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.85it/s]

 58%|█████▊    | 22/38 [00:11<00:08,  1.81it/s]

 61%|██████    | 23/38 [00:11<00:08,  1.86it/s]

 63%|██████▎   | 24/38 [00:12<00:07,  1.95it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  2.01it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.14it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.36it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.07it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.87it/s]

 79%|███████▉  | 30/38 [00:15<00:04,  1.87it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.90it/s]

 84%|████████▍ | 32/38 [00:16<00:03,  1.93it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.96it/s]

 89%|████████▉ | 34/38 [00:16<00:01,  2.03it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.15it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.37it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.18it/s]

100%|██████████| 38/38 [00:18<00:00,  2.07it/s]
                                                   

                                               
{'eval_loss': 0.01953456737101078, 'eval_runtime': 20.3994, 'eval_samples_per_second': 14.706, 'eval_steps_per_second': 1.863, 'epoch': 0.57}

 57%|█████▋    | 500/870 [1:02:05<30:20,  4.92s/it]

100%|██████████| 38/38 [00:19<00:00,  2.07it/s]

                                               
 58%|█████▊    | 501/870 [1:02:17<1:20:57, 13.16s/it]
 58%|█████▊    | 502/870 [1:02:27<1:15:22, 12.29s/it]
 58%|█████▊    | 503/870 [1:02:37<1:10:29, 11.53s/it]
 58%|█████▊    | 504/870 [1:02:46<1:06:10, 10.85s/it]
 58%|█████▊    | 505/870 [1:02:55<1:01:49, 10.16s/it]
 58%|█████▊    | 506/870 [1:03:03<58:26,  9.63s/it]  
 58%|█████▊    | 507/870 [1:03:11<55:25,  9.16s/it]
 58%|█████▊    | 508/870 [1:03:20<53:56,  8.94s/it]
 59%|█████▊    | 509/870 [1:03:28<51:54,  8.63s/it]
 59%|█████▊    | 510/870 [1:03:35<49:32,  8.26s/it]
                                                   
{'loss': 0.0214, 'grad_norm': 0.08593538403511047, 'learning_rate': 7.764411220969414e-05, 'epoch': 0.59}

 59%|█████▊    | 510/870 [1:03:35<49:32,  8.26s/it]
 59%|█████▊    | 511/870 [1:03:43<48:11,  8.05s/it]
 59%|█████▉    | 512/870 [1:03:50<46:57,  7.87s/it]
 59%|█████▉    | 513/870 [1:03:57<45:31,  7.65s/it]
 59%|█████▉    | 514/870 [1:04:04<44:12,  7.45s/it]
 59%|█████▉    | 515/870 [1:04:11<42:45,  7.23s/it]
 59%|█████▉    | 516/870 [1:04:17<41:36,  7.05s/it]
 59%|█████▉    | 517/870 [1:04:24<41:08,  6.99s/it]
 60%|█████▉    | 518/870 [1:04:30<39:29,  6.73s/it]
 60%|█████▉    | 519/870 [1:04:36<37:58,  6.49s/it]
 60%|█████▉    | 520/870 [1:04:42<36:58,  6.34s/it]
                                                   
{'loss': 0.0174, 'grad_norm': 0.08397146314382553, 'learning_rate': 7.402811428368832e-05, 'epoch': 0.6}

 60%|█████▉    | 520/870 [1:04:42<36:58,  6.34s/it]
 60%|█████▉    | 521/870 [1:04:48<36:14,  6.23s/it]
 60%|██████    | 522/870 [1:04:54<34:53,  6.02s/it]
 60%|██████    | 523/870 [1:04:59<33:14,  5.75s/it]
 60%|██████    | 524/870 [1:05:04<31:26,  5.45s/it]
 60%|██████    | 525/870 [1:05:07<27:32,  4.79s/it]
 60%|██████    | 526/870 [1:05:19<39:48,  6.94s/it]
 61%|██████    | 527/870 [1:05:29<45:16,  7.92s/it]
 61%|██████    | 528/870 [1:05:39<47:53,  8.40s/it]
 61%|██████    | 529/870 [1:05:48<49:15,  8.67s/it]
 61%|██████    | 530/870 [1:05:57<49:26,  8.73s/it]
                                                   
{'loss': 0.0163, 'grad_norm': 0.07681377232074738, 'learning_rate': 7.044818234418576e-05, 'epoch': 0.61}

 61%|██████    | 530/870 [1:05:57<49:26,  8.73s/it]
 61%|██████    | 531/870 [1:06:06<49:16,  8.72s/it]
 61%|██████    | 532/870 [1:06:14<48:22,  8.59s/it]
 61%|██████▏   | 533/870 [1:06:22<47:14,  8.41s/it]
 61%|██████▏   | 534/870 [1:06:30<46:11,  8.25s/it]
 61%|██████▏   | 535/870 [1:06:37<45:00,  8.06s/it]
 62%|██████▏   | 536/870 [1:06:45<44:25,  7.98s/it]
 62%|██████▏   | 537/870 [1:06:52<42:59,  7.75s/it]
 62%|██████▏   | 538/870 [1:06:59<41:32,  7.51s/it]
 62%|██████▏   | 539/870 [1:07:06<40:22,  7.32s/it]
 62%|██████▏   | 540/870 [1:07:13<39:39,  7.21s/it]
                                                   
{'loss': 0.0135, 'grad_norm': 0.061246950179338455, 'learning_rate': 6.69092876812353e-05, 'epoch': 0.62}

 62%|██████▏   | 540/870 [1:07:13<39:39,  7.21s/it]
 62%|██████▏   | 541/870 [1:07:19<38:06,  6.95s/it]
 62%|██████▏   | 542/870 [1:07:26<36:41,  6.71s/it]
 62%|██████▏   | 543/870 [1:07:32<35:28,  6.51s/it]
 63%|██████▎   | 544/870 [1:07:38<34:22,  6.33s/it]
 63%|██████▎   | 545/870 [1:07:44<34:03,  6.29s/it]
 63%|██████▎   | 546/870 [1:07:49<32:58,  6.11s/it]
 63%|██████▎   | 547/870 [1:07:55<31:39,  5.88s/it]
 63%|██████▎   | 548/870 [1:08:00<30:10,  5.62s/it]
 63%|██████▎   | 549/870 [1:08:04<28:21,  5.30s/it]
 63%|██████▎   | 550/870 [1:08:08<25:17,  4.74s/it]
                                                   
{'loss': 0.0111, 'grad_norm': 0.1086721196770668, 'learning_rate': 6.341634459827052e-05, 'epoch': 0.63}

 63%|██████▎   | 550/870 [1:08:08<25:17,  4.74s/it]
 63%|██████▎   | 551/870 [1:08:20<36:24,  6.85s/it]
 63%|██████▎   | 552/870 [1:08:30<41:18,  7.79s/it]
 64%|██████▎   | 553/870 [1:08:39<43:59,  8.33s/it]
 64%|██████▎   | 554/870 [1:08:48<45:18,  8.60s/it]
 64%|██████▍   | 555/870 [1:08:57<45:41,  8.70s/it]
 64%|██████▍   | 556/870 [1:09:06<45:01,  8.60s/it]
 64%|██████▍   | 557/870 [1:09:14<44:17,  8.49s/it]
 64%|██████▍   | 558/870 [1:09:22<43:17,  8.33s/it]
 64%|██████▍   | 559/870 [1:09:30<42:35,  8.22s/it]
 64%|██████▍   | 560/870 [1:09:37<41:04,  7.95s/it]
                                                   
{'loss': 0.0185, 'grad_norm': 0.10877221822738647, 'learning_rate': 5.9974203587837986e-05, 'epoch': 0.64}

 64%|██████▍   | 560/870 [1:09:37<41:04,  7.95s/it]
 64%|██████▍   | 561/870 [1:09:44<40:02,  7.78s/it]
 65%|██████▍   | 562/870 [1:09:52<38:46,  7.55s/it]
 65%|██████▍   | 563/870 [1:09:59<37:56,  7.42s/it]
 65%|██████▍   | 564/870 [1:10:06<37:29,  7.35s/it]
 65%|██████▍   | 565/870 [1:10:12<36:07,  7.11s/it]
 65%|██████▌   | 566/870 [1:10:19<35:20,  6.98s/it]
 65%|██████▌   | 567/870 [1:10:25<34:13,  6.78s/it]
 65%|██████▌   | 568/870 [1:10:32<33:19,  6.62s/it]
 65%|██████▌   | 569/870 [1:10:38<32:50,  6.55s/it]
 66%|██████▌   | 570/870 [1:10:44<31:47,  6.36s/it]
                                                   
{'loss': 0.0134, 'grad_norm': 0.25607386231422424, 'learning_rate': 5.658764459593691e-05, 'epoch': 0.66}

 66%|██████▌   | 570/870 [1:10:44<31:47,  6.36s/it]
 66%|██████▌   | 571/870 [1:10:50<30:39,  6.15s/it]
 66%|██████▌   | 572/870 [1:10:55<29:31,  5.94s/it]
 66%|██████▌   | 573/870 [1:11:01<28:46,  5.81s/it]
 66%|██████▌   | 574/870 [1:11:05<26:32,  5.38s/it]
 66%|██████▌   | 575/870 [1:11:08<22:50,  4.65s/it]
 66%|██████▌   | 576/870 [1:11:20<33:34,  6.85s/it]
 66%|██████▋   | 577/870 [1:11:30<38:41,  7.92s/it]
 66%|██████▋   | 578/870 [1:11:40<41:22,  8.50s/it]
 67%|██████▋   | 579/870 [1:11:49<42:23,  8.74s/it]
 67%|██████▋   | 580/870 [1:11:58<41:58,  8.68s/it]
                                                   
{'loss': 0.0155, 'grad_norm': 0.06046198308467865, 'learning_rate': 5.3261370384323984e-05, 'epoch': 0.67}

 67%|██████▋   | 580/870 [1:11:58<41:58,  8.68s/it]
 67%|██████▋   | 581/870 [1:12:06<41:24,  8.60s/it]
 67%|██████▋   | 582/870 [1:12:14<40:27,  8.43s/it]
 67%|██████▋   | 583/870 [1:12:22<39:25,  8.24s/it]
 67%|██████▋   | 584/870 [1:12:30<38:25,  8.06s/it]
 67%|██████▋   | 585/870 [1:12:37<37:24,  7.88s/it]
 67%|██████▋   | 586/870 [1:12:45<36:47,  7.77s/it]
 67%|██████▋   | 587/870 [1:12:52<35:50,  7.60s/it]
 68%|██████▊   | 588/870 [1:12:59<35:26,  7.54s/it]
 68%|██████▊   | 589/870 [1:13:06<34:33,  7.38s/it]
 68%|██████▊   | 590/870 [1:13:13<33:17,  7.13s/it]
                                                   
{'loss': 0.0156, 'grad_norm': 0.08047638833522797, 'learning_rate': 5.000000000000002e-05, 'epoch': 0.68}

 68%|██████▊   | 590/870 [1:13:13<33:17,  7.13s/it]
 68%|██████▊   | 591/870 [1:13:19<32:17,  6.94s/it]
 68%|██████▊   | 592/870 [1:13:26<31:17,  6.75s/it]
 68%|██████▊   | 593/870 [1:13:32<30:55,  6.70s/it]
 68%|██████▊   | 594/870 [1:13:38<30:00,  6.52s/it]
 68%|██████▊   | 595/870 [1:13:44<29:08,  6.36s/it]
 69%|██████▊   | 596/870 [1:13:50<28:06,  6.15s/it]
 69%|██████▊   | 597/870 [1:13:56<27:26,  6.03s/it]
 69%|██████▊   | 598/870 [1:14:01<26:04,  5.75s/it]
 69%|██████▉   | 599/870 [1:14:05<24:20,  5.39s/it]
 69%|██████▉   | 600/870 [1:14:09<21:06,  4.69s/it]
                                                   
{'loss': 0.0118, 'grad_norm': 0.09806278347969055, 'learning_rate': 4.680806236094756e-05, 'epoch': 0.69}

 69%|██████▉   | 600/870 [1:14:09<21:06,  4.69s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:12,  2.78it/s]

  8%|▊         | 3/38 [00:01<00:15,  2.29it/s]

 11%|█         | 4/38 [00:01<00:16,  2.09it/s]

 13%|█▎        | 5/38 [00:02<00:16,  2.04it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.05it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.17it/s]

 21%|██        | 8/38 [00:03<00:13,  2.28it/s]

 24%|██▎       | 9/38 [00:03<00:11,  2.62it/s]

 26%|██▋       | 10/38 [00:04<00:12,  2.25it/s]

 29%|██▉       | 11/38 [00:05<00:13,  1.95it/s]

 32%|███▏      | 12/38 [00:05<00:13,  1.91it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.86it/s]

 37%|███▋      | 14/38 [00:06<00:12,  1.90it/s]

 39%|███▉      | 15/38 [00:07<00:11,  1.93it/s]

 42%|████▏     | 16/38 [00:07<00:10,  2.01it/s]

 45%|████▍     | 17/38 [00:08<00:09,  2.12it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.28it/s]

 50%|█████     | 19/38 [00:09<00:09,  2.02it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.80it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.78it/s]

 58%|█████▊    | 22/38 [00:10<00:08,  1.84it/s]

 61%|██████    | 23/38 [00:11<00:07,  1.89it/s]

 63%|██████▎   | 24/38 [00:11<00:07,  1.98it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  2.05it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.16it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.26it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.05it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.87it/s]

 79%|███████▉  | 30/38 [00:14<00:04,  1.87it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.88it/s]

 84%|████████▍ | 32/38 [00:15<00:03,  1.90it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.98it/s]

 89%|████████▉ | 34/38 [00:16<00:01,  2.05it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.17it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.31it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.14it/s]

100%|██████████| 38/38 [00:18<00:00,  2.07it/s]
                                                   

                                               
{'eval_loss': 0.016160501167178154, 'eval_runtime': 20.2233, 'eval_samples_per_second': 14.834, 'eval_steps_per_second': 1.879, 'epoch': 0.69}

 69%|██████▉   | 600/870 [1:14:29<21:06,  4.69s/it]

100%|██████████| 38/38 [00:19<00:00,  2.07it/s]

                                               
 69%|██████▉   | 601/870 [1:14:42<59:36, 13.30s/it]
 69%|██████▉   | 602/870 [1:14:53<55:49, 12.50s/it]
 69%|██████▉   | 603/870 [1:15:02<51:40, 11.61s/it]
 69%|██████▉   | 604/870 [1:15:11<48:18, 10.90s/it]
 70%|██████▉   | 605/870 [1:15:20<44:58, 10.18s/it]
 70%|██████▉   | 606/870 [1:15:28<42:06,  9.57s/it]
 70%|██████▉   | 607/870 [1:15:37<40:32,  9.25s/it]
 70%|██████▉   | 608/870 [1:15:45<38:50,  8.89s/it]
 70%|███████   | 609/870 [1:15:52<37:12,  8.55s/it]
 70%|███████   | 610/870 [1:16:00<35:35,  8.21s/it]
                                                   
{'loss': 0.0154, 'grad_norm': 0.12618760764598846, 'learning_rate': 4.3689989967026935e-05, 'epoch': 0.7}

 70%|███████   | 610/870 [1:16:00<35:35,  8.21s/it]
 70%|███████   | 611/870 [1:16:07<34:39,  8.03s/it]
 70%|███████   | 612/870 [1:16:15<33:25,  7.77s/it]
 70%|███████   | 613/870 [1:16:21<32:03,  7.48s/it]
 71%|███████   | 614/870 [1:16:28<31:09,  7.30s/it]
 71%|███████   | 615/870 [1:16:35<30:24,  7.15s/it]
 71%|███████   | 616/870 [1:16:42<29:59,  7.09s/it]
 71%|███████   | 617/870 [1:16:48<28:44,  6.82s/it]
 71%|███████   | 618/870 [1:16:54<27:50,  6.63s/it]
 71%|███████   | 619/870 [1:17:00<26:40,  6.37s/it]
 71%|███████▏  | 620/870 [1:17:06<25:58,  6.23s/it]
                                                   
{'loss': 0.0105, 'grad_norm': 0.16422143578529358, 'learning_rate': 4.065011274476349e-05, 'epoch': 0.71}

 71%|███████▏  | 620/870 [1:17:06<25:58,  6.23s/it]
 71%|███████▏  | 621/870 [1:17:12<25:25,  6.13s/it]
 71%|███████▏  | 622/870 [1:17:17<24:32,  5.94s/it]
 72%|███████▏  | 623/870 [1:17:22<23:23,  5.68s/it]
 72%|███████▏  | 624/870 [1:17:27<21:49,  5.32s/it]
 72%|███████▏  | 625/870 [1:17:30<19:02,  4.66s/it]
 72%|███████▏  | 626/870 [1:17:42<28:11,  6.93s/it]
 72%|███████▏  | 627/870 [1:17:53<32:06,  7.93s/it]
 72%|███████▏  | 628/870 [1:18:02<33:55,  8.41s/it]
 72%|███████▏  | 629/870 [1:18:11<34:45,  8.65s/it]
 72%|███████▏  | 630/870 [1:18:20<34:27,  8.62s/it]
                                                   
{'loss': 0.0155, 'grad_norm': 0.3045821487903595, 'learning_rate': 3.769265203457375e-05, 'epoch': 0.72}

 72%|███████▏  | 630/870 [1:18:20<34:27,  8.62s/it]
 73%|███████▎  | 631/870 [1:18:28<34:21,  8.63s/it]
 73%|███████▎  | 632/870 [1:18:37<33:45,  8.51s/it]
 73%|███████▎  | 633/870 [1:18:45<33:00,  8.36s/it]
 73%|███████▎  | 634/870 [1:18:52<32:06,  8.16s/it]
 73%|███████▎  | 635/870 [1:19:00<31:38,  8.08s/it]
 73%|███████▎  | 636/870 [1:19:08<30:40,  7.86s/it]
 73%|███████▎  | 637/870 [1:19:15<29:34,  7.61s/it]
 73%|███████▎  | 638/870 [1:19:22<28:31,  7.38s/it]
 73%|███████▎  | 639/870 [1:19:28<27:52,  7.24s/it]
 74%|███████▎  | 640/870 [1:19:35<27:14,  7.11s/it]
                                                   
{'loss': 0.0158, 'grad_norm': 0.12457603216171265, 'learning_rate': 3.482171472878062e-05, 'epoch': 0.74}

 74%|███████▎  | 640/870 [1:19:35<27:14,  7.11s/it]
 74%|███████▎  | 641/870 [1:19:42<26:30,  6.94s/it]
 74%|███████▍  | 642/870 [1:19:48<25:31,  6.72s/it]
 74%|███████▍  | 643/870 [1:19:54<24:51,  6.57s/it]
 74%|███████▍  | 644/870 [1:20:00<24:08,  6.41s/it]
 74%|███████▍  | 645/870 [1:20:06<23:36,  6.30s/it]
 74%|███████▍  | 646/870 [1:20:12<22:45,  6.10s/it]
 74%|███████▍  | 647/870 [1:20:17<21:48,  5.87s/it]
 74%|███████▍  | 648/870 [1:20:22<20:50,  5.64s/it]
 75%|███████▍  | 649/870 [1:20:27<19:46,  5.37s/it]
 75%|███████▍  | 650/870 [1:20:30<17:04,  4.66s/it]
                                                   
{'loss': 0.0096, 'grad_norm': 0.04488776624202728, 'learning_rate': 3.20412875685569e-05, 'epoch': 0.75}

 75%|███████▍  | 650/870 [1:20:30<17:04,  4.66s/it]
 75%|███████▍  | 651/870 [1:20:42<25:05,  6.87s/it]
 75%|███████▍  | 652/870 [1:20:52<28:44,  7.91s/it]
 75%|███████▌  | 653/870 [1:21:02<30:26,  8.42s/it]
 75%|███████▌  | 654/870 [1:21:12<31:38,  8.79s/it]
 75%|███████▌  | 655/870 [1:21:20<31:12,  8.71s/it]
 75%|███████▌  | 656/870 [1:21:29<30:39,  8.60s/it]
 76%|███████▌  | 657/870 [1:21:37<30:06,  8.48s/it]
 76%|███████▌  | 658/870 [1:21:45<29:23,  8.32s/it]
 76%|███████▌  | 659/870 [1:21:53<28:58,  8.24s/it]
 76%|███████▌  | 660/870 [1:22:00<28:09,  8.05s/it]
                                                   
{'loss': 0.0164, 'grad_norm': 0.05114804208278656, 'learning_rate': 2.935523160771786e-05, 'epoch': 0.76}

 76%|███████▌  | 660/870 [1:22:00<28:09,  8.05s/it]
 76%|███████▌  | 661/870 [1:22:08<27:14,  7.82s/it]
 76%|███████▌  | 662/870 [1:22:15<26:26,  7.63s/it]
 76%|███████▌  | 663/870 [1:22:22<26:02,  7.55s/it]
 76%|███████▋  | 664/870 [1:22:29<25:17,  7.37s/it]
 76%|███████▋  | 665/870 [1:22:36<24:37,  7.21s/it]
 77%|███████▋  | 666/870 [1:22:42<23:37,  6.95s/it]
 77%|███████▋  | 667/870 [1:22:49<22:49,  6.75s/it]
 77%|███████▋  | 668/870 [1:22:55<22:23,  6.65s/it]
 77%|███████▋  | 669/870 [1:23:01<21:36,  6.45s/it]
 77%|███████▋  | 670/870 [1:23:07<20:48,  6.24s/it]
                                                   
{'loss': 0.0115, 'grad_norm': 0.03534635156393051, 'learning_rate': 2.6767276851049816e-05, 'epoch': 0.77}

 77%|███████▋  | 670/870 [1:23:07<20:48,  6.24s/it]
 77%|███████▋  | 671/870 [1:23:12<20:05,  6.06s/it]
 77%|███████▋  | 672/870 [1:23:18<19:20,  5.86s/it]
 77%|███████▋  | 673/870 [1:23:23<18:47,  5.72s/it]
 77%|███████▋  | 674/870 [1:23:28<17:23,  5.32s/it]
 78%|███████▊  | 675/870 [1:23:31<15:05,  4.64s/it]
 78%|███████▊  | 676/870 [1:23:42<21:27,  6.64s/it]
 78%|███████▊  | 677/870 [1:23:52<24:34,  7.64s/it]
 78%|███████▊  | 678/870 [1:24:02<26:23,  8.25s/it]
 78%|███████▊  | 679/870 [1:24:11<27:12,  8.55s/it]
 78%|███████▊  | 680/870 [1:24:20<27:13,  8.60s/it]
                                                   
{'loss': 0.0138, 'grad_norm': 0.06601171940565109, 'learning_rate': 2.4281017074620584e-05, 'epoch': 0.78}

 78%|███████▊  | 680/870 [1:24:20<27:13,  8.60s/it]
 78%|███████▊  | 681/870 [1:24:28<26:39,  8.46s/it]
 78%|███████▊  | 682/870 [1:24:36<26:32,  8.47s/it]
 79%|███████▊  | 683/870 [1:24:44<25:56,  8.33s/it]
 79%|███████▊  | 684/870 [1:24:52<25:20,  8.18s/it]
 79%|███████▊  | 685/870 [1:25:00<24:39,  8.00s/it]
 79%|███████▉  | 686/870 [1:25:07<24:00,  7.83s/it]
 79%|███████▉  | 687/870 [1:25:14<23:30,  7.71s/it]
 79%|███████▉  | 688/870 [1:25:22<22:50,  7.53s/it]
 79%|███████▉  | 689/870 [1:25:29<22:10,  7.35s/it]
 79%|███████▉  | 690/870 [1:25:35<21:34,  7.19s/it]
                                                   
{'loss': 0.0142, 'grad_norm': 0.10086585581302643, 'learning_rate': 2.1899904835264896e-05, 'epoch': 0.79}

 79%|███████▉  | 690/870 [1:25:35<21:34,  7.19s/it]
 79%|███████▉  | 691/870 [1:25:42<20:45,  6.96s/it]
 80%|███████▉  | 692/870 [1:25:48<20:12,  6.81s/it]
 80%|███████▉  | 693/870 [1:25:54<19:30,  6.61s/it]
 80%|███████▉  | 694/870 [1:26:00<18:46,  6.40s/it]
 80%|███████▉  | 695/870 [1:26:06<18:13,  6.25s/it]
 80%|████████  | 696/870 [1:26:12<17:36,  6.07s/it]
 80%|████████  | 697/870 [1:26:18<17:13,  5.97s/it]
 80%|████████  | 698/870 [1:26:23<16:15,  5.67s/it]
 80%|████████  | 699/870 [1:26:27<14:49,  5.20s/it]
 80%|████████  | 700/870 [1:26:30<12:51,  4.54s/it]
                                                   
{'loss': 0.0098, 'grad_norm': 0.11490526050329208, 'learning_rate': 1.962724667617436e-05, 'epoch': 0.8}

 80%|████████  | 700/870 [1:26:30<12:51,  4.54s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:13,  2.62it/s]

  8%|▊         | 3/38 [00:01<00:15,  2.20it/s]

 11%|█         | 4/38 [00:01<00:16,  2.08it/s]

 13%|█▎        | 5/38 [00:02<00:15,  2.08it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.10it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.18it/s]

 21%|██        | 8/38 [00:03<00:13,  2.27it/s]

 24%|██▎       | 9/38 [00:03<00:11,  2.44it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.10it/s]

 29%|██▉       | 11/38 [00:05<00:14,  1.82it/s]

 32%|███▏      | 12/38 [00:05<00:15,  1.73it/s]

 34%|███▍      | 13/38 [00:06<00:14,  1.76it/s]

 37%|███▋      | 14/38 [00:06<00:13,  1.83it/s]

 39%|███▉      | 15/38 [00:07<00:12,  1.91it/s]

 42%|████▏     | 16/38 [00:07<00:11,  1.99it/s]

 45%|████▍     | 17/38 [00:08<00:09,  2.12it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.28it/s]

 50%|█████     | 19/38 [00:09<00:09,  2.01it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.83it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.84it/s]

 58%|█████▊    | 22/38 [00:11<00:08,  1.82it/s]

 61%|██████    | 23/38 [00:11<00:08,  1.87it/s]

 63%|██████▎   | 24/38 [00:12<00:07,  1.91it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  1.98it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.08it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.27it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.02it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.83it/s]

 79%|███████▉  | 30/38 [00:15<00:04,  1.82it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.85it/s]

 84%|████████▍ | 32/38 [00:16<00:03,  1.89it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.98it/s]

 89%|████████▉ | 34/38 [00:17<00:01,  2.10it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.19it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.41it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.29it/s]

100%|██████████| 38/38 [00:18<00:00,  2.17it/s]
                                                   

                                               
{'eval_loss': 0.012033017352223396, 'eval_runtime': 20.3491, 'eval_samples_per_second': 14.743, 'eval_steps_per_second': 1.867, 'epoch': 0.8}

 80%|████████  | 700/870 [1:26:50<12:51,  4.54s/it]

100%|██████████| 38/38 [00:19<00:00,  2.17it/s]

                                               
 81%|████████  | 701/870 [1:27:02<36:11, 12.85s/it]
 81%|████████  | 702/870 [1:27:12<33:54, 12.11s/it]
 81%|████████  | 703/870 [1:27:22<31:32, 11.33s/it]
 81%|████████  | 704/870 [1:27:31<29:36, 10.70s/it]
 81%|████████  | 705/870 [1:27:40<27:46, 10.10s/it]
 81%|████████  | 706/870 [1:27:49<26:35,  9.73s/it]
 81%|████████▏ | 707/870 [1:27:56<24:54,  9.17s/it]
 81%|████████▏ | 708/870 [1:28:04<23:35,  8.74s/it]
 81%|████████▏ | 709/870 [1:28:12<22:38,  8.44s/it]
 82%|████████▏ | 710/870 [1:28:20<21:56,  8.23s/it]
                                                   
{'loss': 0.0183, 'grad_norm': 0.17799951136112213, 'learning_rate': 1.746619853524991e-05, 'epoch': 0.82}

 82%|████████▏ | 710/870 [1:28:20<21:56,  8.23s/it]
 82%|████████▏ | 711/870 [1:28:27<21:28,  8.11s/it]
 82%|████████▏ | 712/870 [1:28:35<20:45,  7.88s/it]
 82%|████████▏ | 713/870 [1:28:42<20:01,  7.65s/it]
 82%|████████▏ | 714/870 [1:28:49<19:20,  7.44s/it]
 82%|████████▏ | 715/870 [1:28:56<18:37,  7.21s/it]
 82%|████████▏ | 716/870 [1:29:02<18:17,  7.12s/it]
 82%|████████▏ | 717/870 [1:29:09<17:27,  6.84s/it]
 83%|████████▎ | 718/870 [1:29:15<16:50,  6.65s/it]
 83%|████████▎ | 719/870 [1:29:21<16:15,  6.46s/it]
 83%|████████▎ | 720/870 [1:29:27<15:59,  6.40s/it]
                                                   
{'loss': 0.0125, 'grad_norm': 0.1625465303659439, 'learning_rate': 1.5419761362593332e-05, 'epoch': 0.83}

 83%|████████▎ | 720/870 [1:29:27<15:59,  6.40s/it]
 83%|████████▎ | 721/870 [1:29:33<15:22,  6.19s/it]
 83%|████████▎ | 722/870 [1:29:38<14:46,  5.99s/it]
 83%|████████▎ | 723/870 [1:29:44<14:05,  5.75s/it]
 83%|████████▎ | 724/870 [1:29:48<13:07,  5.39s/it]
 83%|████████▎ | 725/870 [1:29:52<11:38,  4.82s/it]
 83%|████████▎ | 726/870 [1:30:04<16:53,  7.04s/it]
 84%|████████▎ | 727/870 [1:30:14<19:02,  7.99s/it]
 84%|████████▎ | 728/870 [1:30:23<19:58,  8.44s/it]
 84%|████████▍ | 729/870 [1:30:33<20:23,  8.68s/it]
 84%|████████▍ | 730/870 [1:30:41<20:18,  8.71s/it]
                                                   
{'loss': 0.0111, 'grad_norm': 0.04061046242713928, 'learning_rate': 1.3490776953223105e-05, 'epoch': 0.84}

 84%|████████▍ | 730/870 [1:30:41<20:18,  8.71s/it]
 84%|████████▍ | 731/870 [1:30:50<19:56,  8.61s/it]
 84%|████████▍ | 732/870 [1:30:58<19:24,  8.44s/it]
 84%|████████▍ | 733/870 [1:31:06<18:50,  8.25s/it]
 84%|████████▍ | 734/870 [1:31:14<18:31,  8.18s/it]
 84%|████████▍ | 735/870 [1:31:21<18:03,  8.02s/it]
 85%|████████▍ | 736/870 [1:31:29<17:35,  7.88s/it]
 85%|████████▍ | 737/870 [1:31:36<16:54,  7.63s/it]
 85%|████████▍ | 738/870 [1:31:43<16:26,  7.48s/it]
 85%|████████▍ | 739/870 [1:31:50<16:04,  7.36s/it]
 85%|████████▌ | 740/870 [1:31:57<15:30,  7.16s/it]
                                                   
{'loss': 0.0102, 'grad_norm': 0.026591680943965912, 'learning_rate': 1.1681924000801791e-05, 'epoch': 0.85}

 85%|████████▌ | 740/870 [1:31:57<15:30,  7.16s/it]
 85%|████████▌ | 741/870 [1:32:03<14:58,  6.97s/it]
 85%|████████▌ | 742/870 [1:32:10<14:21,  6.73s/it]
 85%|████████▌ | 743/870 [1:32:16<13:55,  6.58s/it]
 86%|████████▌ | 744/870 [1:32:22<13:39,  6.50s/it]
 86%|████████▌ | 745/870 [1:32:28<13:12,  6.34s/it]
 86%|████████▌ | 746/870 [1:32:34<12:41,  6.14s/it]
 86%|████████▌ | 747/870 [1:32:39<12:12,  5.95s/it]
 86%|████████▌ | 748/870 [1:32:45<11:39,  5.73s/it]
 86%|████████▌ | 749/870 [1:32:49<11:03,  5.49s/it]
 86%|████████▌ | 750/870 [1:32:53<09:33,  4.78s/it]
                                                   
{'loss': 0.0098, 'grad_norm': 0.04456641525030136, 'learning_rate': 9.995714377854936e-06, 'epoch': 0.86}

 86%|████████▌ | 750/870 [1:32:53<09:33,  4.78s/it]
 86%|████████▋ | 751/870 [1:33:04<13:34,  6.85s/it]
 86%|████████▋ | 752/870 [1:33:14<15:27,  7.86s/it]
 87%|████████▋ | 753/870 [1:33:24<16:06,  8.26s/it]
 87%|████████▋ | 754/870 [1:33:33<16:33,  8.56s/it]
 87%|████████▋ | 755/870 [1:33:42<16:28,  8.60s/it]
 87%|████████▋ | 756/870 [1:33:50<16:13,  8.54s/it]
 87%|████████▋ | 757/870 [1:33:58<15:54,  8.45s/it]
 87%|████████▋ | 758/870 [1:34:06<15:38,  8.38s/it]
 87%|████████▋ | 759/870 [1:34:14<15:09,  8.19s/it]
 87%|████████▋ | 760/870 [1:34:22<14:35,  7.96s/it]
                                                   
{'loss': 0.0162, 'grad_norm': 0.19221937656402588, 'learning_rate': 8.434489647647092e-06, 'epoch': 0.87}

 87%|████████▋ | 760/870 [1:34:22<14:35,  7.96s/it]
 87%|████████▋ | 761/870 [1:34:29<14:05,  7.76s/it]
 88%|████████▊ | 762/870 [1:34:36<13:42,  7.62s/it]
 88%|████████▊ | 763/870 [1:34:43<13:19,  7.47s/it]
 88%|████████▊ | 764/870 [1:34:50<12:50,  7.27s/it]
 88%|████████▊ | 765/870 [1:34:57<12:29,  7.13s/it]
 88%|████████▊ | 766/870 [1:35:04<12:07,  6.99s/it]
 88%|████████▊ | 767/870 [1:35:10<11:51,  6.91s/it]
 88%|████████▊ | 768/870 [1:35:17<11:23,  6.71s/it]
 88%|████████▊ | 769/870 [1:35:23<10:56,  6.50s/it]
 89%|████████▊ | 770/870 [1:35:28<10:31,  6.32s/it]
                                                   
{'loss': 0.0099, 'grad_norm': 0.04924648255109787, 'learning_rate': 7.000417812558302e-06, 'epoch': 0.89}

 89%|████████▊ | 770/870 [1:35:28<10:31,  6.32s/it]
 89%|████████▊ | 771/870 [1:35:34<10:06,  6.13s/it]
 89%|████████▊ | 772/870 [1:35:40<09:52,  6.04s/it]
 89%|████████▉ | 773/870 [1:35:45<09:21,  5.79s/it]
 89%|████████▉ | 774/870 [1:35:50<08:34,  5.36s/it]
 89%|████████▉ | 775/870 [1:35:53<07:23,  4.67s/it]
 89%|████████▉ | 776/870 [1:36:05<11:01,  7.04s/it]
 89%|████████▉ | 777/870 [1:36:16<12:26,  8.03s/it]
 89%|████████▉ | 778/870 [1:36:25<12:59,  8.48s/it]
 90%|████████▉ | 779/870 [1:36:34<13:11,  8.70s/it]
 90%|████████▉ | 780/870 [1:36:43<12:58,  8.65s/it]
                                                   
{'loss': 0.0127, 'grad_norm': 0.09919004142284393, 'learning_rate': 5.695490303477124e-06, 'epoch': 0.9}

 90%|████████▉ | 780/870 [1:36:43<12:58,  8.65s/it]
 90%|████████▉ | 781/870 [1:36:51<12:48,  8.64s/it]
 90%|████████▉ | 782/870 [1:36:59<12:24,  8.46s/it]
 90%|█████████ | 783/870 [1:37:07<11:57,  8.24s/it]
 90%|█████████ | 784/870 [1:37:15<11:39,  8.14s/it]
 90%|█████████ | 785/870 [1:37:23<11:20,  8.00s/it]
 90%|█████████ | 786/870 [1:37:31<11:07,  7.95s/it]
 90%|█████████ | 787/870 [1:37:38<10:37,  7.68s/it]
 91%|█████████ | 788/870 [1:37:45<10:12,  7.47s/it]
 91%|█████████ | 789/870 [1:37:51<09:51,  7.30s/it]
 91%|█████████ | 790/870 [1:37:58<09:30,  7.13s/it]
                                                   
{'loss': 0.0111, 'grad_norm': 0.04783658683300018, 'learning_rate': 4.521519214390257e-06, 'epoch': 0.91}

 91%|█████████ | 790/870 [1:37:58<09:30,  7.13s/it]
 91%|█████████ | 791/870 [1:38:05<09:18,  7.07s/it]
 91%|█████████ | 792/870 [1:38:11<08:53,  6.84s/it]
 91%|█████████ | 793/870 [1:38:18<08:28,  6.61s/it]
 91%|█████████▏| 794/870 [1:38:24<08:08,  6.43s/it]
 91%|█████████▏| 795/870 [1:38:29<07:46,  6.23s/it]
 91%|█████████▏| 796/870 [1:38:35<07:35,  6.16s/it]
 92%|█████████▏| 797/870 [1:38:41<07:15,  5.97s/it]
 92%|█████████▏| 798/870 [1:38:46<06:53,  5.74s/it]
 92%|█████████▏| 799/870 [1:38:51<06:27,  5.46s/it]
 92%|█████████▏| 800/870 [1:38:54<05:36,  4.80s/it]
                                                   
{'loss': 0.0094, 'grad_norm': 0.11320256441831589, 'learning_rate': 3.4801347860093924e-06, 'epoch': 0.92}

 92%|█████████▏| 800/870 [1:38:54<05:36,  4.80s/it]

  0%|          | 0/38 [00:00<?, ?it/s]

  5%|▌         | 2/38 [00:00<00:13,  2.71it/s]

  8%|▊         | 3/38 [00:01<00:15,  2.26it/s]

 11%|█         | 4/38 [00:01<00:15,  2.16it/s]

 13%|█▎        | 5/38 [00:02<00:15,  2.07it/s]

 16%|█▌        | 6/38 [00:02<00:15,  2.12it/s]

 18%|█▊        | 7/38 [00:03<00:14,  2.19it/s]

 21%|██        | 8/38 [00:03<00:13,  2.29it/s]

 24%|██▎       | 9/38 [00:03<00:12,  2.41it/s]

 26%|██▋       | 10/38 [00:04<00:13,  2.08it/s]

 29%|██▉       | 11/38 [00:05<00:14,  1.86it/s]

 32%|███▏      | 12/38 [00:05<00:14,  1.83it/s]

 34%|███▍      | 13/38 [00:06<00:13,  1.83it/s]

 37%|███▋      | 14/38 [00:06<00:12,  1.87it/s]

 39%|███▉      | 15/38 [00:07<00:11,  1.94it/s]

 42%|████▏     | 16/38 [00:07<00:11,  2.00it/s]

 45%|████▍     | 17/38 [00:08<00:09,  2.12it/s]

 47%|████▋     | 18/38 [00:08<00:08,  2.32it/s]

 50%|█████     | 19/38 [00:09<00:09,  2.10it/s]

 53%|█████▎    | 20/38 [00:09<00:09,  1.90it/s]

 55%|█████▌    | 21/38 [00:10<00:09,  1.87it/s]

 58%|█████▊    | 22/38 [00:10<00:08,  1.83it/s]

 61%|██████    | 23/38 [00:11<00:07,  1.88it/s]

 63%|██████▎   | 24/38 [00:11<00:07,  1.92it/s]

 66%|██████▌   | 25/38 [00:12<00:06,  1.98it/s]

 68%|██████▊   | 26/38 [00:12<00:05,  2.11it/s]

 71%|███████   | 27/38 [00:13<00:04,  2.26it/s]

 74%|███████▎  | 28/38 [00:13<00:04,  2.01it/s]

 76%|███████▋  | 29/38 [00:14<00:04,  1.81it/s]

 79%|███████▉  | 30/38 [00:14<00:04,  1.79it/s]

 82%|████████▏ | 31/38 [00:15<00:03,  1.80it/s]

 84%|████████▍ | 32/38 [00:16<00:03,  1.84it/s]

 87%|████████▋ | 33/38 [00:16<00:02,  1.90it/s]

 89%|████████▉ | 34/38 [00:16<00:02,  1.99it/s]

 92%|█████████▏| 35/38 [00:17<00:01,  2.13it/s]

 95%|█████████▍| 36/38 [00:17<00:00,  2.30it/s]

 97%|█████████▋| 37/38 [00:18<00:00,  2.23it/s]

100%|██████████| 38/38 [00:18<00:00,  2.17it/s]
                                                   

                                               
{'eval_loss': 0.013403651304543018, 'eval_runtime': 20.315, 'eval_samples_per_second': 14.767, 'eval_steps_per_second': 1.871, 'epoch': 0.92}

 92%|█████████▏| 800/870 [1:39:14<05:36,  4.80s/it]

100%|██████████| 38/38 [00:19<00:00,  2.17it/s]

                                               
 92%|█████████▏| 801/870 [1:39:28<15:33, 13.53s/it]
 92%|█████████▏| 802/870 [1:39:38<14:12, 12.54s/it]
 92%|█████████▏| 803/870 [1:39:48<13:01, 11.66s/it]
 92%|█████████▏| 804/870 [1:39:57<12:03, 10.96s/it]
 93%|█████████▎| 805/870 [1:40:06<11:15, 10.40s/it]
 93%|█████████▎| 806/870 [1:40:15<10:27,  9.80s/it]
 93%|█████████▎| 807/870 [1:40:23<09:48,  9.34s/it]
 93%|█████████▎| 808/870 [1:40:31<09:12,  8.91s/it]
 93%|█████████▎| 809/870 [1:40:39<08:41,  8.55s/it]
 93%|█████████▎| 810/870 [1:40:46<08:20,  8.35s/it]
                                                   
{'loss': 0.016, 'grad_norm': 0.32974982261657715, 'learning_rate': 2.572783141929469e-06, 'epoch': 0.93}

 93%|█████████▎| 810/870 [1:40:46<08:20,  8.35s/it]
 93%|█████████▎| 811/870 [1:40:54<07:55,  8.05s/it]
 93%|█████████▎| 812/870 [1:41:01<07:31,  7.79s/it]
 93%|█████████▎| 813/870 [1:41:08<07:12,  7.58s/it]
 94%|█████████▎| 814/870 [1:41:15<06:53,  7.38s/it]
 94%|█████████▎| 815/870 [1:41:22<06:36,  7.21s/it]
 94%|█████████▍| 816/870 [1:41:28<06:18,  7.01s/it]
 94%|█████████▍| 817/870 [1:41:34<05:57,  6.75s/it]
 94%|█████████▍| 818/870 [1:41:41<05:41,  6.57s/it]
 94%|█████████▍| 819/870 [1:41:47<05:26,  6.39s/it]
 94%|█████████▍| 820/870 [1:41:53<05:14,  6.28s/it]
                                                   
{'loss': 0.0095, 'grad_norm': 0.24235671758651733, 'learning_rate': 1.8007242804619628e-06, 'epoch': 0.94}

 94%|█████████▍| 820/870 [1:41:53<05:14,  6.28s/it]
 94%|█████████▍| 821/870 [1:41:58<04:59,  6.11s/it]
 94%|█████████▍| 822/870 [1:42:04<04:44,  5.93s/it]
 95%|█████████▍| 823/870 [1:42:09<04:26,  5.68s/it]
 95%|█████████▍| 824/870 [1:42:14<04:09,  5.42s/it]
 95%|█████████▍| 825/870 [1:42:17<03:31,  4.70s/it]
 95%|█████████▍| 826/870 [1:42:28<04:59,  6.81s/it]
 95%|█████████▌| 827/870 [1:42:38<05:34,  7.77s/it]
 95%|█████████▌| 828/870 [1:42:48<05:47,  8.27s/it]
 95%|█████████▌| 829/870 [1:42:57<05:53,  8.63s/it]
 95%|█████████▌| 830/870 [1:43:06<05:46,  8.66s/it]
                                                   
{'loss': 0.01, 'grad_norm': 0.08694616705179214, 'learning_rate': 1.165030324932137e-06, 'epoch': 0.95}

 95%|█████████▌| 830/870 [1:43:06<05:46,  8.66s/it]
 96%|█████████▌| 831/870 [1:43:15<05:35,  8.61s/it]
 96%|█████████▌| 832/870 [1:43:23<05:23,  8.51s/it]
 96%|█████████▌| 833/870 [1:43:31<05:10,  8.38s/it]
 96%|█████████▌| 834/870 [1:43:39<04:58,  8.30s/it]
 96%|█████████▌| 835/870 [1:43:47<04:44,  8.12s/it]
 96%|█████████▌| 836/870 [1:43:54<04:30,  7.94s/it]
 96%|█████████▌| 837/870 [1:44:01<04:13,  7.69s/it]
 96%|█████████▋| 838/870 [1:44:09<04:00,  7.52s/it]
 96%|█████████▋| 839/870 [1:44:16<03:50,  7.43s/it]
 97%|█████████▋| 840/870 [1:44:22<03:36,  7.21s/it]
                                                   
{'loss': 0.0113, 'grad_norm': 0.09477345645427704, 'learning_rate': 6.665840348696994e-07, 'epoch': 0.97}

 97%|█████████▋| 840/870 [1:44:22<03:36,  7.21s/it]
 97%|█████████▋| 841/870 [1:44:29<03:22,  6.98s/it]
 97%|█████████▋| 842/870 [1:44:35<03:09,  6.77s/it]
 97%|█████████▋| 843/870 [1:44:42<03:00,  6.68s/it]
 97%|█████████▋| 844/870 [1:44:48<02:48,  6.48s/it]
 97%|█████████▋| 845/870 [1:44:54<02:37,  6.30s/it]
 97%|█████████▋| 846/870 [1:44:59<02:26,  6.11s/it]
 97%|█████████▋| 847/870 [1:45:05<02:15,  5.91s/it]
 97%|█████████▋| 848/870 [1:45:10<02:07,  5.80s/it]
 98%|█████████▊| 849/870 [1:45:15<01:54,  5.45s/it]
 98%|█████████▊| 850/870 [1:45:18<01:35,  4.78s/it]
                                                   
{'loss': 0.0086, 'grad_norm': 0.048642598092556, 'learning_rate': 3.060775801604354e-07, 'epoch': 0.98}

 98%|█████████▊| 850/870 [1:45:18<01:35,  4.78s/it]
 98%|█████████▊| 851/870 [1:45:29<02:08,  6.75s/it]
 98%|█████████▊| 852/870 [1:45:39<02:16,  7.59s/it]
 98%|█████████▊| 853/870 [1:45:49<02:19,  8.20s/it]
 98%|█████████▊| 854/870 [1:45:57<02:12,  8.29s/it]
 98%|█████████▊| 855/870 [1:46:05<02:04,  8.28s/it]
 98%|█████████▊| 856/870 [1:46:13<01:54,  8.19s/it]
 99%|█████████▊| 857/870 [1:46:21<01:45,  8.09s/it]
 99%|█████████▊| 858/870 [1:46:29<01:36,  8.03s/it]
 99%|█████████▊| 859/870 [1:46:36<01:26,  7.83s/it]
 99%|█████████▉| 860/870 [1:46:43<01:16,  7.62s/it]
                                                   
{'loss': 0.0148, 'grad_norm': 0.06935633718967438, 'learning_rate': 8.401157986109853e-08, 'epoch': 0.99}

 99%|█████████▉| 860/870 [1:46:43<01:16,  7.62s/it]
 99%|█████████▉| 861/870 [1:46:50<01:06,  7.37s/it]
 99%|█████████▉| 862/870 [1:46:57<00:57,  7.23s/it]
 99%|█████████▉| 863/870 [1:47:04<00:48,  6.99s/it]
 99%|█████████▉| 864/870 [1:47:10<00:40,  6.75s/it]
 99%|█████████▉| 865/870 [1:47:16<00:32,  6.53s/it]
100%|█████████▉| 866/870 [1:47:22<00:25,  6.34s/it]
100%|█████████▉| 867/870 [1:47:28<00:18,  6.23s/it]
100%|█████████▉| 868/870 [1:47:33<00:11,  5.96s/it]
100%|█████████▉| 869/870 [1:47:38<00:05,  5.62s/it]
100%|██████████| 870/870 [1:47:46<00:00,  6.37s/it]
                                                   
{'loss': 0.0087, 'grad_norm': 0.06433246284723282, 'learning_rate': 6.944070123582336e-10, 'epoch': 1.0}

100%|██████████| 870/870 [1:47:46<00:00,  6.37s/it]
                                                   
{'train_runtime': 6469.3899, 'train_samples_per_second': 4.303, 'train_steps_per_second': 0.134, 'train_loss': 0.024751038421159503, 'epoch': 1.0}

100%|██████████| 870/870 [1:47:47<00:00,  6.37s/it]
100%|██████████| 870/870 [1:47:47<00:00,  7.43s/it]
[rank0]:[W1024 16:20:21.677824557 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())