File size: 369,655 Bytes
5a15826 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 | === full run start Mon Sep 14 16:44:32 UTC 2026 ===
[2026-09-14 16:44:37,556] [WARNING] [runner.py:232:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2026-09-14 16:44:37,556] [INFO] [runner.py:630:main] cmd = /opt/conda/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMV19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None --log_level=info scripts/training/train.py --model_config configs/model/model_300m.yaml --tokenizer_path tokenizer --train_data_dir /workspace/data/pretrain/train --val_data_dir /workspace/data/pretrain/val_small --output_dir outputs/devops-300m-4096-bf16-vast --max_steps 4000 --warmup_steps 40 --learning_rate 6e-4 --batch_size 1 --grad_accum 16 --logging_steps 10 --save_steps 2000 --eval_steps 2000 --deepspeed_config configs/training/deepspeed_zero2_bf16.json --bf16 --seq_length 4096 --run_name devops-300m-4096-bf16-vast
[2026-09-14 16:44:42,440] [INFO] [launch.py:155:main] 0 NCCL_DEBUG=INFO
[2026-09-14 16:44:42,440] [INFO] [launch.py:162:main] WORLD INFO DICT: {'localhost': [0, 1]}
[2026-09-14 16:44:42,440] [INFO] [launch.py:168:main] nnodes=1, num_local_procs=2, node_rank=0
[2026-09-14 16:44:42,440] [INFO] [launch.py:179:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1]})
[2026-09-14 16:44:42,440] [INFO] [launch.py:180:main] dist_world_size=2
[2026-09-14 16:44:42,440] [INFO] [launch.py:184:main] Setting CUDA_VISIBLE_DEVICES=0,1
[2026-09-14 16:44:42,441] [INFO] [launch.py:272:main] process 10692 spawned with command: ['/opt/conda/bin/python', '-u', 'scripts/training/train.py', '--local_rank=0', '--model_config', 'configs/model/model_300m.yaml', '--tokenizer_path', 'tokenizer', '--train_data_dir', '/workspace/data/pretrain/train', '--val_data_dir', '/workspace/data/pretrain/val_small', '--output_dir', 'outputs/devops-300m-4096-bf16-vast', '--max_steps', '4000', '--warmup_steps', '40', '--learning_rate', '6e-4', '--batch_size', '1', '--grad_accum', '16', '--logging_steps', '10', '--save_steps', '2000', '--eval_steps', '2000', '--deepspeed_config', 'configs/training/deepspeed_zero2_bf16.json', '--bf16', '--seq_length', '4096', '--run_name', 'devops-300m-4096-bf16-vast']
[2026-09-14 16:44:42,441] [INFO] [launch.py:272:main] process 10693 spawned with command: ['/opt/conda/bin/python', '-u', 'scripts/training/train.py', '--local_rank=1', '--model_config', 'configs/model/model_300m.yaml', '--tokenizer_path', 'tokenizer', '--train_data_dir', '/workspace/data/pretrain/train', '--val_data_dir', '/workspace/data/pretrain/val_small', '--output_dir', 'outputs/devops-300m-4096-bf16-vast', '--max_steps', '4000', '--warmup_steps', '40', '--learning_rate', '6e-4', '--batch_size', '1', '--grad_accum', '16', '--logging_steps', '10', '--save_steps', '2000', '--eval_steps', '2000', '--deepspeed_config', 'configs/training/deepspeed_zero2_bf16.json', '--bf16', '--seq_length', '4096', '--run_name', 'devops-300m-4096-bf16-vast']
[Tokenizer] vocab_size=65536
[Tokenizer] vocab_size=65536
[Dataset] 1509 files, 132,068 non-overlapping samples, seq_len=4096
[Dataset] 10 files, 22 non-overlapping samples, seq_len=4096
[Dataset] 1509 files, 132,068 non-overlapping samples, seq_len=4096
[Dataset] 10 files, 22 non-overlapping samples, seq_len=4096
[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
- If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
- If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
- If you are not the owner of the model architecture class, please contact the model code owner to update it.
[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
- If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
- If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
- If you are not the owner of the model architecture class, please contact the model code owner to update it.
[Model] 287.3M parameters
[Env] 2 GPUs detected, using DeepSpeed (launch with: deepspeed --master_port 29500 train.py ...)
[Model] 287.3M parameters
[Env] 2 GPUs detected, using DeepSpeed (launch with: deepspeed --master_port 29500 train.py ...)
81d666a3c511:10692:10692 [0] NCCL INFO Bootstrap : Using eth0:172.17.0.4<0>
81d666a3c511:10692:10692 [0] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
81d666a3c511:10692:10692 [0] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
81d666a3c511:10692:10692 [0] NCCL INFO NET/Plugin: Using internal network plugin.
81d666a3c511:10692:10692 [0] NCCL INFO cudaDriverVersion 13020
NCCL version 2.21.5+cuda12.4
81d666a3c511:10693:10693 [1] NCCL INFO cudaDriverVersion 13020
81d666a3c511:10693:10693 [1] NCCL INFO Bootstrap : Using eth0:172.17.0.4<0>
81d666a3c511:10693:10693 [1] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
81d666a3c511:10693:10693 [1] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
81d666a3c511:10693:10693 [1] NCCL INFO NET/Plugin: Using internal network plugin.
81d666a3c511:10693:10962 [1] NCCL INFO Failed to open libibverbs.so[.1]
81d666a3c511:10693:10962 [1] NCCL INFO NET/Socket : Using [0]eth0:172.17.0.4<0>
81d666a3c511:10693:10962 [1] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10693:10962 [1] NCCL INFO Using network Socket
81d666a3c511:10692:10964 [0] NCCL INFO Failed to open libibverbs.so[.1]
81d666a3c511:10692:10964 [0] NCCL INFO NET/Socket : Using [0]eth0:172.17.0.4<0>
81d666a3c511:10692:10964 [0] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10692:10964 [0] NCCL INFO Using network Socket
81d666a3c511:10692:10964 [0] NCCL INFO ncclCommInitRank comm 0x1adc3b90 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 commId 0x86d35acf72d855e7 - Init START
81d666a3c511:10693:10962 [1] NCCL INFO ncclCommInitRank comm 0x1aa33b30 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 commId 0x86d35acf72d855e7 - Init START
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10692:10964 [0] NCCL INFO comm 0x1adc3b90 rank 0 nRanks 2 nNodes 1 localRanks 2 localRank 0 MNNVL 0
81d666a3c511:10693:10962 [1] NCCL INFO comm 0x1aa33b30 rank 1 nRanks 2 nNodes 1 localRanks 2 localRank 1 MNNVL 0
81d666a3c511:10692:10964 [0] NCCL INFO Channel 00/02 : 0 1
81d666a3c511:10692:10964 [0] NCCL INFO Channel 01/02 : 0 1
81d666a3c511:10692:10964 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1
81d666a3c511:10692:10964 [0] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10693:10962 [1] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0
81d666a3c511:10693:10962 [1] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10692:10964 [0] NCCL INFO Channel 00 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10692:10964 [0] NCCL INFO Channel 01 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10693:10962 [1] NCCL INFO Channel 00 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10693:10962 [1] NCCL INFO Channel 01 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10692:10964 [0] NCCL INFO Connected all rings
81d666a3c511:10692:10964 [0] NCCL INFO Connected all trees
81d666a3c511:10693:10962 [1] NCCL INFO Connected all rings
81d666a3c511:10693:10962 [1] NCCL INFO Connected all trees
81d666a3c511:10693:10962 [1] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10693:10962 [1] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:10964 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10692:10964 [0] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:10964 [0] NCCL INFO TUNER/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
81d666a3c511:10693:10962 [1] NCCL INFO TUNER/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
81d666a3c511:10692:10964 [0] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
81d666a3c511:10693:10962 [1] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
81d666a3c511:10692:10964 [0] NCCL INFO ncclCommInitRank comm 0x1adc3b90 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 commId 0x86d35acf72d855e7 - Init COMPLETE
81d666a3c511:10693:10962 [1] NCCL INFO ncclCommInitRank comm 0x1aa33b30 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 commId 0x86d35acf72d855e7 - Init COMPLETE
[Train] Starting: 4000 steps, batch=1×16×world_size, lr=0.0006, seq_len=4096
[Train] Starting: 4000 steps, batch=1×16×world_size, lr=0.0006, seq_len=4096
[RANK 0] Gradient accumulation steps mismatch: GradientAccumulationPlugin has 1, DeepSpeed config has 16. Using DeepSpeed's value.
81d666a3c511:10693:11106 [1] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10693:11106 [1] NCCL INFO Using network Socket
81d666a3c511:10692:11110 [0] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10692:11110 [0] NCCL INFO Using network Socket
81d666a3c511:10693:11106 [1] NCCL INFO bootstrapSplit: comm 0xc798460 parent 0x1aa33b30 rank 1 nranks 2 color 1530306504 key 1 prev 0 next 0 - DONE
81d666a3c511:10693:11106 [1] NCCL INFO ncclCommSplit comm 0xc798460 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 parent 0x1aa33b30 color 1530306504 key 1 commId 0xbc296a4dcdede661 - Init START
81d666a3c511:10692:11110 [0] NCCL INFO bootstrapSplit: comm 0xcf26230 parent 0x1adc3b90 rank 0 nranks 2 color 1530306504 key 0 prev 1 next 1 - DONE
81d666a3c511:10692:11110 [0] NCCL INFO ncclCommSplit comm 0xcf26230 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 parent 0x1adc3b90 color 1530306504 key 0 commId 0xbc296a4dcdede661 - Init START
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10693:11106 [1] NCCL INFO comm 0xc798460 rank 1 nRanks 2 nNodes 1 localRanks 2 localRank 1 MNNVL 0
81d666a3c511:10692:11110 [0] NCCL INFO comm 0xcf26230 rank 0 nRanks 2 nNodes 1 localRanks 2 localRank 0 MNNVL 0
81d666a3c511:10693:11106 [1] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0
81d666a3c511:10693:11106 [1] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10692:11110 [0] NCCL INFO Channel 00/02 : 0 1
81d666a3c511:10692:11110 [0] NCCL INFO Channel 01/02 : 0 1
81d666a3c511:10692:11110 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1
81d666a3c511:10692:11110 [0] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10693:11106 [1] NCCL INFO Channel 00 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10693:11106 [1] NCCL INFO Channel 01 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10692:11110 [0] NCCL INFO Channel 00 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10692:11110 [0] NCCL INFO Channel 01 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10693:11106 [1] NCCL INFO Connected all rings
81d666a3c511:10693:11106 [1] NCCL INFO Connected all trees
81d666a3c511:10693:11106 [1] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10693:11106 [1] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:11110 [0] NCCL INFO Connected all rings
81d666a3c511:10692:11110 [0] NCCL INFO Connected all trees
81d666a3c511:10692:11110 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10692:11110 [0] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:11110 [0] NCCL INFO ncclCommSplit comm 0xcf26230 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 parent 0x1adc3b90 color 1530306504 key 0 commId 0xbc296a4dcdede661 - Init COMPLETE
81d666a3c511:10693:11106 [1] NCCL INFO ncclCommSplit comm 0xc798460 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 parent 0x1aa33b30 color 1530306504 key 1 commId 0xbc296a4dcdede661 - Init COMPLETE
[2026-09-14 16:44:56,871] [WARNING] [lr_schedules.py:693:get_lr] Attempting to get learning rate from scheduler before it has started
[2026-09-14 16:44:57,091] [WARNING] [lr_schedules.py:693:get_lr] Attempting to get learning rate from scheduler before it has started
0%| | 0/4000 [00:00<?, ?it/s][transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
0%| | 1/4000 [00:05<5:36:20, 5.05s/it]
0%| | 2/4000 [00:09<5:01:22, 4.52s/it]
0%| | 3/4000 [00:13<4:49:48, 4.35s/it]
0%| | 4/4000 [00:17<4:43:51, 4.26s/it]
0%| | 5/4000 [00:21<4:40:02, 4.21s/it]
0%| | 6/4000 [00:25<4:37:30, 4.17s/it]
0%| | 7/4000 [00:30<4:45:38, 4.29s/it]
0%| | 8/4000 [00:34<4:41:01, 4.22s/it]
0%| | 9/4000 [00:38<4:37:14, 4.17s/it]
0%| | 10/4000 [00:42<4:34:56, 4.13s/it]
{'loss': '9.602', 'grad_norm': '2.286', 'learning_rate': '0.0003745', 'epoch': '0.002423'}
0%| | 10/4000 [00:42<4:34:56, 4.13s/it]
0%| | 11/4000 [00:46<4:33:45, 4.12s/it]
0%| | 12/4000 [00:50<4:33:06, 4.11s/it]
0%| | 13/4000 [00:54<4:36:32, 4.16s/it]
0%| | 14/4000 [00:59<4:38:38, 4.19s/it]
0%| | 15/4000 [01:03<4:36:01, 4.16s/it]
0%| | 16/4000 [01:07<4:34:29, 4.13s/it]
0%| | 17/4000 [01:11<4:33:13, 4.12s/it]
0%| | 18/4000 [01:15<4:32:44, 4.11s/it]
0%| | 19/4000 [01:19<4:32:15, 4.10s/it]
0%| | 20/4000 [01:24<4:39:36, 4.22s/it]
{'loss': '6.495', 'grad_norm': '0.9675', 'learning_rate': '0.0004873', 'epoch': '0.004846'}
0%| | 20/4000 [01:24<4:39:36, 4.22s/it]
1%| | 21/4000 [01:28<4:36:45, 4.17s/it]
1%| | 22/4000 [01:32<4:34:35, 4.14s/it]
1%| | 23/4000 [01:36<4:33:19, 4.12s/it]
1%| | 24/4000 [01:40<4:32:11, 4.11s/it]
1%| | 25/4000 [01:44<4:31:40, 4.10s/it]
1%| | 26/4000 [01:48<4:31:10, 4.09s/it]
1%| | 27/4000 [01:52<4:38:34, 4.21s/it]
1%| | 28/4000 [01:57<4:35:46, 4.17s/it]
1%| | 29/4000 [02:01<4:34:07, 4.14s/it]
1%| | 30/4000 [02:05<4:33:20, 4.13s/it]
{'loss': '5.982', 'grad_norm': '1.023', 'learning_rate': '0.0005532', 'epoch': '0.007269'}
1%| | 30/4000 [02:05<4:33:20, 4.13s/it]
1%| | 31/4000 [02:09<4:33:04, 4.13s/it]
1%| | 32/4000 [02:13<4:33:06, 4.13s/it]
1%| | 33/4000 [02:17<4:32:41, 4.12s/it]
1%| | 34/4000 [02:22<4:40:00, 4.24s/it]
1%| | 35/4000 [02:26<4:36:57, 4.19s/it]
1%| | 36/4000 [02:30<4:34:41, 4.16s/it]
1%| | 37/4000 [02:34<4:33:18, 4.14s/it]
1%| | 38/4000 [02:38<4:32:43, 4.13s/it]
1%| | 39/4000 [02:42<4:32:33, 4.13s/it]
1%| | 40/4000 [02:46<4:35:49, 4.18s/it]
{'loss': '5.683', 'grad_norm': '1.414', 'learning_rate': '0.0006', 'epoch': '0.009692'}
1%| | 40/4000 [02:46<4:35:49, 4.18s/it]
1%| | 41/4000 [02:51<4:38:33, 4.22s/it]
1%| | 42/4000 [02:55<4:37:00, 4.20s/it]
1%| | 43/4000 [02:59<4:35:33, 4.18s/it]
1%| | 44/4000 [03:03<4:33:54, 4.15s/it]
1%| | 45/4000 [03:07<4:32:41, 4.14s/it]
1%| | 46/4000 [03:11<4:32:04, 4.13s/it]
1%| | 47/4000 [03:16<4:39:07, 4.24s/it]
1%| | 48/4000 [03:20<4:36:54, 4.20s/it]
1%| | 49/4000 [03:24<4:35:00, 4.18s/it]
1%|▏ | 50/4000 [03:28<4:33:25, 4.15s/it]
{'loss': '5.258', 'grad_norm': '2.436', 'learning_rate': '0.0005986', 'epoch': '0.01211'}
1%|▏ | 50/4000 [03:28<4:33:25, 4.15s/it]
1%|▏ | 51/4000 [03:32<4:32:41, 4.14s/it]
1%|▏ | 52/4000 [03:36<4:31:52, 4.13s/it]
1%|▏ | 53/4000 [03:40<4:30:50, 4.12s/it]
1%|▏ | 54/4000 [03:45<4:37:37, 4.22s/it]
1%|▏ | 55/4000 [03:49<4:34:50, 4.18s/it]
1%|▏ | 56/4000 [03:53<4:32:55, 4.15s/it]
1%|▏ | 57/4000 [03:57<4:31:40, 4.13s/it]
1%|▏ | 58/4000 [04:01<4:30:43, 4.12s/it]
1%|▏ | 59/4000 [04:05<4:30:07, 4.11s/it]
2%|▏ | 60/4000 [04:09<4:29:39, 4.11s/it]
{'loss': '4.63', 'grad_norm': '1.147', 'learning_rate': '0.0005971', 'epoch': '0.01454'}
2%|▏ | 60/4000 [04:09<4:29:39, 4.11s/it]
2%|▏ | 61/4000 [04:14<4:36:15, 4.21s/it]
2%|▏ | 62/4000 [04:18<4:34:06, 4.18s/it]
2%|▏ | 63/4000 [04:22<4:32:23, 4.15s/it]
2%|▏ | 64/4000 [04:26<4:31:10, 4.13s/it]
2%|▏ | 65/4000 [04:30<4:30:13, 4.12s/it]
2%|▏ | 66/4000 [04:34<4:29:33, 4.11s/it]
2%|▏ | 67/4000 [04:39<4:32:41, 4.16s/it]
2%|▏ | 68/4000 [04:43<4:34:43, 4.19s/it]
2%|▏ | 69/4000 [04:47<4:32:46, 4.16s/it]
2%|▏ | 70/4000 [04:51<4:31:28, 4.14s/it]
{'loss': '3.956', 'grad_norm': '1.276', 'learning_rate': '0.0005956', 'epoch': '0.01696'}
2%|▏ | 70/4000 [04:51<4:31:28, 4.14s/it]
2%|▏ | 71/4000 [04:55<4:30:15, 4.13s/it]
2%|▏ | 72/4000 [04:59<4:29:14, 4.11s/it]
2%|▏ | 73/4000 [05:03<4:28:36, 4.10s/it]
2%|▏ | 74/4000 [05:08<4:35:56, 4.22s/it]
2%|▏ | 75/4000 [05:12<4:33:28, 4.18s/it]
2%|▏ | 76/4000 [05:16<4:31:42, 4.15s/it]
2%|▏ | 77/4000 [05:20<4:30:22, 4.14s/it]
2%|▏ | 78/4000 [05:24<4:29:19, 4.12s/it]
2%|▏ | 79/4000 [05:28<4:28:46, 4.11s/it]
2%|▏ | 80/4000 [05:32<4:28:17, 4.11s/it]
{'loss': '3.471', 'grad_norm': '0.7973', 'learning_rate': '0.0005941', 'epoch': '0.01938'}
2%|▏ | 80/4000 [05:32<4:28:17, 4.11s/it]
2%|▏ | 81/4000 [05:37<4:33:40, 4.19s/it]
2%|▏ | 82/4000 [05:41<4:31:34, 4.16s/it]
2%|▏ | 83/4000 [05:45<4:30:06, 4.14s/it]
2%|▏ | 84/4000 [05:49<4:29:04, 4.12s/it]
2%|▏ | 85/4000 [05:53<4:28:28, 4.11s/it]
2%|▏ | 86/4000 [05:57<4:28:15, 4.11s/it]
2%|▏ | 87/4000 [06:01<4:28:05, 4.11s/it]
2%|▏ | 88/4000 [06:06<4:35:19, 4.22s/it]
2%|▏ | 89/4000 [06:10<4:32:36, 4.18s/it]
2%|▏ | 90/4000 [06:14<4:31:02, 4.16s/it]
{'loss': '3.021', 'grad_norm': '0.7119', 'learning_rate': '0.0005926', 'epoch': '0.02181'}
2%|▏ | 90/4000 [06:14<4:31:02, 4.16s/it]
2%|▏ | 91/4000 [06:18<4:29:50, 4.14s/it]
2%|▏ | 92/4000 [06:22<4:28:49, 4.13s/it]
2%|▏ | 93/4000 [06:26<4:28:11, 4.12s/it]
2%|▏ | 94/4000 [06:31<4:33:26, 4.20s/it]
2%|▏ | 95/4000 [06:35<4:31:19, 4.17s/it]
2%|▏ | 96/4000 [06:39<4:29:39, 4.14s/it]
2%|▏ | 97/4000 [06:43<4:28:49, 4.13s/it]
2%|▏ | 98/4000 [06:47<4:27:56, 4.12s/it]
2%|▏ | 99/4000 [06:51<4:27:28, 4.11s/it]
2%|▎ | 100/4000 [06:55<4:26:55, 4.11s/it]
{'loss': '2.637', 'grad_norm': '0.6688', 'learning_rate': '0.0005911', 'epoch': '0.02423'}
2%|▎ | 100/4000 [06:55<4:26:55, 4.11s/it]
3%|▎ | 101/4000 [07:00<4:34:09, 4.22s/it]
3%|▎ | 102/4000 [07:04<4:31:45, 4.18s/it]
3%|▎ | 103/4000 [07:08<4:29:58, 4.16s/it]
3%|▎ | 104/4000 [07:12<4:28:31, 4.14s/it]
3%|▎ | 105/4000 [07:16<4:27:33, 4.12s/it]
3%|▎ | 106/4000 [07:20<4:26:52, 4.11s/it]
3%|▎ | 107/4000 [07:24<4:26:21, 4.11s/it]
3%|▎ | 108/4000 [07:29<4:32:45, 4.20s/it]
3%|▎ | 109/4000 [07:33<4:30:28, 4.17s/it]
3%|▎ | 110/4000 [07:37<4:28:50, 4.15s/it]
{'loss': '2.353', 'grad_norm': '0.5264', 'learning_rate': '0.0005895', 'epoch': '0.02665'}
3%|▎ | 110/4000 [07:37<4:28:50, 4.15s/it]
3%|▎ | 111/4000 [07:41<4:27:40, 4.13s/it]
3%|▎ | 112/4000 [07:45<4:26:59, 4.12s/it]
3%|▎ | 113/4000 [07:49<4:26:34, 4.11s/it]
3%|▎ | 114/4000 [07:53<4:26:06, 4.11s/it]
3%|▎ | 115/4000 [07:58<4:32:41, 4.21s/it]
3%|▎ | 116/4000 [08:02<4:30:24, 4.18s/it]
3%|▎ | 117/4000 [08:06<4:28:57, 4.16s/it]
3%|▎ | 118/4000 [08:10<4:27:45, 4.14s/it]
3%|▎ | 119/4000 [08:14<4:27:05, 4.13s/it]
3%|▎ | 120/4000 [08:18<4:26:31, 4.12s/it]
{'loss': '2.108', 'grad_norm': '0.4656', 'learning_rate': '0.000588', 'epoch': '0.02908'}
3%|▎ | 120/4000 [08:18<4:26:31, 4.12s/it]
3%|▎ | 121/4000 [08:23<4:33:00, 4.22s/it]
3%|▎ | 122/4000 [08:27<4:30:33, 4.19s/it]
3%|▎ | 123/4000 [08:31<4:28:35, 4.16s/it]
3%|▎ | 124/4000 [08:35<4:27:29, 4.14s/it]
3%|▎ | 125/4000 [08:39<4:26:33, 4.13s/it]
3%|▎ | 126/4000 [08:43<4:26:02, 4.12s/it]
3%|▎ | 127/4000 [08:47<4:25:25, 4.11s/it]
3%|▎ | 128/4000 [08:52<4:30:59, 4.20s/it]
3%|▎ | 129/4000 [08:56<4:28:58, 4.17s/it]
3%|▎ | 130/4000 [09:00<4:27:26, 4.15s/it]
{'loss': '1.974', 'grad_norm': '0.4384', 'learning_rate': '0.0005865', 'epoch': '0.0315'}
3%|▎ | 130/4000 [09:00<4:27:26, 4.15s/it]
3%|▎ | 131/4000 [09:04<4:26:40, 4.14s/it]
3%|▎ | 132/4000 [09:08<4:25:50, 4.12s/it]
3%|▎ | 133/4000 [09:12<4:25:21, 4.12s/it]
3%|▎ | 134/4000 [09:16<4:24:53, 4.11s/it]
3%|▎ | 135/4000 [09:21<4:31:25, 4.21s/it]
3%|▎ | 136/4000 [09:25<4:29:01, 4.18s/it]
3%|▎ | 137/4000 [09:29<4:27:19, 4.15s/it]
3%|▎ | 138/4000 [09:33<4:26:11, 4.14s/it]
3%|▎ | 139/4000 [09:37<4:25:21, 4.12s/it]
4%|▎ | 140/4000 [09:41<4:24:34, 4.11s/it]
{'loss': '1.871', 'grad_norm': '0.443', 'learning_rate': '0.000585', 'epoch': '0.03392'}
4%|▎ | 140/4000 [09:41<4:24:34, 4.11s/it]
4%|▎ | 141/4000 [09:45<4:27:53, 4.17s/it]
4%|▎ | 142/4000 [09:50<4:29:39, 4.19s/it]
4%|▎ | 143/4000 [09:54<4:27:37, 4.16s/it]
4%|▎ | 144/4000 [09:58<4:26:16, 4.14s/it]
4%|▎ | 145/4000 [10:02<4:25:30, 4.13s/it]
4%|▎ | 146/4000 [10:06<4:24:50, 4.12s/it]
4%|▎ | 147/4000 [10:10<4:24:09, 4.11s/it]
4%|▎ | 148/4000 [10:15<4:30:09, 4.21s/it]
4%|▎ | 149/4000 [10:19<4:27:54, 4.17s/it]
4%|▍ | 150/4000 [10:23<4:26:15, 4.15s/it]
{'loss': '1.745', 'grad_norm': '0.3832', 'learning_rate': '0.0005835', 'epoch': '0.03634'}
4%|▍ | 150/4000 [10:23<4:26:15, 4.15s/it]
4%|▍ | 151/4000 [10:27<4:25:09, 4.13s/it]
4%|▍ | 152/4000 [10:31<4:24:16, 4.12s/it]
4%|▍ | 153/4000 [10:35<4:23:44, 4.11s/it]
4%|▍ | 154/4000 [10:39<4:23:24, 4.11s/it]
4%|▍ | 155/4000 [10:44<4:29:38, 4.21s/it]
4%|▍ | 156/4000 [10:48<4:27:29, 4.18s/it]
4%|▍ | 157/4000 [10:52<4:26:01, 4.15s/it]
4%|▍ | 158/4000 [10:56<4:24:47, 4.14s/it]
4%|▍ | 159/4000 [11:00<4:24:03, 4.12s/it]
4%|▍ | 160/4000 [11:04<4:23:40, 4.12s/it]
{'loss': '1.724', 'grad_norm': '0.3886', 'learning_rate': '0.000582', 'epoch': '0.03877'}
4%|▍ | 160/4000 [11:04<4:23:40, 4.12s/it]
4%|▍ | 161/4000 [11:08<4:23:12, 4.11s/it]
4%|▍ | 162/4000 [11:13<4:28:48, 4.20s/it]
4%|▍ | 163/4000 [11:17<4:26:48, 4.17s/it]
4%|▍ | 164/4000 [11:21<4:25:17, 4.15s/it]
4%|▍ | 165/4000 [11:25<4:24:09, 4.13s/it]
4%|▍ | 166/4000 [11:29<4:23:26, 4.12s/it]
4%|▍ | 167/4000 [11:33<4:22:49, 4.11s/it]
4%|▍ | 168/4000 [11:37<4:25:55, 4.16s/it]
4%|▍ | 169/4000 [11:42<4:28:16, 4.20s/it]
4%|▍ | 170/4000 [11:46<4:26:00, 4.17s/it]
{'loss': '1.625', 'grad_norm': '0.5112', 'learning_rate': '0.0005805', 'epoch': '0.04119'}
4%|▍ | 170/4000 [11:46<4:26:00, 4.17s/it]
4%|▍ | 171/4000 [11:50<4:24:38, 4.15s/it]
4%|▍ | 172/4000 [11:54<4:23:38, 4.13s/it]
4%|▍ | 173/4000 [11:58<4:22:43, 4.12s/it]
4%|▍ | 174/4000 [12:02<4:22:21, 4.11s/it]
4%|▍ | 175/4000 [12:07<4:28:10, 4.21s/it]
4%|▍ | 176/4000 [12:11<4:26:01, 4.17s/it]
4%|▍ | 177/4000 [12:15<4:24:22, 4.15s/it]
4%|▍ | 178/4000 [12:19<4:23:30, 4.14s/it]
4%|▍ | 179/4000 [12:23<4:22:46, 4.13s/it]
4%|▍ | 180/4000 [12:27<4:22:07, 4.12s/it]
{'loss': '1.559', 'grad_norm': '0.3595', 'learning_rate': '0.0005789', 'epoch': '0.04361'}
4%|▍ | 180/4000 [12:27<4:22:07, 4.12s/it]
5%|▍ | 181/4000 [12:31<4:21:39, 4.11s/it]
5%|▍ | 182/4000 [12:36<4:27:47, 4.21s/it]
5%|▍ | 183/4000 [12:40<4:25:34, 4.17s/it]
5%|▍ | 184/4000 [12:44<4:24:02, 4.15s/it]
5%|▍ | 185/4000 [12:48<4:22:48, 4.13s/it]
5%|▍ | 186/4000 [12:52<4:22:00, 4.12s/it]
5%|▍ | 187/4000 [12:56<4:21:24, 4.11s/it]
5%|▍ | 188/4000 [13:00<4:21:00, 4.11s/it]
5%|▍ | 189/4000 [13:05<4:27:27, 4.21s/it]
5%|▍ | 190/4000 [13:09<4:25:15, 4.18s/it]
{'loss': '1.55', 'grad_norm': '0.2786', 'learning_rate': '0.0005774', 'epoch': '0.04604'}
5%|▍ | 190/4000 [13:09<4:25:15, 4.18s/it]
5%|▍ | 191/4000 [13:13<4:23:40, 4.15s/it]
5%|▍ | 192/4000 [13:17<4:22:23, 4.13s/it]
5%|▍ | 193/4000 [13:21<4:21:38, 4.12s/it]
5%|▍ | 194/4000 [13:25<4:21:07, 4.12s/it]
5%|▍ | 195/4000 [13:29<4:23:39, 4.16s/it]
5%|▍ | 196/4000 [13:34<4:25:30, 4.19s/it]
5%|▍ | 197/4000 [13:38<4:23:39, 4.16s/it]
5%|▍ | 198/4000 [13:42<4:22:29, 4.14s/it]
5%|▍ | 199/4000 [13:46<4:21:33, 4.13s/it]
5%|▌ | 200/4000 [13:50<4:21:04, 4.12s/it]
{'loss': '1.418', 'grad_norm': '0.3144', 'learning_rate': '0.0005759', 'epoch': '0.04846'}
5%|▌ | 200/4000 [13:50<4:21:04, 4.12s/it]
5%|▌ | 201/4000 [13:54<4:20:37, 4.12s/it]
5%|▌ | 202/4000 [13:59<4:27:28, 4.23s/it]
5%|▌ | 203/4000 [14:03<4:25:03, 4.19s/it]
5%|▌ | 204/4000 [14:07<4:23:04, 4.16s/it]
5%|▌ | 205/4000 [14:11<4:21:50, 4.14s/it]
5%|▌ | 206/4000 [14:15<4:20:49, 4.12s/it]
5%|▌ | 207/4000 [14:19<4:20:10, 4.12s/it]
5%|▌ | 208/4000 [14:23<4:19:47, 4.11s/it]
5%|▌ | 209/4000 [14:28<4:26:08, 4.21s/it]
5%|▌ | 210/4000 [14:32<4:23:52, 4.18s/it]
{'loss': '1.449', 'grad_norm': '0.3036', 'learning_rate': '0.0005744', 'epoch': '0.05088'}
5%|▌ | 210/4000 [14:32<4:23:52, 4.18s/it]
5%|▌ | 211/4000 [14:36<4:22:19, 4.15s/it]
5%|▌ | 212/4000 [14:40<4:21:20, 4.14s/it]
5%|▌ | 213/4000 [14:44<4:20:31, 4.13s/it]
5%|▌ | 214/4000 [14:48<4:19:59, 4.12s/it]
5%|▌ | 215/4000 [14:52<4:19:29, 4.11s/it]
5%|▌ | 216/4000 [14:57<4:25:25, 4.21s/it]
5%|▌ | 217/4000 [15:01<4:23:12, 4.17s/it]
5%|▌ | 218/4000 [15:05<4:21:32, 4.15s/it]
5%|▌ | 219/4000 [15:09<4:20:36, 4.14s/it]
6%|▌ | 220/4000 [15:13<4:19:40, 4.12s/it]
{'loss': '1.435', 'grad_norm': '0.4483', 'learning_rate': '0.0005729', 'epoch': '0.05331'}
6%|▌ | 220/4000 [15:13<4:19:40, 4.12s/it]
6%|▌ | 221/4000 [15:17<4:19:14, 4.12s/it]
6%|▌ | 222/4000 [15:22<4:25:10, 4.21s/it]
6%|▌ | 223/4000 [15:26<4:22:49, 4.18s/it]
6%|▌ | 224/4000 [15:30<4:21:28, 4.15s/it]
6%|▌ | 225/4000 [15:34<4:20:13, 4.14s/it]
6%|▌ | 226/4000 [15:38<4:19:24, 4.12s/it]
6%|▌ | 227/4000 [15:42<4:18:53, 4.12s/it]
6%|▌ | 228/4000 [15:46<4:18:26, 4.11s/it]
6%|▌ | 229/4000 [15:51<4:23:41, 4.20s/it]
6%|▌ | 230/4000 [15:55<4:21:44, 4.17s/it]
{'loss': '1.39', 'grad_norm': '0.251', 'learning_rate': '0.0005714', 'epoch': '0.05573'}
6%|▌ | 230/4000 [15:55<4:21:44, 4.17s/it]
6%|▌ | 231/4000 [15:59<4:20:26, 4.15s/it]
6%|▌ | 232/4000 [16:03<4:19:24, 4.13s/it]
6%|▌ | 233/4000 [16:07<4:18:37, 4.12s/it]
6%|▌ | 234/4000 [16:11<4:18:08, 4.11s/it]
6%|▌ | 235/4000 [16:15<4:17:46, 4.11s/it]
6%|▌ | 236/4000 [16:20<4:24:34, 4.22s/it]
6%|▌ | 237/4000 [16:24<4:22:22, 4.18s/it]
6%|▌ | 238/4000 [16:28<4:20:31, 4.16s/it]
6%|▌ | 239/4000 [16:32<4:19:13, 4.14s/it]
6%|▌ | 240/4000 [16:36<4:18:23, 4.12s/it]
{'loss': '1.308', 'grad_norm': '0.2457', 'learning_rate': '0.0005698', 'epoch': '0.05815'}
6%|▌ | 240/4000 [16:36<4:18:23, 4.12s/it]
6%|▌ | 241/4000 [16:40<4:17:50, 4.12s/it]
6%|▌ | 242/4000 [16:44<4:17:18, 4.11s/it]
6%|▌ | 243/4000 [16:49<4:23:12, 4.20s/it]
6%|▌ | 244/4000 [16:53<4:21:07, 4.17s/it]
6%|▌ | 245/4000 [16:57<4:19:47, 4.15s/it]
6%|▌ | 246/4000 [17:01<4:18:42, 4.13s/it]
6%|▌ | 247/4000 [17:05<4:18:06, 4.13s/it]
6%|▌ | 248/4000 [17:09<4:17:43, 4.12s/it]
6%|▌ | 249/4000 [17:14<4:23:47, 4.22s/it]
6%|▋ | 250/4000 [17:18<4:21:24, 4.18s/it]
{'loss': '1.299', 'grad_norm': '0.2576', 'learning_rate': '0.0005683', 'epoch': '0.06057'}
6%|▋ | 250/4000 [17:18<4:21:24, 4.18s/it]
6%|▋ | 251/4000 [17:22<4:19:46, 4.16s/it]
6%|▋ | 252/4000 [17:26<4:18:36, 4.14s/it]
6%|▋ | 253/4000 [17:30<4:17:57, 4.13s/it]
6%|▋ | 254/4000 [17:34<4:17:16, 4.12s/it]
6%|▋ | 255/4000 [17:38<4:16:44, 4.11s/it]
6%|▋ | 256/4000 [17:43<4:22:49, 4.21s/it]
6%|▋ | 257/4000 [17:47<4:20:36, 4.18s/it]
6%|▋ | 258/4000 [17:51<4:19:00, 4.15s/it]
6%|▋ | 259/4000 [17:55<4:17:49, 4.14s/it]
6%|▋ | 260/4000 [17:59<4:17:05, 4.12s/it]
{'loss': '1.242', 'grad_norm': '0.2659', 'learning_rate': '0.0005668', 'epoch': '0.063'}
6%|▋ | 260/4000 [17:59<4:17:05, 4.12s/it]
7%|▋ | 261/4000 [18:03<4:16:37, 4.12s/it]
7%|▋ | 262/4000 [18:07<4:16:13, 4.11s/it]
7%|▋ | 263/4000 [18:12<4:21:17, 4.20s/it]
7%|▋ | 264/4000 [18:16<4:19:31, 4.17s/it]
7%|▋ | 265/4000 [18:20<4:18:02, 4.15s/it]
7%|▋ | 266/4000 [18:24<4:17:04, 4.13s/it]
7%|▋ | 267/4000 [18:28<4:16:23, 4.12s/it]
7%|▋ | 268/4000 [18:32<4:15:58, 4.12s/it]
7%|▋ | 269/4000 [18:36<4:19:12, 4.17s/it]
7%|▋ | 270/4000 [18:41<4:21:23, 4.20s/it]
{'loss': '1.182', 'grad_norm': '0.286', 'learning_rate': '0.0005653', 'epoch': '0.06542'}
7%|▋ | 270/4000 [18:41<4:21:23, 4.20s/it]
7%|▋ | 271/4000 [18:45<4:19:14, 4.17s/it]
7%|▋ | 272/4000 [18:49<4:17:45, 4.15s/it]
7%|▋ | 273/4000 [18:53<4:16:32, 4.13s/it]
7%|▋ | 274/4000 [18:57<4:15:53, 4.12s/it]
7%|▋ | 275/4000 [19:01<4:15:24, 4.11s/it]
7%|▋ | 276/4000 [19:06<4:21:28, 4.21s/it]
7%|▋ | 277/4000 [19:10<4:19:08, 4.18s/it]
7%|▋ | 278/4000 [19:14<4:17:37, 4.15s/it]
7%|▋ | 279/4000 [19:18<4:16:21, 4.13s/it]
7%|▋ | 280/4000 [19:22<4:15:38, 4.12s/it]
{'loss': '1.113', 'grad_norm': '0.2803', 'learning_rate': '0.0005638', 'epoch': '0.06784'}
7%|▋ | 280/4000 [19:22<4:15:38, 4.12s/it]
7%|▋ | 281/4000 [19:26<4:14:59, 4.11s/it]
7%|▋ | 282/4000 [19:30<4:14:38, 4.11s/it]
7%|▋ | 283/4000 [19:35<4:19:35, 4.19s/it]
7%|▋ | 284/4000 [19:39<4:17:50, 4.16s/it]
7%|▋ | 285/4000 [19:43<4:16:32, 4.14s/it]
7%|▋ | 286/4000 [19:47<4:15:42, 4.13s/it]
7%|▋ | 287/4000 [19:51<4:15:06, 4.12s/it]
7%|▋ | 288/4000 [19:55<4:14:33, 4.11s/it]
7%|▋ | 289/4000 [19:59<4:14:12, 4.11s/it]
7%|▋ | 290/4000 [20:04<4:20:31, 4.21s/it]
{'loss': '1.162', 'grad_norm': '0.3629', 'learning_rate': '0.0005623', 'epoch': '0.07027'}
7%|▋ | 290/4000 [20:04<4:20:31, 4.21s/it]
7%|▋ | 291/4000 [20:08<4:18:26, 4.18s/it]
7%|▋ | 292/4000 [20:12<4:16:52, 4.16s/it]
7%|▋ | 293/4000 [20:16<4:15:44, 4.14s/it]
7%|▋ | 294/4000 [20:20<4:14:48, 4.13s/it]
7%|▋ | 295/4000 [20:24<4:14:18, 4.12s/it]
7%|▋ | 296/4000 [20:28<4:17:19, 4.17s/it]
7%|▋ | 297/4000 [20:33<4:18:51, 4.19s/it]
7%|▋ | 298/4000 [20:37<4:17:01, 4.17s/it]
7%|▋ | 299/4000 [20:41<4:15:36, 4.14s/it]
8%|▊ | 300/4000 [20:45<4:14:30, 4.13s/it]
{'loss': '1.117', 'grad_norm': '0.2806', 'learning_rate': '0.0005608', 'epoch': '0.07269'}
8%|▊ | 300/4000 [20:45<4:14:30, 4.13s/it]
8%|▊ | 301/4000 [20:49<4:13:55, 4.12s/it]
8%|▊ | 302/4000 [20:53<4:13:33, 4.11s/it]
8%|▊ | 303/4000 [20:58<4:19:20, 4.21s/it]
8%|▊ | 304/4000 [21:02<4:17:13, 4.18s/it]
8%|▊ | 305/4000 [21:06<4:15:38, 4.15s/it]
8%|▊ | 306/4000 [21:10<4:14:41, 4.14s/it]
8%|▊ | 307/4000 [21:14<4:13:49, 4.12s/it]
8%|▊ | 308/4000 [21:18<4:13:20, 4.12s/it]
8%|▊ | 309/4000 [21:22<4:12:46, 4.11s/it]
8%|▊ | 310/4000 [21:27<4:19:34, 4.22s/it]
{'loss': '1.126', 'grad_norm': '0.2639', 'learning_rate': '0.0005592', 'epoch': '0.07511'}
8%|▊ | 310/4000 [21:27<4:19:34, 4.22s/it]
8%|▊ | 311/4000 [21:31<4:17:11, 4.18s/it]
8%|▊ | 312/4000 [21:35<4:15:25, 4.16s/it]
8%|▊ | 313/4000 [21:39<4:14:03, 4.13s/it]
8%|▊ | 314/4000 [21:43<4:13:10, 4.12s/it]
8%|▊ | 315/4000 [21:47<4:12:26, 4.11s/it]
8%|▊ | 316/4000 [21:51<4:12:01, 4.10s/it]
8%|▊ | 317/4000 [21:56<4:17:13, 4.19s/it]
8%|▊ | 318/4000 [22:00<4:15:12, 4.16s/it]
8%|▊ | 319/4000 [22:04<4:13:41, 4.14s/it]
8%|▊ | 320/4000 [22:08<4:12:47, 4.12s/it]
{'loss': '1.049', 'grad_norm': '0.2683', 'learning_rate': '0.0005577', 'epoch': '0.07754'}
8%|▊ | 320/4000 [22:08<4:12:47, 4.12s/it]
8%|▊ | 321/4000 [22:12<4:12:10, 4.11s/it]
8%|▊ | 322/4000 [22:16<4:11:41, 4.11s/it]
8%|▊ | 323/4000 [22:20<4:14:10, 4.15s/it]
8%|▊ | 324/4000 [22:24<4:16:21, 4.18s/it]
8%|▊ | 325/4000 [22:29<4:14:33, 4.16s/it]
8%|▊ | 326/4000 [22:33<4:13:19, 4.14s/it]
8%|▊ | 327/4000 [22:37<4:12:25, 4.12s/it]
8%|▊ | 328/4000 [22:41<4:11:44, 4.11s/it]
8%|▊ | 329/4000 [22:45<4:11:12, 4.11s/it]
8%|▊ | 330/4000 [22:49<4:16:56, 4.20s/it]
{'loss': '1.116', 'grad_norm': '0.2458', 'learning_rate': '0.0005562', 'epoch': '0.07996'}
8%|▊ | 330/4000 [22:49<4:16:56, 4.20s/it]
8%|▊ | 331/4000 [22:53<4:15:01, 4.17s/it]
8%|▊ | 332/4000 [22:58<4:13:29, 4.15s/it]
8%|▊ | 333/4000 [23:02<4:12:41, 4.13s/it]
8%|▊ | 334/4000 [23:06<4:12:04, 4.13s/it]
8%|▊ | 335/4000 [23:10<4:11:33, 4.12s/it]
8%|▊ | 336/4000 [23:14<4:11:08, 4.11s/it]
8%|▊ | 337/4000 [23:18<4:17:13, 4.21s/it]
8%|▊ | 338/4000 [23:23<4:15:01, 4.18s/it]
8%|▊ | 339/4000 [23:27<4:13:30, 4.15s/it]
8%|▊ | 340/4000 [23:31<4:12:36, 4.14s/it]
{'loss': '1.064', 'grad_norm': '0.2021', 'learning_rate': '0.0005547', 'epoch': '0.08238'}
8%|▊ | 340/4000 [23:31<4:12:36, 4.14s/it]
9%|▊ | 341/4000 [23:35<4:11:48, 4.13s/it]
9%|▊ | 342/4000 [23:39<4:11:19, 4.12s/it]
9%|▊ | 343/4000 [23:43<4:10:43, 4.11s/it]
9%|▊ | 344/4000 [23:47<4:16:51, 4.22s/it]
9%|▊ | 345/4000 [23:52<4:14:38, 4.18s/it]
9%|▊ | 346/4000 [23:56<4:13:07, 4.16s/it]
9%|▊ | 347/4000 [24:00<4:12:03, 4.14s/it]
9%|▊ | 348/4000 [24:04<4:11:15, 4.13s/it]
9%|▊ | 349/4000 [24:08<4:10:32, 4.12s/it]
9%|▉ | 350/4000 [24:12<4:15:49, 4.21s/it]
{'loss': '1.056', 'grad_norm': '0.2233', 'learning_rate': '0.0005532', 'epoch': '0.0848'}
9%|▉ | 350/4000 [24:12<4:15:49, 4.21s/it]
9%|▉ | 351/4000 [24:16<4:13:43, 4.17s/it]
9%|▉ | 352/4000 [24:21<4:12:09, 4.15s/it]
9%|▉ | 353/4000 [24:25<4:11:05, 4.13s/it]
9%|▉ | 354/4000 [24:29<4:10:23, 4.12s/it]
9%|▉ | 355/4000 [24:33<4:09:44, 4.11s/it]
9%|▉ | 356/4000 [24:37<4:09:24, 4.11s/it]
9%|▉ | 357/4000 [24:41<4:15:41, 4.21s/it]
9%|▉ | 358/4000 [24:45<4:13:29, 4.18s/it]
9%|▉ | 359/4000 [24:50<4:11:42, 4.15s/it]
9%|▉ | 360/4000 [24:54<4:10:44, 4.13s/it]
{'loss': '1.067', 'grad_norm': '0.2488', 'learning_rate': '0.0005517', 'epoch': '0.08723'}
9%|▉ | 360/4000 [24:54<4:10:44, 4.13s/it]
9%|▉ | 361/4000 [24:58<4:09:55, 4.12s/it]
9%|▉ | 362/4000 [25:02<4:09:18, 4.11s/it]
9%|▉ | 363/4000 [25:06<4:09:08, 4.11s/it]
9%|▉ | 364/4000 [25:10<4:14:42, 4.20s/it]
9%|▉ | 365/4000 [25:14<4:12:35, 4.17s/it]
9%|▉ | 366/4000 [25:19<4:11:01, 4.14s/it]
9%|▉ | 367/4000 [25:23<4:09:58, 4.13s/it]
9%|▉ | 368/4000 [25:27<4:09:10, 4.12s/it]
9%|▉ | 369/4000 [25:31<4:08:32, 4.11s/it]
9%|▉ | 370/4000 [25:35<4:08:18, 4.10s/it]
{'loss': '1.039', 'grad_norm': '0.2644', 'learning_rate': '0.0005502', 'epoch': '0.08965'}
9%|▉ | 370/4000 [25:35<4:08:18, 4.10s/it]
9%|▉ | 371/4000 [25:39<4:14:43, 4.21s/it]
9%|▉ | 372/4000 [25:43<4:12:28, 4.18s/it]
9%|▉ | 373/4000 [25:48<4:10:47, 4.15s/it]
9%|▉ | 374/4000 [25:52<4:09:34, 4.13s/it]
9%|▉ | 375/4000 [25:56<4:08:52, 4.12s/it]
9%|▉ | 376/4000 [26:00<4:08:07, 4.11s/it]
9%|▉ | 377/4000 [26:04<4:13:59, 4.21s/it]
9%|▉ | 378/4000 [26:08<4:11:47, 4.17s/it]
9%|▉ | 379/4000 [26:12<4:10:30, 4.15s/it]
10%|▉ | 380/4000 [26:17<4:09:22, 4.13s/it]
{'loss': '1.02', 'grad_norm': '0.2621', 'learning_rate': '0.0005486', 'epoch': '0.09207'}
10%|▉ | 380/4000 [26:17<4:09:22, 4.13s/it]
10%|▉ | 381/4000 [26:21<4:08:34, 4.12s/it]
10%|▉ | 382/4000 [26:25<4:07:59, 4.11s/it]
10%|▉ | 383/4000 [26:29<4:07:43, 4.11s/it]
10%|▉ | 384/4000 [26:33<4:13:46, 4.21s/it]
10%|▉ | 385/4000 [26:37<4:11:30, 4.17s/it]
10%|▉ | 386/4000 [26:41<4:09:58, 4.15s/it]
10%|▉ | 387/4000 [26:46<4:08:59, 4.13s/it]
10%|▉ | 388/4000 [26:50<4:08:02, 4.12s/it]
10%|▉ | 389/4000 [26:54<4:07:20, 4.11s/it]
10%|▉ | 390/4000 [26:58<4:06:52, 4.10s/it]
{'loss': '0.9707', 'grad_norm': '0.2696', 'learning_rate': '0.0005471', 'epoch': '0.0945'}
10%|▉ | 390/4000 [26:58<4:06:52, 4.10s/it]
10%|▉ | 391/4000 [27:02<4:12:38, 4.20s/it]
10%|▉ | 392/4000 [27:06<4:10:38, 4.17s/it]
10%|▉ | 393/4000 [27:10<4:09:06, 4.14s/it]
10%|▉ | 394/4000 [27:14<4:07:53, 4.12s/it]
10%|▉ | 395/4000 [27:19<4:07:10, 4.11s/it]
10%|▉ | 396/4000 [27:23<4:06:34, 4.11s/it]
10%|▉ | 397/4000 [27:27<4:09:34, 4.16s/it]
10%|▉ | 398/4000 [27:31<4:11:09, 4.18s/it]
10%|▉ | 399/4000 [27:35<4:09:32, 4.16s/it]
10%|█ | 400/4000 [27:39<4:08:22, 4.14s/it]
{'loss': '0.9642', 'grad_norm': '0.2032', 'learning_rate': '0.0005456', 'epoch': '0.09692'}
10%|█ | 400/4000 [27:39<4:08:22, 4.14s/it]
10%|█ | 401/4000 [27:43<4:07:21, 4.12s/it]
10%|█ | 402/4000 [27:48<4:06:41, 4.11s/it]
10%|█ | 403/4000 [27:52<4:06:12, 4.11s/it]
10%|█ | 404/4000 [27:56<4:12:35, 4.21s/it]
10%|█ | 405/4000 [28:00<4:10:15, 4.18s/it]
10%|█ | 406/4000 [28:04<4:08:28, 4.15s/it]
10%|█ | 407/4000 [28:08<4:07:28, 4.13s/it]
10%|█ | 408/4000 [28:12<4:06:42, 4.12s/it]
10%|█ | 409/4000 [28:17<4:06:11, 4.11s/it]
10%|█ | 410/4000 [28:21<4:05:39, 4.11s/it]
{'loss': '1.005', 'grad_norm': '0.2873', 'learning_rate': '0.0005441', 'epoch': '0.09934'}
10%|█ | 410/4000 [28:21<4:05:39, 4.11s/it]
10%|█ | 411/4000 [28:25<4:11:33, 4.21s/it]
10%|█ | 412/4000 [28:29<4:09:37, 4.17s/it]
10%|█ | 413/4000 [28:33<4:07:59, 4.15s/it]
10%|█ | 414/4000 [28:37<4:06:43, 4.13s/it]
10%|█ | 415/4000 [28:41<4:05:57, 4.12s/it]
10%|█ | 416/4000 [28:46<4:05:18, 4.11s/it]
10%|█ | 417/4000 [28:50<4:04:50, 4.10s/it]
10%|█ | 418/4000 [28:54<4:10:06, 4.19s/it]
10%|█ | 419/4000 [28:58<4:08:11, 4.16s/it]
10%|█ | 420/4000 [29:02<4:06:57, 4.14s/it]
{'loss': '0.9969', 'grad_norm': '0.1983', 'learning_rate': '0.0005426', 'epoch': '0.1018'}
10%|█ | 420/4000 [29:02<4:06:57, 4.14s/it]
11%|█ | 421/4000 [29:06<4:06:02, 4.12s/it]
11%|█ | 422/4000 [29:10<4:05:27, 4.12s/it]
11%|█ | 423/4000 [29:15<4:05:16, 4.11s/it]
11%|█ | 424/4000 [29:19<4:05:00, 4.11s/it]
11%|█ | 425/4000 [29:23<4:11:23, 4.22s/it]
11%|█ | 426/4000 [29:27<4:09:24, 4.19s/it]
11%|█ | 427/4000 [29:31<4:08:09, 4.17s/it]
11%|█ | 428/4000 [29:35<4:07:06, 4.15s/it]
11%|█ | 429/4000 [29:40<4:06:24, 4.14s/it]
11%|█ | 430/4000 [29:44<4:05:55, 4.13s/it]
{'loss': '0.9302', 'grad_norm': '0.1899', 'learning_rate': '0.0005411', 'epoch': '0.1042'}
11%|█ | 430/4000 [29:44<4:05:55, 4.13s/it]
11%|█ | 431/4000 [29:48<4:10:57, 4.22s/it]
11%|█ | 432/4000 [29:52<4:08:59, 4.19s/it]
11%|█ | 433/4000 [29:56<4:07:31, 4.16s/it]
11%|█ | 434/4000 [30:00<4:06:39, 4.15s/it]
11%|█ | 435/4000 [30:05<4:06:05, 4.14s/it]
11%|█ | 436/4000 [30:09<4:05:08, 4.13s/it]
11%|█ | 437/4000 [30:13<4:04:31, 4.12s/it]
11%|█ | 438/4000 [30:17<4:12:09, 4.25s/it]
11%|█ | 439/4000 [30:21<4:09:36, 4.21s/it]
11%|█ | 440/4000 [30:26<4:08:02, 4.18s/it]
{'loss': '0.9665', 'grad_norm': '0.2194', 'learning_rate': '0.0005395', 'epoch': '0.1066'}
11%|█ | 440/4000 [30:26<4:08:02, 4.18s/it]
11%|█ | 441/4000 [30:30<4:06:38, 4.16s/it]
11%|█ | 442/4000 [30:34<4:05:51, 4.15s/it]
11%|█ | 443/4000 [30:38<4:05:01, 4.13s/it]
11%|█ | 444/4000 [30:42<4:04:26, 4.12s/it]
11%|█ | 445/4000 [30:46<4:10:01, 4.22s/it]
11%|█ | 446/4000 [30:50<4:07:55, 4.19s/it]
11%|█ | 447/4000 [30:55<4:06:13, 4.16s/it]
11%|█ | 448/4000 [30:59<4:05:14, 4.14s/it]
11%|█ | 449/4000 [31:03<4:06:00, 4.16s/it]
11%|█▏ | 450/4000 [31:07<4:05:34, 4.15s/it]
{'loss': '0.9361', 'grad_norm': '0.232', 'learning_rate': '0.000538', 'epoch': '0.109'}
11%|█▏ | 450/4000 [31:07<4:05:34, 4.15s/it]
11%|█▏ | 451/4000 [31:11<4:07:10, 4.18s/it]
11%|█▏ | 452/4000 [31:16<4:09:29, 4.22s/it]
11%|█▏ | 453/4000 [31:20<4:07:40, 4.19s/it]
11%|█▏ | 454/4000 [31:24<4:06:09, 4.17s/it]
11%|█▏ | 455/4000 [31:28<4:05:25, 4.15s/it]
11%|█▏ | 456/4000 [31:32<4:04:42, 4.14s/it]
11%|█▏ | 457/4000 [31:36<4:04:31, 4.14s/it]
11%|█▏ | 458/4000 [31:41<4:11:03, 4.25s/it]
11%|█▏ | 459/4000 [31:45<4:08:55, 4.22s/it]
12%|█▏ | 460/4000 [31:49<4:07:20, 4.19s/it]
{'loss': '0.9602', 'grad_norm': '0.2184', 'learning_rate': '0.0005365', 'epoch': '0.1115'}
12%|█▏ | 460/4000 [31:49<4:07:20, 4.19s/it]
12%|█▏ | 461/4000 [31:53<4:06:08, 4.17s/it]
12%|█▏ | 462/4000 [31:57<4:05:09, 4.16s/it]
12%|█▏ | 463/4000 [32:01<4:04:41, 4.15s/it]
12%|█▏ | 464/4000 [32:05<4:04:11, 4.14s/it]
12%|█▏ | 465/4000 [32:10<4:09:12, 4.23s/it]
12%|█▏ | 466/4000 [32:14<4:07:08, 4.20s/it]
12%|█▏ | 467/4000 [32:18<4:05:40, 4.17s/it]
12%|█▏ | 468/4000 [32:22<4:05:05, 4.16s/it]
12%|█▏ | 469/4000 [32:26<4:04:18, 4.15s/it]
12%|█▏ | 470/4000 [32:31<4:03:56, 4.15s/it]
{'loss': '0.8988', 'grad_norm': '0.2376', 'learning_rate': '0.000535', 'epoch': '0.1139'}
12%|█▏ | 470/4000 [32:31<4:03:56, 4.15s/it]
12%|█▏ | 471/4000 [32:35<4:03:56, 4.15s/it]
12%|█▏ | 472/4000 [32:39<4:10:09, 4.25s/it]
12%|█▏ | 473/4000 [32:43<4:07:55, 4.22s/it]
12%|█▏ | 474/4000 [32:47<4:06:14, 4.19s/it]
12%|█▏ | 475/4000 [32:52<4:05:14, 4.17s/it]
12%|█▏ | 476/4000 [32:56<4:04:08, 4.16s/it]
12%|█▏ | 477/4000 [33:00<4:03:26, 4.15s/it]
12%|█▏ | 478/4000 [33:04<4:09:43, 4.25s/it]
12%|█▏ | 479/4000 [33:08<4:07:33, 4.22s/it]
12%|█▏ | 480/4000 [33:13<4:05:45, 4.19s/it]
{'loss': '0.8198', 'grad_norm': '0.2114', 'learning_rate': '0.0005335', 'epoch': '0.1163'}
12%|█▏ | 480/4000 [33:13<4:05:45, 4.19s/it]
12%|█▏ | 481/4000 [33:17<4:04:21, 4.17s/it]
12%|█▏ | 482/4000 [33:21<4:03:23, 4.15s/it]
12%|█▏ | 483/4000 [33:25<4:02:42, 4.14s/it]
12%|█▏ | 484/4000 [33:29<4:02:29, 4.14s/it]
12%|█▏ | 485/4000 [33:33<4:07:35, 4.23s/it]
12%|█▏ | 486/4000 [33:38<4:05:56, 4.20s/it]
12%|█▏ | 487/4000 [33:42<4:04:26, 4.17s/it]
12%|█▏ | 488/4000 [33:46<4:03:21, 4.16s/it]
12%|█▏ | 489/4000 [33:50<4:02:54, 4.15s/it]
12%|█▏ | 490/4000 [33:54<4:02:22, 4.14s/it]
{'loss': '0.8315', 'grad_norm': '0.1903', 'learning_rate': '0.000532', 'epoch': '0.1187'}
12%|█▏ | 490/4000 [33:54<4:02:22, 4.14s/it]
12%|█▏ | 491/4000 [33:58<4:02:09, 4.14s/it]
12%|█▏ | 492/4000 [34:03<4:08:39, 4.25s/it]
12%|█▏ | 493/4000 [34:07<4:06:13, 4.21s/it]
12%|█▏ | 494/4000 [34:11<4:04:42, 4.19s/it]
12%|█▏ | 495/4000 [34:15<4:03:32, 4.17s/it]
12%|█▏ | 496/4000 [34:19<4:02:49, 4.16s/it]
12%|█▏ | 497/4000 [34:23<4:02:04, 4.15s/it]
12%|█▏ | 498/4000 [34:28<4:01:56, 4.15s/it]
12%|█▏ | 499/4000 [34:32<4:07:15, 4.24s/it]
12%|█▎ | 500/4000 [34:36<4:05:27, 4.21s/it]
{'loss': '0.8515', 'grad_norm': '0.2477', 'learning_rate': '0.0005305', 'epoch': '0.1211'}
12%|█▎ | 500/4000 [34:36<4:05:27, 4.21s/it]
13%|█▎ | 501/4000 [34:40<4:04:12, 4.19s/it]
13%|█▎ | 502/4000 [34:44<4:03:22, 4.17s/it]
13%|█▎ | 503/4000 [34:49<4:02:14, 4.16s/it]
13%|█▎ | 504/4000 [34:53<4:01:50, 4.15s/it]
13%|█▎ | 505/4000 [34:57<4:04:23, 4.20s/it]
13%|█▎ | 506/4000 [35:01<4:06:37, 4.24s/it]
13%|█▎ | 507/4000 [35:05<4:04:32, 4.20s/it]
13%|█▎ | 508/4000 [35:10<4:03:22, 4.18s/it]
13%|█▎ | 509/4000 [35:14<4:02:11, 4.16s/it]
13%|█▎ | 510/4000 [35:18<4:01:42, 4.16s/it]
{'loss': '0.8253', 'grad_norm': '0.1977', 'learning_rate': '0.0005289', 'epoch': '0.1236'}
13%|█▎ | 510/4000 [35:18<4:01:42, 4.16s/it]
13%|█▎ | 511/4000 [35:22<4:00:59, 4.14s/it]
13%|█▎ | 512/4000 [35:26<4:06:49, 4.25s/it]
13%|█▎ | 513/4000 [35:31<4:04:45, 4.21s/it]
13%|█▎ | 514/4000 [35:35<4:03:11, 4.19s/it]
13%|█▎ | 515/4000 [35:39<4:01:53, 4.16s/it]
13%|█▎ | 516/4000 [35:43<4:01:20, 4.16s/it]
13%|█▎ | 517/4000 [35:47<4:00:38, 4.15s/it]
13%|█▎ | 518/4000 [35:51<4:00:28, 4.14s/it]
13%|█▎ | 519/4000 [35:56<4:05:44, 4.24s/it]
13%|█▎ | 520/4000 [36:00<4:03:55, 4.21s/it]
{'loss': '0.8197', 'grad_norm': '0.2504', 'learning_rate': '0.0005274', 'epoch': '0.126'}
13%|█▎ | 520/4000 [36:00<4:03:55, 4.21s/it]
13%|█▎ | 521/4000 [36:04<4:02:24, 4.18s/it]
13%|█▎ | 522/4000 [36:08<4:01:08, 4.16s/it]
13%|█▎ | 523/4000 [36:12<4:00:48, 4.16s/it]
13%|█▎ | 524/4000 [36:16<4:00:30, 4.15s/it]
13%|█▎ | 525/4000 [36:20<3:59:49, 4.14s/it]
13%|█▎ | 526/4000 [36:25<4:06:01, 4.25s/it]
13%|█▎ | 527/4000 [36:29<4:04:08, 4.22s/it]
13%|█▎ | 528/4000 [36:33<4:02:19, 4.19s/it]
13%|█▎ | 529/4000 [36:37<4:01:01, 4.17s/it]
13%|█▎ | 530/4000 [36:41<4:00:23, 4.16s/it]
{'loss': '0.8134', 'grad_norm': '0.1738', 'learning_rate': '0.0005259', 'epoch': '0.1284'}
13%|█▎ | 530/4000 [36:41<4:00:23, 4.16s/it]
13%|█▎ | 531/4000 [36:46<3:59:42, 4.15s/it]
13%|█▎ | 532/4000 [36:50<4:04:52, 4.24s/it]
13%|█▎ | 533/4000 [36:54<4:02:46, 4.20s/it]
13%|█▎ | 534/4000 [36:58<4:01:35, 4.18s/it]
13%|█▎ | 535/4000 [37:02<4:00:26, 4.16s/it]
13%|█▎ | 536/4000 [37:07<4:00:07, 4.16s/it]
13%|█▎ | 537/4000 [37:11<4:01:48, 4.19s/it]
13%|█▎ | 538/4000 [37:15<4:01:56, 4.19s/it]
13%|█▎ | 539/4000 [37:20<4:08:53, 4.31s/it]
14%|█▎ | 540/4000 [37:24<4:07:56, 4.30s/it]
{'loss': '0.7794', 'grad_norm': '0.2064', 'learning_rate': '0.0005244', 'epoch': '0.1308'}
14%|█▎ | 540/4000 [37:24<4:07:56, 4.30s/it]
14%|█▎ | 541/4000 [37:28<4:07:28, 4.29s/it]
14%|█▎ | 542/4000 [37:32<4:04:54, 4.25s/it]
14%|█▎ | 543/4000 [37:36<4:03:24, 4.22s/it]
14%|█▎ | 544/4000 [37:41<4:02:25, 4.21s/it]
14%|█▎ | 545/4000 [37:45<4:01:26, 4.19s/it]
14%|█▎ | 546/4000 [37:49<4:07:07, 4.29s/it]
14%|█▎ | 547/4000 [37:53<4:04:39, 4.25s/it]
14%|█▎ | 548/4000 [37:58<4:02:56, 4.22s/it]
14%|█▎ | 549/4000 [38:02<4:01:33, 4.20s/it]
14%|█▍ | 550/4000 [38:06<4:00:33, 4.18s/it]
{'loss': '0.8368', 'grad_norm': '0.2211', 'learning_rate': '0.0005229', 'epoch': '0.1333'}
14%|█▍ | 550/4000 [38:06<4:00:33, 4.18s/it]
14%|█▍ | 551/4000 [38:10<4:00:12, 4.18s/it]
14%|█▍ | 552/4000 [38:14<4:02:41, 4.22s/it]
14%|█▍ | 553/4000 [38:19<4:05:13, 4.27s/it]
14%|█▍ | 554/4000 [38:23<4:03:03, 4.23s/it]
14%|█▍ | 555/4000 [38:27<4:01:23, 4.20s/it]
14%|█▍ | 556/4000 [38:31<4:00:14, 4.19s/it]
14%|█▍ | 557/4000 [38:35<3:59:36, 4.18s/it]
14%|█▍ | 558/4000 [38:39<3:59:04, 4.17s/it]
14%|█▍ | 559/4000 [38:44<4:05:51, 4.29s/it]
14%|█▍ | 560/4000 [38:48<4:03:14, 4.24s/it]
{'loss': '0.7846', 'grad_norm': '0.1814', 'learning_rate': '0.0005214', 'epoch': '0.1357'}
14%|█▍ | 560/4000 [38:48<4:03:14, 4.24s/it]
14%|█▍ | 561/4000 [38:52<4:01:12, 4.21s/it]
14%|█▍ | 562/4000 [38:56<3:59:57, 4.19s/it]
14%|█▍ | 563/4000 [39:01<3:59:12, 4.18s/it]
14%|█▍ | 564/4000 [39:05<3:58:20, 4.16s/it]
14%|█▍ | 565/4000 [39:09<3:57:38, 4.15s/it]
14%|█▍ | 566/4000 [39:13<4:03:40, 4.26s/it]
14%|█▍ | 567/4000 [39:18<4:01:43, 4.22s/it]
14%|█▍ | 568/4000 [39:22<4:00:10, 4.20s/it]
14%|█▍ | 569/4000 [39:26<3:59:04, 4.18s/it]
14%|█▍ | 570/4000 [39:30<3:58:16, 4.17s/it]
{'loss': '0.8119', 'grad_norm': '0.3592', 'learning_rate': '0.0005198', 'epoch': '0.1381'}
14%|█▍ | 570/4000 [39:30<3:58:16, 4.17s/it]
14%|█▍ | 571/4000 [39:34<3:57:28, 4.16s/it]
14%|█▍ | 572/4000 [39:38<3:57:07, 4.15s/it]
14%|█▍ | 573/4000 [39:43<4:03:24, 4.26s/it]
14%|█▍ | 574/4000 [39:47<4:01:06, 4.22s/it]
14%|█▍ | 575/4000 [39:51<3:59:09, 4.19s/it]
14%|█▍ | 576/4000 [39:55<3:58:21, 4.18s/it]
14%|█▍ | 577/4000 [39:59<3:57:39, 4.17s/it]
14%|█▍ | 578/4000 [40:03<3:57:13, 4.16s/it]
14%|█▍ | 579/4000 [40:08<4:01:30, 4.24s/it]
14%|█▍ | 580/4000 [40:12<4:04:47, 4.29s/it]
{'loss': '0.8114', 'grad_norm': '0.176', 'learning_rate': '0.0005183', 'epoch': '0.1405'}
14%|█▍ | 580/4000 [40:12<4:04:47, 4.29s/it]
15%|█▍ | 581/4000 [40:16<4:03:06, 4.27s/it]
15%|█▍ | 582/4000 [40:21<4:01:40, 4.24s/it]
15%|█▍ | 583/4000 [40:25<4:02:36, 4.26s/it]
15%|█▍ | 584/4000 [40:29<4:02:05, 4.25s/it]
15%|█▍ | 585/4000 [40:33<3:59:48, 4.21s/it]
15%|█▍ | 586/4000 [40:38<4:04:40, 4.30s/it]
15%|█▍ | 587/4000 [40:42<4:01:28, 4.25s/it]
15%|█▍ | 588/4000 [40:46<3:59:44, 4.22s/it]
15%|█▍ | 589/4000 [40:50<3:58:20, 4.19s/it]
15%|█▍ | 590/4000 [40:54<3:57:02, 4.17s/it]
{'loss': '0.7472', 'grad_norm': '0.1836', 'learning_rate': '0.0005168', 'epoch': '0.143'}
15%|█▍ | 590/4000 [40:54<3:57:02, 4.17s/it]
15%|█▍ | 591/4000 [40:58<3:56:13, 4.16s/it]
15%|█▍ | 592/4000 [41:03<3:55:38, 4.15s/it]
15%|█▍ | 593/4000 [41:07<4:01:42, 4.26s/it]
15%|█▍ | 594/4000 [41:11<3:59:28, 4.22s/it]
15%|█▍ | 595/4000 [41:15<3:57:42, 4.19s/it]
15%|█▍ | 596/4000 [41:19<3:56:27, 4.17s/it]
15%|█▍ | 597/4000 [41:24<3:55:45, 4.16s/it]
15%|█▍ | 598/4000 [41:28<3:55:19, 4.15s/it]
15%|█▍ | 599/4000 [41:32<3:54:58, 4.15s/it]
15%|█▌ | 600/4000 [41:36<4:00:47, 4.25s/it]
{'loss': '0.7377', 'grad_norm': '0.1711', 'learning_rate': '0.0005153', 'epoch': '0.1454'}
15%|█▌ | 600/4000 [41:36<4:00:47, 4.25s/it]
15%|█▌ | 601/4000 [41:40<3:58:49, 4.22s/it]
15%|█▌ | 602/4000 [41:45<3:57:23, 4.19s/it]
15%|█▌ | 603/4000 [41:49<3:56:10, 4.17s/it]
15%|█▌ | 604/4000 [41:53<3:55:28, 4.16s/it]
15%|█▌ | 605/4000 [41:57<3:54:50, 4.15s/it]
15%|█▌ | 606/4000 [42:01<3:57:24, 4.20s/it]
15%|█▌ | 607/4000 [42:06<3:59:43, 4.24s/it]
15%|█▌ | 608/4000 [42:10<3:57:55, 4.21s/it]
15%|█▌ | 609/4000 [42:14<3:56:23, 4.18s/it]
15%|█▌ | 610/4000 [42:18<3:55:32, 4.17s/it]
{'loss': '0.7716', 'grad_norm': '0.2212', 'learning_rate': '0.0005138', 'epoch': '0.1478'}
15%|█▌ | 610/4000 [42:18<3:55:32, 4.17s/it]
15%|█▌ | 611/4000 [42:22<3:54:46, 4.16s/it]
15%|█▌ | 612/4000 [42:26<3:54:12, 4.15s/it]
15%|█▌ | 613/4000 [42:31<4:00:03, 4.25s/it]
15%|█▌ | 614/4000 [42:35<3:57:45, 4.21s/it]
15%|█▌ | 615/4000 [42:39<3:56:20, 4.19s/it]
15%|█▌ | 616/4000 [42:43<3:55:56, 4.18s/it]
15%|█▌ | 617/4000 [42:47<3:55:57, 4.19s/it]
15%|█▌ | 618/4000 [42:52<3:55:24, 4.18s/it]
15%|█▌ | 619/4000 [42:56<3:55:12, 4.17s/it]
16%|█▌ | 620/4000 [43:00<4:02:11, 4.30s/it]
{'loss': '0.7474', 'grad_norm': '0.1852', 'learning_rate': '0.0005123', 'epoch': '0.1502'}
16%|█▌ | 620/4000 [43:00<4:02:11, 4.30s/it]
16%|█▌ | 621/4000 [43:05<4:00:32, 4.27s/it]
16%|█▌ | 622/4000 [43:09<3:58:00, 4.23s/it]
16%|█▌ | 623/4000 [43:13<3:56:20, 4.20s/it]
16%|█▌ | 624/4000 [43:17<3:55:32, 4.19s/it]
16%|█▌ | 625/4000 [43:21<3:54:36, 4.17s/it]
16%|█▌ | 626/4000 [43:25<3:53:57, 4.16s/it]
16%|█▌ | 627/4000 [43:30<3:59:30, 4.26s/it]
16%|█▌ | 628/4000 [43:34<3:57:18, 4.22s/it]
16%|█▌ | 629/4000 [43:38<3:55:49, 4.20s/it]
16%|█▌ | 630/4000 [43:42<3:54:42, 4.18s/it]
{'loss': '0.7208', 'grad_norm': '0.1889', 'learning_rate': '0.0005108', 'epoch': '0.1526'}
16%|█▌ | 630/4000 [43:42<3:54:42, 4.18s/it]
16%|█▌ | 631/4000 [43:46<3:53:49, 4.16s/it]
16%|█▌ | 632/4000 [43:50<3:53:17, 4.16s/it]
16%|█▌ | 633/4000 [43:55<3:58:31, 4.25s/it]
16%|█▌ | 634/4000 [43:59<3:56:38, 4.22s/it]
16%|█▌ | 635/4000 [44:03<3:55:25, 4.20s/it]
16%|█▌ | 636/4000 [44:07<3:54:30, 4.18s/it]
16%|█▌ | 637/4000 [44:11<3:53:39, 4.17s/it]
16%|█▌ | 638/4000 [44:16<3:53:14, 4.16s/it]
16%|█▌ | 639/4000 [44:20<3:54:44, 4.19s/it]
16%|█▌ | 640/4000 [44:24<4:01:01, 4.30s/it]
{'loss': '0.7037', 'grad_norm': '0.1678', 'learning_rate': '0.0005092', 'epoch': '0.1551'}
16%|█▌ | 640/4000 [44:24<4:01:01, 4.30s/it]
16%|█▌ | 641/4000 [44:29<3:59:05, 4.27s/it]
16%|█▌ | 642/4000 [44:33<3:57:50, 4.25s/it]
16%|█▌ | 643/4000 [44:37<3:57:47, 4.25s/it]
16%|█▌ | 644/4000 [44:41<3:55:53, 4.22s/it]
16%|█▌ | 645/4000 [44:45<3:54:35, 4.20s/it]
16%|█▌ | 646/4000 [44:49<3:53:34, 4.18s/it]
16%|█▌ | 647/4000 [44:54<3:59:05, 4.28s/it]
16%|█▌ | 648/4000 [44:58<3:56:55, 4.24s/it]
16%|█▌ | 649/4000 [45:02<3:55:02, 4.21s/it]
16%|█▋ | 650/4000 [45:06<3:53:48, 4.19s/it]
{'loss': '0.7125', 'grad_norm': '0.2202', 'learning_rate': '0.0005077', 'epoch': '0.1575'}
16%|█▋ | 650/4000 [45:06<3:53:48, 4.19s/it]
16%|█▋ | 651/4000 [45:11<3:52:52, 4.17s/it]
16%|█▋ | 652/4000 [45:15<3:52:16, 4.16s/it]
16%|█▋ | 653/4000 [45:19<3:52:24, 4.17s/it]
16%|█▋ | 654/4000 [45:23<3:57:47, 4.26s/it]
16%|█▋ | 655/4000 [45:27<3:55:34, 4.23s/it]
16%|█▋ | 656/4000 [45:32<3:54:18, 4.20s/it]
16%|█▋ | 657/4000 [45:36<3:52:54, 4.18s/it]
16%|█▋ | 658/4000 [45:40<3:51:52, 4.16s/it]
16%|█▋ | 659/4000 [45:44<3:51:32, 4.16s/it]
16%|█▋ | 660/4000 [45:49<3:57:32, 4.27s/it]
{'loss': '0.7117', 'grad_norm': '0.1797', 'learning_rate': '0.0005062', 'epoch': '0.1599'}
16%|█▋ | 660/4000 [45:49<3:57:32, 4.27s/it]
17%|█▋ | 661/4000 [45:53<3:55:14, 4.23s/it]
17%|█▋ | 662/4000 [45:57<3:53:35, 4.20s/it]
17%|█▋ | 663/4000 [46:01<3:52:37, 4.18s/it]
17%|█▋ | 664/4000 [46:05<3:53:21, 4.20s/it]
17%|█▋ | 665/4000 [46:09<3:53:34, 4.20s/it]
17%|█▋ | 666/4000 [46:14<3:53:06, 4.19s/it]
17%|█▋ | 667/4000 [46:18<3:58:25, 4.29s/it]
17%|█▋ | 668/4000 [46:22<3:58:32, 4.30s/it]
17%|█▋ | 669/4000 [46:27<3:57:14, 4.27s/it]
17%|█▋ | 670/4000 [46:31<3:54:46, 4.23s/it]
{'loss': '0.7337', 'grad_norm': '0.1529', 'learning_rate': '0.0005047', 'epoch': '0.1623'}
17%|█▋ | 670/4000 [46:31<3:54:46, 4.23s/it]
17%|█▋ | 671/4000 [46:35<3:52:57, 4.20s/it]
17%|█▋ | 672/4000 [46:39<3:51:51, 4.18s/it]
17%|█▋ | 673/4000 [46:43<3:50:42, 4.16s/it]
17%|█▋ | 674/4000 [46:48<3:56:20, 4.26s/it]
17%|█▋ | 675/4000 [46:52<3:53:51, 4.22s/it]
17%|█▋ | 676/4000 [46:56<3:52:30, 4.20s/it]
17%|█▋ | 677/4000 [47:00<3:51:13, 4.17s/it]
17%|█▋ | 678/4000 [47:04<3:50:25, 4.16s/it]
17%|█▋ | 679/4000 [47:08<3:50:06, 4.16s/it]
17%|█▋ | 680/4000 [47:13<3:52:58, 4.21s/it]
{'loss': '0.724', 'grad_norm': '0.169', 'learning_rate': '0.0005032', 'epoch': '0.1648'}
17%|█▋ | 680/4000 [47:13<3:52:58, 4.21s/it]
17%|█▋ | 681/4000 [47:17<3:55:07, 4.25s/it]
17%|█▋ | 682/4000 [47:21<3:53:55, 4.23s/it]
17%|█▋ | 683/4000 [47:25<3:53:19, 4.22s/it]
17%|█▋ | 684/4000 [47:29<3:51:50, 4.20s/it]
17%|█▋ | 685/4000 [47:34<3:50:30, 4.17s/it]
17%|█▋ | 686/4000 [47:38<3:49:53, 4.16s/it]
17%|█▋ | 687/4000 [47:42<3:54:54, 4.25s/it]
17%|█▋ | 688/4000 [47:46<3:53:15, 4.23s/it]
17%|█▋ | 689/4000 [47:51<3:52:18, 4.21s/it]
17%|█▋ | 690/4000 [47:55<3:51:00, 4.19s/it]
{'loss': '0.7363', 'grad_norm': '0.2151', 'learning_rate': '0.0005017', 'epoch': '0.1672'}
17%|█▋ | 690/4000 [47:55<3:51:00, 4.19s/it]
17%|█▋ | 691/4000 [47:59<3:50:13, 4.17s/it]
17%|█▋ | 692/4000 [48:03<3:49:31, 4.16s/it]
17%|█▋ | 693/4000 [48:07<3:49:16, 4.16s/it]
17%|█▋ | 694/4000 [48:12<3:55:14, 4.27s/it]
17%|█▋ | 695/4000 [48:16<3:53:00, 4.23s/it]
17%|█▋ | 696/4000 [48:20<3:51:10, 4.20s/it]
17%|█▋ | 697/4000 [48:24<3:50:06, 4.18s/it]
17%|█▋ | 698/4000 [48:28<3:49:22, 4.17s/it]
17%|█▋ | 699/4000 [48:32<3:48:59, 4.16s/it]
18%|█▊ | 700/4000 [48:36<3:48:29, 4.15s/it]
{'loss': '0.7571', 'grad_norm': '0.1559', 'learning_rate': '0.0005002', 'epoch': '0.1696'}
18%|█▊ | 700/4000 [48:36<3:48:29, 4.15s/it]
18%|█▊ | 701/4000 [48:41<3:53:51, 4.25s/it]
18%|█▊ | 702/4000 [48:45<3:52:18, 4.23s/it]
18%|█▊ | 703/4000 [48:49<3:51:23, 4.21s/it]
18%|█▊ | 704/4000 [48:53<3:50:34, 4.20s/it]
18%|█▊ | 705/4000 [48:58<3:50:01, 4.19s/it]
18%|█▊ | 706/4000 [49:02<3:49:43, 4.18s/it]
18%|█▊ | 707/4000 [49:06<3:52:33, 4.24s/it]
18%|█▊ | 708/4000 [49:11<3:54:55, 4.28s/it]
18%|█▊ | 709/4000 [49:15<3:52:57, 4.25s/it]
18%|█▊ | 710/4000 [49:19<3:51:35, 4.22s/it]
{'loss': '0.7024', 'grad_norm': '0.2419', 'learning_rate': '0.0004986', 'epoch': '0.172'}
18%|█▊ | 710/4000 [49:19<3:51:35, 4.22s/it]
18%|█▊ | 711/4000 [49:23<3:50:27, 4.20s/it]
18%|█▊ | 712/4000 [49:27<3:49:56, 4.20s/it]
18%|█▊ | 713/4000 [49:31<3:50:28, 4.21s/it]
18%|█▊ | 714/4000 [49:36<3:58:56, 4.36s/it]
18%|█▊ | 715/4000 [49:40<3:57:38, 4.34s/it]
18%|█▊ | 716/4000 [49:45<3:56:43, 4.32s/it]
18%|█▊ | 717/4000 [49:49<3:55:52, 4.31s/it]
18%|█▊ | 718/4000 [49:53<3:56:14, 4.32s/it]
18%|█▊ | 719/4000 [49:58<3:54:39, 4.29s/it]
18%|█▊ | 720/4000 [50:02<3:52:50, 4.26s/it]
{'loss': '0.7251', 'grad_norm': '0.2271', 'learning_rate': '0.0004971', 'epoch': '0.1745'}
18%|█▊ | 720/4000 [50:02<3:52:50, 4.26s/it]
18%|█▊ | 721/4000 [50:06<3:58:06, 4.36s/it]
18%|█▊ | 722/4000 [50:11<3:54:59, 4.30s/it]
18%|█▊ | 723/4000 [50:15<3:52:48, 4.26s/it]
18%|█▊ | 724/4000 [50:19<3:51:06, 4.23s/it]
18%|█▊ | 725/4000 [50:23<3:50:04, 4.22s/it]
18%|█▊ | 726/4000 [50:27<3:49:05, 4.20s/it]
18%|█▊ | 727/4000 [50:31<3:48:34, 4.19s/it]
18%|█▊ | 728/4000 [50:36<3:55:31, 4.32s/it]
18%|█▊ | 729/4000 [50:40<3:55:55, 4.33s/it]
18%|█▊ | 730/4000 [50:45<3:54:36, 4.30s/it]
{'loss': '0.69', 'grad_norm': '0.2366', 'learning_rate': '0.0004956', 'epoch': '0.1769'}
18%|█▊ | 730/4000 [50:45<3:54:36, 4.30s/it]
18%|█▊ | 731/4000 [50:49<3:54:34, 4.31s/it]
18%|█▊ | 732/4000 [50:53<3:54:01, 4.30s/it]
18%|█▊ | 733/4000 [50:58<3:54:26, 4.31s/it]
18%|█▊ | 734/4000 [51:02<3:56:16, 4.34s/it]
18%|█▊ | 735/4000 [51:06<3:56:23, 4.34s/it]
18%|█▊ | 736/4000 [51:10<3:53:39, 4.30s/it]
18%|█▊ | 737/4000 [51:15<3:51:32, 4.26s/it]
18%|█▊ | 738/4000 [51:19<3:50:17, 4.24s/it]
18%|█▊ | 739/4000 [51:23<3:48:58, 4.21s/it]
18%|█▊ | 740/4000 [51:27<3:48:02, 4.20s/it]
{'loss': '0.6789', 'grad_norm': '0.1401', 'learning_rate': '0.0004941', 'epoch': '0.1793'}
18%|█▊ | 740/4000 [51:27<3:48:02, 4.20s/it]
19%|█▊ | 741/4000 [51:32<3:54:16, 4.31s/it]
19%|█▊ | 742/4000 [51:36<3:54:06, 4.31s/it]
19%|█▊ | 743/4000 [51:40<3:55:12, 4.33s/it]
19%|█▊ | 744/4000 [51:45<3:54:03, 4.31s/it]
19%|█▊ | 745/4000 [51:49<3:53:30, 4.30s/it]
19%|█▊ | 746/4000 [51:53<3:52:51, 4.29s/it]
19%|█▊ | 747/4000 [51:57<3:51:36, 4.27s/it]
19%|█▊ | 748/4000 [52:02<3:57:23, 4.38s/it]
19%|█▊ | 749/4000 [52:06<3:53:52, 4.32s/it]
19%|█▉ | 750/4000 [52:10<3:51:38, 4.28s/it]
{'loss': '0.6706', 'grad_norm': '0.156', 'learning_rate': '0.0004926', 'epoch': '0.1817'}
19%|█▉ | 750/4000 [52:10<3:51:38, 4.28s/it]
19%|█▉ | 751/4000 [52:15<3:52:38, 4.30s/it]
19%|█▉ | 752/4000 [52:19<3:51:19, 4.27s/it]
19%|█▉ | 753/4000 [52:23<3:53:05, 4.31s/it]
19%|█▉ | 754/4000 [52:28<3:51:13, 4.27s/it]
19%|█▉ | 755/4000 [52:32<3:56:01, 4.36s/it]
19%|█▉ | 756/4000 [52:36<3:53:07, 4.31s/it]
19%|█▉ | 757/4000 [52:41<3:52:00, 4.29s/it]
19%|█▉ | 758/4000 [52:45<3:51:40, 4.29s/it]
19%|█▉ | 759/4000 [52:49<3:51:37, 4.29s/it]
19%|█▉ | 760/4000 [52:53<3:51:36, 4.29s/it]
{'loss': '0.6586', 'grad_norm': '0.1442', 'learning_rate': '0.0004911', 'epoch': '0.1841'}
19%|█▉ | 760/4000 [52:53<3:51:36, 4.29s/it]
19%|█▉ | 761/4000 [52:58<3:59:55, 4.44s/it]
19%|█▉ | 762/4000 [53:02<3:55:39, 4.37s/it]
19%|█▉ | 763/4000 [53:07<3:52:27, 4.31s/it]
19%|█▉ | 764/4000 [53:11<3:50:29, 4.27s/it]
19%|█▉ | 765/4000 [53:15<3:48:50, 4.24s/it]
19%|█▉ | 766/4000 [53:19<3:47:35, 4.22s/it]
19%|█▉ | 767/4000 [53:23<3:46:43, 4.21s/it]
19%|█▉ | 768/4000 [53:28<3:52:43, 4.32s/it]
19%|█▉ | 769/4000 [53:32<3:50:13, 4.28s/it]
19%|█▉ | 770/4000 [53:36<3:49:21, 4.26s/it]
{'loss': '0.6648', 'grad_norm': '0.1553', 'learning_rate': '0.0004895', 'epoch': '0.1866'}
19%|█▉ | 770/4000 [53:36<3:49:21, 4.26s/it]
19%|█▉ | 771/4000 [53:41<3:49:26, 4.26s/it]
19%|█▉ | 772/4000 [53:45<3:50:41, 4.29s/it]
19%|█▉ | 773/4000 [53:49<3:51:25, 4.30s/it]
19%|█▉ | 774/4000 [53:54<3:52:23, 4.32s/it]
19%|█▉ | 775/4000 [53:59<4:03:02, 4.52s/it]
19%|█▉ | 776/4000 [54:03<3:58:00, 4.43s/it]
19%|█▉ | 777/4000 [54:07<3:54:20, 4.36s/it]
19%|█▉ | 778/4000 [54:11<3:52:39, 4.33s/it]
19%|█▉ | 779/4000 [54:16<3:50:37, 4.30s/it]
20%|█▉ | 780/4000 [54:20<3:49:12, 4.27s/it]
{'loss': '0.7217', 'grad_norm': '0.1571', 'learning_rate': '0.000488', 'epoch': '0.189'}
20%|█▉ | 780/4000 [54:20<3:49:12, 4.27s/it]
20%|█▉ | 781/4000 [54:24<3:48:20, 4.26s/it]
20%|█▉ | 782/4000 [54:29<3:54:48, 4.38s/it]
20%|█▉ | 783/4000 [54:33<3:55:09, 4.39s/it]
20%|█▉ | 784/4000 [54:37<3:53:17, 4.35s/it]
20%|█▉ | 785/4000 [54:42<3:52:56, 4.35s/it]
20%|█▉ | 786/4000 [54:46<3:53:42, 4.36s/it]
20%|█▉ | 787/4000 [54:50<3:54:26, 4.38s/it]
20%|█▉ | 788/4000 [54:55<4:02:20, 4.53s/it]
20%|█▉ | 789/4000 [55:00<3:57:32, 4.44s/it]
20%|█▉ | 790/4000 [55:04<3:54:51, 4.39s/it]
{'loss': '0.656', 'grad_norm': '0.1701', 'learning_rate': '0.0004865', 'epoch': '0.1914'}
20%|█▉ | 790/4000 [55:04<3:54:51, 4.39s/it]
20%|█▉ | 791/4000 [55:08<3:52:38, 4.35s/it]
20%|█▉ | 792/4000 [55:12<3:51:46, 4.33s/it]
20%|█▉ | 793/4000 [55:17<3:49:52, 4.30s/it]
20%|█▉ | 794/4000 [55:21<3:48:11, 4.27s/it]
20%|█▉ | 795/4000 [55:25<3:53:31, 4.37s/it]
20%|█▉ | 796/4000 [55:30<3:50:38, 4.32s/it]
20%|█▉ | 797/4000 [55:34<3:51:04, 4.33s/it]
20%|█▉ | 798/4000 [55:38<3:51:17, 4.33s/it]
20%|█▉ | 799/4000 [55:43<3:50:39, 4.32s/it]
20%|██ | 800/4000 [55:47<3:50:36, 4.32s/it]
{'loss': '0.6637', 'grad_norm': '0.1514', 'learning_rate': '0.000485', 'epoch': '0.1938'}
20%|██ | 800/4000 [55:47<3:50:36, 4.32s/it]
20%|██ | 801/4000 [55:51<3:51:50, 4.35s/it]
20%|██ | 802/4000 [55:56<3:56:26, 4.44s/it]
20%|██ | 803/4000 [56:00<3:52:28, 4.36s/it]
20%|██ | 804/4000 [56:04<3:49:47, 4.31s/it]
20%|██ | 805/4000 [56:09<3:49:33, 4.31s/it]
20%|██ | 806/4000 [56:13<3:48:36, 4.29s/it]
20%|██ | 807/4000 [56:17<3:46:47, 4.26s/it]
20%|██ | 808/4000 [56:22<3:49:21, 4.31s/it]
20%|██ | 809/4000 [56:26<3:50:54, 4.34s/it]
20%|██ | 810/4000 [56:30<3:52:48, 4.38s/it]
{'loss': '0.648', 'grad_norm': '0.1341', 'learning_rate': '0.0004835', 'epoch': '0.1963'}
20%|██ | 810/4000 [56:30<3:52:48, 4.38s/it]
20%|██ | 811/4000 [56:35<3:52:57, 4.38s/it]
20%|██ | 812/4000 [56:39<3:51:45, 4.36s/it]
20%|██ | 813/4000 [56:43<3:50:28, 4.34s/it]
20%|██ | 814/4000 [56:48<3:51:27, 4.36s/it]
20%|██ | 815/4000 [56:52<3:56:44, 4.46s/it]
20%|██ | 816/4000 [56:57<3:52:43, 4.39s/it]
20%|██ | 817/4000 [57:01<3:50:02, 4.34s/it]
20%|██ | 818/4000 [57:05<3:47:47, 4.30s/it]
20%|██ | 819/4000 [57:09<3:46:11, 4.27s/it]
20%|██ | 820/4000 [57:14<3:45:59, 4.26s/it]
{'loss': '0.6421', 'grad_norm': '0.1593', 'learning_rate': '0.000482', 'epoch': '0.1987'}
20%|██ | 820/4000 [57:14<3:45:59, 4.26s/it]
21%|██ | 821/4000 [57:18<3:46:30, 4.28s/it]
21%|██ | 822/4000 [57:23<3:52:15, 4.38s/it]
21%|██ | 823/4000 [57:27<3:50:27, 4.35s/it]
21%|██ | 824/4000 [57:31<3:49:54, 4.34s/it]
21%|██ | 825/4000 [57:35<3:48:31, 4.32s/it]
21%|██ | 826/4000 [57:40<3:48:30, 4.32s/it]
21%|██ | 827/4000 [57:44<3:47:52, 4.31s/it]
21%|██ | 828/4000 [57:48<3:46:05, 4.28s/it]
21%|██ | 829/4000 [57:53<3:56:48, 4.48s/it]
21%|██ | 830/4000 [57:57<3:52:37, 4.40s/it]
{'loss': '0.6431', 'grad_norm': '0.1672', 'learning_rate': '0.0004805', 'epoch': '0.2011'}
21%|██ | 830/4000 [57:57<3:52:37, 4.40s/it]
21%|██ | 831/4000 [58:02<3:49:07, 4.34s/it]
21%|██ | 832/4000 [58:06<3:46:14, 4.28s/it]
21%|██ | 833/4000 [58:10<3:44:27, 4.25s/it]
21%|██ | 834/4000 [58:14<3:43:39, 4.24s/it]
21%|██ | 835/4000 [58:19<3:50:44, 4.37s/it]
21%|██ | 836/4000 [58:23<3:51:36, 4.39s/it]
21%|██ | 837/4000 [58:27<3:49:27, 4.35s/it]
21%|██ | 838/4000 [58:32<3:49:00, 4.35s/it]
21%|██ | 839/4000 [58:36<3:48:07, 4.33s/it]
21%|██ | 840/4000 [58:40<3:48:48, 4.34s/it]
{'loss': '0.6408', 'grad_norm': '0.1539', 'learning_rate': '0.0004789', 'epoch': '0.2035'}
21%|██ | 840/4000 [58:40<3:48:48, 4.34s/it]
21%|██ | 841/4000 [58:45<3:46:56, 4.31s/it]
21%|██ | 842/4000 [58:49<3:52:12, 4.41s/it]
21%|██ | 843/4000 [58:54<3:52:03, 4.41s/it]
21%|██ | 844/4000 [58:58<3:48:45, 4.35s/it]
21%|██ | 845/4000 [59:02<3:46:20, 4.30s/it]
21%|██ | 846/4000 [59:06<3:44:45, 4.28s/it]
21%|██ | 847/4000 [59:11<3:43:36, 4.26s/it]
21%|██ | 848/4000 [59:15<3:43:06, 4.25s/it]
21%|██ | 849/4000 [59:20<3:53:34, 4.45s/it]
21%|██▏ | 850/4000 [59:24<3:50:59, 4.40s/it]
{'loss': '0.5943', 'grad_norm': '0.14', 'learning_rate': '0.0004774', 'epoch': '0.206'}
21%|██▏ | 850/4000 [59:24<3:50:59, 4.40s/it]
21%|██▏ | 851/4000 [59:28<3:49:34, 4.37s/it]
21%|██▏ | 852/4000 [59:33<3:50:19, 4.39s/it]
21%|██▏ | 853/4000 [59:37<3:50:07, 4.39s/it]
21%|██▏ | 854/4000 [59:41<3:48:16, 4.35s/it]
21%|██▏ | 855/4000 [59:46<3:46:32, 4.32s/it]
21%|██▏ | 856/4000 [59:50<3:50:33, 4.40s/it]
21%|██▏ | 857/4000 [59:54<3:47:19, 4.34s/it]
21%|██▏ | 858/4000 [59:59<3:48:42, 4.37s/it]
21%|██▏ | 859/4000 [1:00:03<3:45:32, 4.31s/it]
22%|██▏ | 860/4000 [1:00:07<3:43:28, 4.27s/it]
{'loss': '0.6131', 'grad_norm': '0.136', 'learning_rate': '0.0004759', 'epoch': '0.2084'}
22%|██▏ | 860/4000 [1:00:07<3:43:28, 4.27s/it]
22%|██▏ | 861/4000 [1:00:11<3:42:03, 4.24s/it]
22%|██▏ | 862/4000 [1:00:16<3:49:05, 4.38s/it]
22%|██▏ | 863/4000 [1:00:21<3:52:01, 4.44s/it]
22%|██▏ | 864/4000 [1:00:25<3:49:57, 4.40s/it]
22%|██▏ | 865/4000 [1:00:29<3:48:36, 4.38s/it]
22%|██▏ | 866/4000 [1:00:34<3:46:59, 4.35s/it]
22%|██▏ | 867/4000 [1:00:38<3:44:40, 4.30s/it]
22%|██▏ | 868/4000 [1:00:42<3:42:56, 4.27s/it]
22%|██▏ | 869/4000 [1:00:47<3:48:39, 4.38s/it]
22%|██▏ | 870/4000 [1:00:51<3:46:05, 4.33s/it]
{'loss': '0.6394', 'grad_norm': '0.1506', 'learning_rate': '0.0004744', 'epoch': '0.2108'}
22%|██▏ | 870/4000 [1:00:51<3:46:05, 4.33s/it]
22%|██▏ | 871/4000 [1:00:55<3:43:53, 4.29s/it]
22%|██▏ | 872/4000 [1:00:59<3:44:02, 4.30s/it]
22%|██▏ | 873/4000 [1:01:04<3:42:18, 4.27s/it]
22%|██▏ | 874/4000 [1:01:08<3:41:38, 4.25s/it]
22%|██▏ | 875/4000 [1:01:12<3:43:16, 4.29s/it]
22%|██▏ | 876/4000 [1:01:17<3:51:04, 4.44s/it]
22%|██▏ | 877/4000 [1:01:21<3:48:46, 4.40s/it]
22%|██▏ | 878/4000 [1:01:26<3:47:29, 4.37s/it]
22%|██▏ | 879/4000 [1:01:30<3:45:50, 4.34s/it]
22%|██▏ | 880/4000 [1:01:34<3:45:03, 4.33s/it]
{'loss': '0.6229', 'grad_norm': '0.1507', 'learning_rate': '0.0004729', 'epoch': '0.2132'}
22%|██▏ | 880/4000 [1:01:34<3:45:03, 4.33s/it]
22%|██▏ | 881/4000 [1:01:38<3:43:23, 4.30s/it]
22%|██▏ | 882/4000 [1:01:43<3:41:35, 4.26s/it]
22%|██▏ | 883/4000 [1:01:47<3:47:20, 4.38s/it]
22%|██▏ | 884/4000 [1:01:51<3:44:30, 4.32s/it]
22%|██▏ | 885/4000 [1:01:56<3:42:15, 4.28s/it]
22%|██▏ | 886/4000 [1:02:00<3:41:53, 4.28s/it]
22%|██▏ | 887/4000 [1:02:04<3:40:53, 4.26s/it]
22%|██▏ | 888/4000 [1:02:08<3:41:00, 4.26s/it]
22%|██▏ | 889/4000 [1:02:13<3:49:07, 4.42s/it]
22%|██▏ | 890/4000 [1:02:17<3:46:40, 4.37s/it]
{'loss': '0.6182', 'grad_norm': '0.1306', 'learning_rate': '0.0004714', 'epoch': '0.2156'}
22%|██▏ | 890/4000 [1:02:17<3:46:40, 4.37s/it]
22%|██▏ | 891/4000 [1:02:22<3:46:04, 4.36s/it]
22%|██▏ | 892/4000 [1:02:26<3:44:27, 4.33s/it]
22%|██▏ | 893/4000 [1:02:30<3:44:48, 4.34s/it]
22%|██▏ | 894/4000 [1:02:35<3:42:27, 4.30s/it]
22%|██▏ | 895/4000 [1:02:39<3:40:49, 4.27s/it]
22%|██▏ | 896/4000 [1:02:43<3:46:28, 4.38s/it]
22%|██▏ | 897/4000 [1:02:48<3:43:20, 4.32s/it]
22%|██▏ | 898/4000 [1:02:52<3:40:54, 4.27s/it]
22%|██▏ | 899/4000 [1:02:56<3:39:16, 4.24s/it]
22%|██▎ | 900/4000 [1:03:00<3:38:25, 4.23s/it]
{'loss': '0.614', 'grad_norm': '0.149', 'learning_rate': '0.0004698', 'epoch': '0.2181'}
22%|██▎ | 900/4000 [1:03:00<3:38:25, 4.23s/it]
23%|██▎ | 901/4000 [1:03:04<3:37:29, 4.21s/it]
23%|██▎ | 902/4000 [1:03:08<3:37:28, 4.21s/it]
23%|██▎ | 903/4000 [1:03:13<3:48:08, 4.42s/it]
23%|██▎ | 904/4000 [1:03:18<3:46:45, 4.39s/it]
23%|██▎ | 905/4000 [1:03:22<3:44:57, 4.36s/it]
23%|██▎ | 906/4000 [1:03:26<3:44:40, 4.36s/it]
23%|██▎ | 907/4000 [1:03:31<3:43:33, 4.34s/it]
23%|██▎ | 908/4000 [1:03:35<3:41:25, 4.30s/it]
23%|██▎ | 909/4000 [1:03:39<3:39:36, 4.26s/it]
23%|██▎ | 910/4000 [1:03:44<3:44:52, 4.37s/it]
{'loss': '0.6052', 'grad_norm': '0.152', 'learning_rate': '0.0004683', 'epoch': '0.2205'}
23%|██▎ | 910/4000 [1:03:44<3:44:52, 4.37s/it]
23%|██▎ | 911/4000 [1:03:48<3:42:41, 4.33s/it]
23%|██▎ | 912/4000 [1:03:52<3:40:24, 4.28s/it]
23%|██▎ | 913/4000 [1:03:56<3:38:34, 4.25s/it]
23%|██▎ | 914/4000 [1:04:00<3:37:35, 4.23s/it]
23%|██▎ | 915/4000 [1:04:05<3:41:48, 4.31s/it]
23%|██▎ | 916/4000 [1:04:10<3:47:35, 4.43s/it]
23%|██▎ | 917/4000 [1:04:14<3:44:55, 4.38s/it]
23%|██▎ | 918/4000 [1:04:18<3:43:22, 4.35s/it]
23%|██▎ | 919/4000 [1:04:22<3:42:54, 4.34s/it]
23%|██▎ | 920/4000 [1:04:27<3:42:23, 4.33s/it]
{'loss': '0.6114', 'grad_norm': '0.1508', 'learning_rate': '0.0004668', 'epoch': '0.2229'}
23%|██▎ | 920/4000 [1:04:27<3:42:23, 4.33s/it]
23%|██▎ | 921/4000 [1:04:31<3:40:01, 4.29s/it]
23%|██▎ | 922/4000 [1:04:35<3:38:17, 4.26s/it]
23%|██▎ | 923/4000 [1:04:40<3:48:44, 4.46s/it]
23%|██▎ | 924/4000 [1:04:44<3:46:53, 4.43s/it]
23%|██▎ | 925/4000 [1:04:49<3:43:38, 4.36s/it]
23%|██▎ | 926/4000 [1:04:53<3:40:53, 4.31s/it]
23%|██▎ | 927/4000 [1:04:57<3:38:46, 4.27s/it]
23%|██▎ | 928/4000 [1:05:01<3:37:28, 4.25s/it]
23%|██▎ | 929/4000 [1:05:05<3:37:18, 4.25s/it]
23%|██▎ | 930/4000 [1:05:10<3:44:37, 4.39s/it]
{'loss': '0.6094', 'grad_norm': '0.1683', 'learning_rate': '0.0004653', 'epoch': '0.2253'}
23%|██▎ | 930/4000 [1:05:10<3:44:37, 4.39s/it]
23%|██▎ | 931/4000 [1:05:14<3:43:07, 4.36s/it]
23%|██▎ | 932/4000 [1:05:19<3:42:08, 4.34s/it]
23%|██▎ | 933/4000 [1:05:23<3:43:06, 4.36s/it]
23%|██▎ | 934/4000 [1:05:27<3:41:42, 4.34s/it]
23%|██▎ | 935/4000 [1:05:32<3:39:51, 4.30s/it]
23%|██▎ | 936/4000 [1:05:36<3:41:43, 4.34s/it]
23%|██▎ | 937/4000 [1:05:40<3:42:18, 4.35s/it]
23%|██▎ | 938/4000 [1:05:45<3:39:46, 4.31s/it]
23%|██▎ | 939/4000 [1:05:49<3:38:25, 4.28s/it]
24%|██▎ | 940/4000 [1:05:53<3:37:47, 4.27s/it]
{'loss': '0.6596', 'grad_norm': '0.1666', 'learning_rate': '0.0004638', 'epoch': '0.2278'}
24%|██▎ | 940/4000 [1:05:53<3:37:47, 4.27s/it]
24%|██▎ | 941/4000 [1:05:57<3:37:03, 4.26s/it]
24%|██▎ | 942/4000 [1:06:02<3:40:17, 4.32s/it]
24%|██▎ | 943/4000 [1:06:07<3:47:54, 4.47s/it]
24%|██▎ | 944/4000 [1:06:11<3:46:16, 4.44s/it]
24%|██▎ | 945/4000 [1:06:15<3:44:41, 4.41s/it]
24%|██▎ | 946/4000 [1:06:20<3:41:15, 4.35s/it]
24%|██▎ | 947/4000 [1:06:24<3:38:45, 4.30s/it]
24%|██▎ | 948/4000 [1:06:28<3:36:44, 4.26s/it]
24%|██▎ | 949/4000 [1:06:32<3:35:24, 4.24s/it]
24%|██▍ | 950/4000 [1:06:37<3:40:55, 4.35s/it]
{'loss': '0.5961', 'grad_norm': '0.1414', 'learning_rate': '0.0004623', 'epoch': '0.2302'}
24%|██▍ | 950/4000 [1:06:37<3:40:55, 4.35s/it]
24%|██▍ | 951/4000 [1:06:41<3:39:27, 4.32s/it]
24%|██▍ | 952/4000 [1:06:45<3:37:29, 4.28s/it]
24%|██▍ | 953/4000 [1:06:49<3:36:54, 4.27s/it]
24%|██▍ | 954/4000 [1:06:54<3:36:52, 4.27s/it]
24%|██▍ | 955/4000 [1:06:58<3:37:24, 4.28s/it]
24%|██▍ | 956/4000 [1:07:02<3:38:12, 4.30s/it]
24%|██▍ | 957/4000 [1:07:07<3:44:41, 4.43s/it]
24%|██▍ | 958/4000 [1:07:11<3:41:04, 4.36s/it]
24%|██▍ | 959/4000 [1:07:15<3:38:25, 4.31s/it]
24%|██▍ | 960/4000 [1:07:20<3:39:04, 4.32s/it]
{'loss': '0.6108', 'grad_norm': '0.1701', 'learning_rate': '0.0004608', 'epoch': '0.2326'}
24%|██▍ | 960/4000 [1:07:20<3:39:04, 4.32s/it]
24%|██▍ | 961/4000 [1:07:24<3:37:16, 4.29s/it]
24%|██▍ | 962/4000 [1:07:28<3:36:34, 4.28s/it]
24%|██▍ | 963/4000 [1:07:32<3:35:28, 4.26s/it]
24%|██▍ | 964/4000 [1:07:37<3:40:45, 4.36s/it]
24%|██▍ | 965/4000 [1:07:41<3:38:05, 4.31s/it]
24%|██▍ | 966/4000 [1:07:46<3:38:42, 4.33s/it]
24%|██▍ | 967/4000 [1:07:50<3:37:51, 4.31s/it]
24%|██▍ | 968/4000 [1:07:54<3:37:41, 4.31s/it]
24%|██▍ | 969/4000 [1:07:59<3:39:12, 4.34s/it]
24%|██▍ | 970/4000 [1:08:03<3:45:44, 4.47s/it]
{'loss': '0.5866', 'grad_norm': '0.1793', 'learning_rate': '0.0004592', 'epoch': '0.235'}
24%|██▍ | 970/4000 [1:08:03<3:45:44, 4.47s/it]
24%|██▍ | 971/4000 [1:08:08<3:41:43, 4.39s/it]
24%|██▍ | 972/4000 [1:08:12<3:39:32, 4.35s/it]
24%|██▍ | 973/4000 [1:08:16<3:37:06, 4.30s/it]
24%|██▍ | 974/4000 [1:08:20<3:35:54, 4.28s/it]
24%|██▍ | 975/4000 [1:08:24<3:34:29, 4.25s/it]
24%|██▍ | 976/4000 [1:08:29<3:33:21, 4.23s/it]
24%|██▍ | 977/4000 [1:08:33<3:39:25, 4.36s/it]
24%|██▍ | 978/4000 [1:08:38<3:39:12, 4.35s/it]
24%|██▍ | 979/4000 [1:08:42<3:39:31, 4.36s/it]
24%|██▍ | 980/4000 [1:08:46<3:38:16, 4.34s/it]
{'loss': '0.5984', 'grad_norm': '0.1629', 'learning_rate': '0.0004577', 'epoch': '0.2375'}
24%|██▍ | 980/4000 [1:08:46<3:38:16, 4.34s/it]
25%|██▍ | 981/4000 [1:08:51<3:37:43, 4.33s/it]
25%|██▍ | 982/4000 [1:08:55<3:36:46, 4.31s/it]
25%|██▍ | 983/4000 [1:08:59<3:35:29, 4.29s/it]
25%|██▍ | 984/4000 [1:09:04<3:40:05, 4.38s/it]
25%|██▍ | 985/4000 [1:09:08<3:37:05, 4.32s/it]
25%|██▍ | 986/4000 [1:09:12<3:34:49, 4.28s/it]
25%|██▍ | 987/4000 [1:09:16<3:34:28, 4.27s/it]
25%|██▍ | 988/4000 [1:09:21<3:36:21, 4.31s/it]
25%|██▍ | 989/4000 [1:09:25<3:35:07, 4.29s/it]
25%|██▍ | 990/4000 [1:09:29<3:36:32, 4.32s/it]
{'loss': '0.6663', 'grad_norm': '0.1632', 'learning_rate': '0.0004562', 'epoch': '0.2399'}
25%|██▍ | 990/4000 [1:09:29<3:36:32, 4.32s/it]
25%|██▍ | 991/4000 [1:09:34<3:40:24, 4.40s/it]
25%|██▍ | 992/4000 [1:09:38<3:37:22, 4.34s/it]
25%|██▍ | 993/4000 [1:09:42<3:34:34, 4.28s/it]
25%|██▍ | 994/4000 [1:09:46<3:32:50, 4.25s/it]
25%|██▍ | 995/4000 [1:09:51<3:31:39, 4.23s/it]
25%|██▍ | 996/4000 [1:09:55<3:30:56, 4.21s/it]
25%|██▍ | 997/4000 [1:09:59<3:36:43, 4.33s/it]
25%|██▍ | 998/4000 [1:10:04<3:34:22, 4.28s/it]
25%|██▍ | 999/4000 [1:10:08<3:32:22, 4.25s/it]
25%|██▌ | 1000/4000 [1:10:12<3:31:16, 4.23s/it]
{'loss': '0.5965', 'grad_norm': '0.1698', 'learning_rate': '0.0004547', 'epoch': '0.2423'}
25%|██▌ | 1000/4000 [1:10:12<3:31:16, 4.23s/it]
25%|██▌ | 1001/4000 [1:10:16<3:30:00, 4.20s/it]
25%|██▌ | 1002/4000 [1:10:20<3:29:21, 4.19s/it]
25%|██▌ | 1003/4000 [1:10:25<3:31:30, 4.23s/it]
25%|██▌ | 1004/4000 [1:10:29<3:35:56, 4.32s/it]
25%|██▌ | 1005/4000 [1:10:33<3:33:28, 4.28s/it]
25%|██▌ | 1006/4000 [1:10:37<3:31:30, 4.24s/it]
25%|██▌ | 1007/4000 [1:10:42<3:30:08, 4.21s/it]
25%|██▌ | 1008/4000 [1:10:46<3:29:17, 4.20s/it]
25%|██▌ | 1009/4000 [1:10:50<3:28:38, 4.19s/it]
25%|██▌ | 1010/4000 [1:10:54<3:28:24, 4.18s/it]
{'loss': '0.6223', 'grad_norm': '0.1658', 'learning_rate': '0.0004532', 'epoch': '0.2447'}
25%|██▌ | 1010/4000 [1:10:54<3:28:24, 4.18s/it]
25%|██▌ | 1011/4000 [1:10:59<3:34:11, 4.30s/it]
25%|██▌ | 1012/4000 [1:11:03<3:32:00, 4.26s/it]
25%|██▌ | 1013/4000 [1:11:07<3:30:35, 4.23s/it]
25%|██▌ | 1014/4000 [1:11:11<3:29:18, 4.21s/it]
25%|██▌ | 1015/4000 [1:11:15<3:28:14, 4.19s/it]
25%|██▌ | 1016/4000 [1:11:19<3:27:55, 4.18s/it]
25%|██▌ | 1017/4000 [1:11:24<3:34:00, 4.30s/it]
25%|██▌ | 1018/4000 [1:11:28<3:31:23, 4.25s/it]
25%|██▌ | 1019/4000 [1:11:32<3:30:00, 4.23s/it]
26%|██▌ | 1020/4000 [1:11:36<3:28:57, 4.21s/it]
{'loss': '0.5967', 'grad_norm': '0.1394', 'learning_rate': '0.0004517', 'epoch': '0.2471'}
26%|██▌ | 1020/4000 [1:11:36<3:28:57, 4.21s/it]
26%|██▌ | 1021/4000 [1:11:41<3:28:26, 4.20s/it]
26%|██▌ | 1022/4000 [1:11:45<3:28:07, 4.19s/it]
26%|██▌ | 1023/4000 [1:11:49<3:27:41, 4.19s/it]
26%|██▌ | 1024/4000 [1:11:54<3:33:08, 4.30s/it]
26%|██▌ | 1025/4000 [1:11:58<3:30:59, 4.26s/it]
26%|██▌ | 1026/4000 [1:12:02<3:29:34, 4.23s/it]
26%|██▌ | 1027/4000 [1:12:06<3:28:48, 4.21s/it]
26%|██▌ | 1028/4000 [1:12:10<3:27:57, 4.20s/it]
26%|██▌ | 1029/4000 [1:12:14<3:27:52, 4.20s/it]
26%|██▌ | 1030/4000 [1:12:19<3:27:18, 4.19s/it]
{'loss': '0.6391', 'grad_norm': '0.1589', 'learning_rate': '0.0004502', 'epoch': '0.2496'}
26%|██▌ | 1030/4000 [1:12:19<3:27:18, 4.19s/it]
26%|██▌ | 1031/4000 [1:12:23<3:32:38, 4.30s/it]
26%|██▌ | 1032/4000 [1:12:27<3:30:38, 4.26s/it]
26%|██▌ | 1033/4000 [1:12:31<3:29:06, 4.23s/it]
26%|██▌ | 1034/4000 [1:12:36<3:28:13, 4.21s/it]
26%|██▌ | 1035/4000 [1:12:40<3:27:22, 4.20s/it]
26%|██▌ | 1036/4000 [1:12:44<3:26:48, 4.19s/it]
26%|██▌ | 1037/4000 [1:12:48<3:26:25, 4.18s/it]
26%|██▌ | 1038/4000 [1:12:53<3:31:27, 4.28s/it]
26%|██▌ | 1039/4000 [1:12:57<3:29:32, 4.25s/it]
26%|██▌ | 1040/4000 [1:13:01<3:28:20, 4.22s/it]
{'loss': '0.5962', 'grad_norm': '0.1483', 'learning_rate': '0.0004486', 'epoch': '0.252'}
26%|██▌ | 1040/4000 [1:13:01<3:28:20, 4.22s/it]
26%|██▌ | 1041/4000 [1:13:05<3:28:02, 4.22s/it]
26%|██▌ | 1042/4000 [1:13:09<3:27:18, 4.21s/it]
26%|██▌ | 1043/4000 [1:13:14<3:26:32, 4.19s/it]
26%|██▌ | 1044/4000 [1:13:18<3:29:26, 4.25s/it]
26%|██▌ | 1045/4000 [1:13:22<3:30:58, 4.28s/it]
26%|██▌ | 1046/4000 [1:13:26<3:29:14, 4.25s/it]
26%|██▌ | 1047/4000 [1:13:31<3:30:29, 4.28s/it]
26%|██▌ | 1048/4000 [1:13:35<3:28:37, 4.24s/it]
26%|██▌ | 1049/4000 [1:13:39<3:27:17, 4.21s/it]
26%|██▋ | 1050/4000 [1:13:43<3:26:28, 4.20s/it]
{'loss': '0.634', 'grad_norm': '0.1758', 'learning_rate': '0.0004471', 'epoch': '0.2544'}
26%|██▋ | 1050/4000 [1:13:43<3:26:28, 4.20s/it]
26%|██▋ | 1051/4000 [1:13:48<3:31:17, 4.30s/it]
26%|██▋ | 1052/4000 [1:13:52<3:29:30, 4.26s/it]
26%|██▋ | 1053/4000 [1:13:56<3:28:15, 4.24s/it]
26%|██▋ | 1054/4000 [1:14:00<3:27:14, 4.22s/it]
26%|██▋ | 1055/4000 [1:14:05<3:28:07, 4.24s/it]
26%|██▋ | 1056/4000 [1:14:09<3:28:43, 4.25s/it]
26%|██▋ | 1057/4000 [1:14:13<3:27:08, 4.22s/it]
26%|██▋ | 1058/4000 [1:14:18<3:32:01, 4.32s/it]
26%|██▋ | 1059/4000 [1:14:22<3:29:40, 4.28s/it]
26%|██▋ | 1060/4000 [1:14:26<3:27:55, 4.24s/it]
{'loss': '0.5492', 'grad_norm': '0.1251', 'learning_rate': '0.0004456', 'epoch': '0.2568'}
26%|██▋ | 1060/4000 [1:14:26<3:27:55, 4.24s/it]
27%|██▋ | 1061/4000 [1:14:30<3:26:32, 4.22s/it]
27%|██▋ | 1062/4000 [1:14:34<3:25:39, 4.20s/it]
27%|██▋ | 1063/4000 [1:14:38<3:25:30, 4.20s/it]
27%|██▋ | 1064/4000 [1:14:43<3:24:50, 4.19s/it]
27%|██▋ | 1065/4000 [1:14:47<3:30:17, 4.30s/it]
27%|██▋ | 1066/4000 [1:14:51<3:28:07, 4.26s/it]
27%|██▋ | 1067/4000 [1:14:55<3:26:47, 4.23s/it]
27%|██▋ | 1068/4000 [1:15:00<3:25:46, 4.21s/it]
27%|██▋ | 1069/4000 [1:15:04<3:24:54, 4.19s/it]
27%|██▋ | 1070/4000 [1:15:08<3:24:18, 4.18s/it]
{'loss': '0.5665', 'grad_norm': '0.1419', 'learning_rate': '0.0004441', 'epoch': '0.2593'}
27%|██▋ | 1070/4000 [1:15:08<3:24:18, 4.18s/it]
27%|██▋ | 1071/4000 [1:15:13<3:29:25, 4.29s/it]
27%|██▋ | 1072/4000 [1:15:17<3:27:34, 4.25s/it]
27%|██▋ | 1073/4000 [1:15:21<3:26:40, 4.24s/it]
27%|██▋ | 1074/4000 [1:15:25<3:25:27, 4.21s/it]
27%|██▋ | 1075/4000 [1:15:29<3:24:49, 4.20s/it]
27%|██▋ | 1076/4000 [1:15:33<3:24:21, 4.19s/it]
27%|██▋ | 1077/4000 [1:15:38<3:24:22, 4.20s/it]
27%|██▋ | 1078/4000 [1:15:42<3:29:11, 4.30s/it]
27%|██▋ | 1079/4000 [1:15:46<3:27:32, 4.26s/it]
27%|██▋ | 1080/4000 [1:15:50<3:25:56, 4.23s/it]
{'loss': '0.5894', 'grad_norm': '0.1408', 'learning_rate': '0.0004426', 'epoch': '0.2617'}
27%|██▋ | 1080/4000 [1:15:50<3:25:56, 4.23s/it]
27%|██▋ | 1081/4000 [1:15:55<3:24:53, 4.21s/it]
27%|██▋ | 1082/4000 [1:15:59<3:23:50, 4.19s/it]
27%|██▋ | 1083/4000 [1:16:03<3:23:21, 4.18s/it]
27%|██▋ | 1084/4000 [1:16:07<3:23:32, 4.19s/it]
27%|██▋ | 1085/4000 [1:16:12<3:28:26, 4.29s/it]
27%|██▋ | 1086/4000 [1:16:16<3:26:16, 4.25s/it]
27%|██▋ | 1087/4000 [1:16:20<3:24:48, 4.22s/it]
27%|██▋ | 1088/4000 [1:16:24<3:23:57, 4.20s/it]
27%|██▋ | 1089/4000 [1:16:28<3:23:45, 4.20s/it]
27%|██▋ | 1090/4000 [1:16:33<3:23:45, 4.20s/it]
{'loss': '0.5948', 'grad_norm': '0.1558', 'learning_rate': '0.0004411', 'epoch': '0.2641'}
27%|██▋ | 1090/4000 [1:16:33<3:23:45, 4.20s/it]
27%|██▋ | 1091/4000 [1:16:37<3:26:06, 4.25s/it]
27%|██▋ | 1092/4000 [1:16:41<3:27:25, 4.28s/it]
27%|██▋ | 1093/4000 [1:16:45<3:25:39, 4.24s/it]
27%|██▋ | 1094/4000 [1:16:50<3:24:21, 4.22s/it]
27%|██▋ | 1095/4000 [1:16:54<3:23:10, 4.20s/it]
27%|██▋ | 1096/4000 [1:16:58<3:22:48, 4.19s/it]
27%|██▋ | 1097/4000 [1:17:02<3:22:44, 4.19s/it]
27%|██▋ | 1098/4000 [1:17:07<3:28:17, 4.31s/it]
27%|██▋ | 1099/4000 [1:17:11<3:26:16, 4.27s/it]
28%|██▊ | 1100/4000 [1:17:15<3:24:41, 4.24s/it]
{'loss': '0.5363', 'grad_norm': '0.2345', 'learning_rate': '0.0004395', 'epoch': '0.2665'}
28%|██▊ | 1100/4000 [1:17:15<3:24:41, 4.24s/it]
28%|██▊ | 1101/4000 [1:17:19<3:23:34, 4.21s/it]
28%|██▊ | 1102/4000 [1:17:23<3:22:49, 4.20s/it]
28%|██▊ | 1103/4000 [1:17:27<3:22:32, 4.19s/it]
28%|██▊ | 1104/4000 [1:17:32<3:22:08, 4.19s/it]
28%|██▊ | 1105/4000 [1:17:36<3:26:39, 4.28s/it]
28%|██▊ | 1106/4000 [1:17:40<3:24:41, 4.24s/it]
28%|██▊ | 1107/4000 [1:17:44<3:23:28, 4.22s/it]
28%|██▊ | 1108/4000 [1:17:49<3:22:22, 4.20s/it]
28%|██▊ | 1109/4000 [1:17:53<3:21:30, 4.18s/it]
28%|██▊ | 1110/4000 [1:17:57<3:21:02, 4.17s/it]
{'loss': '0.5761', 'grad_norm': '0.1357', 'learning_rate': '0.000438', 'epoch': '0.269'}
28%|██▊ | 1110/4000 [1:17:57<3:21:02, 4.17s/it]
28%|██▊ | 1111/4000 [1:18:01<3:20:43, 4.17s/it]
28%|██▊ | 1112/4000 [1:18:06<3:26:13, 4.28s/it]
28%|██▊ | 1113/4000 [1:18:10<3:24:13, 4.24s/it]
28%|██▊ | 1114/4000 [1:18:14<3:23:01, 4.22s/it]
28%|██▊ | 1115/4000 [1:18:18<3:22:10, 4.20s/it]
28%|██▊ | 1116/4000 [1:18:22<3:21:41, 4.20s/it]
28%|██▊ | 1117/4000 [1:18:26<3:21:20, 4.19s/it]
28%|██▊ | 1118/4000 [1:18:31<3:23:45, 4.24s/it]
28%|██▊ | 1119/4000 [1:18:35<3:25:22, 4.28s/it]
28%|██▊ | 1120/4000 [1:18:39<3:23:31, 4.24s/it]
{'loss': '0.5042', 'grad_norm': '0.119', 'learning_rate': '0.0004365', 'epoch': '0.2714'}
28%|██▊ | 1120/4000 [1:18:39<3:23:31, 4.24s/it]
28%|██▊ | 1121/4000 [1:18:44<3:22:14, 4.21s/it]
28%|██▊ | 1122/4000 [1:18:48<3:21:13, 4.20s/it]
28%|██▊ | 1123/4000 [1:18:52<3:20:30, 4.18s/it]
28%|██▊ | 1124/4000 [1:18:56<3:23:00, 4.24s/it]
28%|██▊ | 1125/4000 [1:19:01<3:27:01, 4.32s/it]
28%|██▊ | 1126/4000 [1:19:05<3:24:39, 4.27s/it]
28%|██▊ | 1127/4000 [1:19:09<3:22:43, 4.23s/it]
28%|██▊ | 1128/4000 [1:19:13<3:21:34, 4.21s/it]
28%|██▊ | 1129/4000 [1:19:17<3:20:55, 4.20s/it]
28%|██▊ | 1130/4000 [1:19:22<3:20:33, 4.19s/it]
{'loss': '0.6019', 'grad_norm': '0.1415', 'learning_rate': '0.000435', 'epoch': '0.2738'}
28%|██▊ | 1130/4000 [1:19:22<3:20:33, 4.19s/it]
28%|██▊ | 1131/4000 [1:19:26<3:19:55, 4.18s/it]
28%|██▊ | 1132/4000 [1:19:30<3:25:01, 4.29s/it]
28%|██▊ | 1133/4000 [1:19:34<3:23:10, 4.25s/it]
28%|██▊ | 1134/4000 [1:19:39<3:21:44, 4.22s/it]
28%|██▊ | 1135/4000 [1:19:43<3:20:48, 4.21s/it]
28%|██▊ | 1136/4000 [1:19:47<3:19:42, 4.18s/it]
28%|██▊ | 1137/4000 [1:19:51<3:19:15, 4.18s/it]
28%|██▊ | 1138/4000 [1:19:55<3:19:04, 4.17s/it]
28%|██▊ | 1139/4000 [1:20:00<3:24:02, 4.28s/it]
28%|██▊ | 1140/4000 [1:20:04<3:22:04, 4.24s/it]
{'loss': '0.5809', 'grad_norm': '0.1289', 'learning_rate': '0.0004335', 'epoch': '0.2762'}
28%|██▊ | 1140/4000 [1:20:04<3:22:04, 4.24s/it]
29%|██▊ | 1141/4000 [1:20:08<3:20:53, 4.22s/it]
29%|██▊ | 1142/4000 [1:20:12<3:19:56, 4.20s/it]
29%|██▊ | 1143/4000 [1:20:16<3:18:57, 4.18s/it]
29%|██▊ | 1144/4000 [1:20:20<3:18:23, 4.17s/it]
29%|██▊ | 1145/4000 [1:20:25<3:20:43, 4.22s/it]
29%|██▊ | 1146/4000 [1:20:29<3:22:21, 4.25s/it]
29%|██▊ | 1147/4000 [1:20:33<3:20:44, 4.22s/it]
29%|██▊ | 1148/4000 [1:20:37<3:19:33, 4.20s/it]
29%|██▊ | 1149/4000 [1:20:42<3:18:51, 4.18s/it]
29%|██▉ | 1150/4000 [1:20:46<3:18:19, 4.18s/it]
{'loss': '0.5696', 'grad_norm': '0.1419', 'learning_rate': '0.000432', 'epoch': '0.2786'}
29%|██▉ | 1150/4000 [1:20:46<3:18:19, 4.18s/it]
29%|██▉ | 1151/4000 [1:20:50<3:17:59, 4.17s/it]
29%|██▉ | 1152/4000 [1:20:54<3:23:10, 4.28s/it]
29%|██▉ | 1153/4000 [1:20:59<3:21:13, 4.24s/it]
29%|██▉ | 1154/4000 [1:21:03<3:19:54, 4.21s/it]
29%|██▉ | 1155/4000 [1:21:07<3:18:54, 4.19s/it]
29%|██▉ | 1156/4000 [1:21:11<3:18:08, 4.18s/it]
29%|██▉ | 1157/4000 [1:21:15<3:17:38, 4.17s/it]
29%|██▉ | 1158/4000 [1:21:19<3:17:23, 4.17s/it]
29%|██▉ | 1159/4000 [1:21:24<3:22:44, 4.28s/it]
29%|██▉ | 1160/4000 [1:21:28<3:20:43, 4.24s/it]
{'loss': '0.5416', 'grad_norm': '0.1512', 'learning_rate': '0.0004305', 'epoch': '0.2811'}
29%|██▉ | 1160/4000 [1:21:28<3:20:43, 4.24s/it]
29%|██▉ | 1161/4000 [1:21:32<3:19:19, 4.21s/it]
29%|██▉ | 1162/4000 [1:21:36<3:18:27, 4.20s/it]
29%|██▉ | 1163/4000 [1:21:40<3:17:48, 4.18s/it]
29%|██▉ | 1164/4000 [1:21:45<3:17:17, 4.17s/it]
29%|██▉ | 1165/4000 [1:21:49<3:16:48, 4.17s/it]
29%|██▉ | 1166/4000 [1:21:53<3:22:26, 4.29s/it]
29%|██▉ | 1167/4000 [1:21:57<3:20:54, 4.25s/it]
29%|██▉ | 1168/4000 [1:22:02<3:19:58, 4.24s/it]
29%|██▉ | 1169/4000 [1:22:06<3:18:47, 4.21s/it]
29%|██▉ | 1170/4000 [1:22:10<3:17:49, 4.19s/it]
{'loss': '0.5622', 'grad_norm': '0.1497', 'learning_rate': '0.0004289', 'epoch': '0.2835'}
29%|██▉ | 1170/4000 [1:22:10<3:17:49, 4.19s/it]
29%|██▉ | 1171/4000 [1:22:14<3:17:13, 4.18s/it]
29%|██▉ | 1172/4000 [1:22:19<3:21:58, 4.29s/it]
29%|██▉ | 1173/4000 [1:22:23<3:20:18, 4.25s/it]
29%|██▉ | 1174/4000 [1:22:27<3:18:46, 4.22s/it]
29%|██▉ | 1175/4000 [1:22:31<3:17:43, 4.20s/it]
29%|██▉ | 1176/4000 [1:22:35<3:17:01, 4.19s/it]
29%|██▉ | 1177/4000 [1:22:39<3:16:36, 4.18s/it]
29%|██▉ | 1178/4000 [1:22:44<3:16:44, 4.18s/it]
29%|██▉ | 1179/4000 [1:22:48<3:22:28, 4.31s/it]
30%|██▉ | 1180/4000 [1:22:52<3:20:41, 4.27s/it]
{'loss': '0.6044', 'grad_norm': '0.1439', 'learning_rate': '0.0004274', 'epoch': '0.2859'}
30%|██▉ | 1180/4000 [1:22:52<3:20:41, 4.27s/it]
30%|██▉ | 1181/4000 [1:22:57<3:18:56, 4.23s/it]
30%|██▉ | 1182/4000 [1:23:01<3:17:52, 4.21s/it]
30%|██▉ | 1183/4000 [1:23:05<3:17:08, 4.20s/it]
30%|██▉ | 1184/4000 [1:23:09<3:16:52, 4.19s/it]
30%|██▉ | 1185/4000 [1:23:13<3:16:52, 4.20s/it]
30%|██▉ | 1186/4000 [1:23:18<3:21:37, 4.30s/it]
30%|██▉ | 1187/4000 [1:23:22<3:19:35, 4.26s/it]
30%|██▉ | 1188/4000 [1:23:26<3:17:59, 4.22s/it]
30%|██▉ | 1189/4000 [1:23:30<3:16:59, 4.20s/it]
30%|██▉ | 1190/4000 [1:23:34<3:16:08, 4.19s/it]
{'loss': '0.5693', 'grad_norm': '0.153', 'learning_rate': '0.0004259', 'epoch': '0.2883'}
30%|██▉ | 1190/4000 [1:23:34<3:16:08, 4.19s/it]
30%|██▉ | 1191/4000 [1:23:39<3:15:39, 4.18s/it]
30%|██▉ | 1192/4000 [1:23:43<3:15:28, 4.18s/it]
30%|██▉ | 1193/4000 [1:23:47<3:20:37, 4.29s/it]
30%|██▉ | 1194/4000 [1:23:51<3:18:51, 4.25s/it]
30%|██▉ | 1195/4000 [1:23:56<3:17:26, 4.22s/it]
30%|██▉ | 1196/4000 [1:24:00<3:16:29, 4.20s/it]
30%|██▉ | 1197/4000 [1:24:04<3:15:49, 4.19s/it]
30%|██▉ | 1198/4000 [1:24:08<3:15:22, 4.18s/it]
30%|██▉ | 1199/4000 [1:24:13<3:20:43, 4.30s/it]
30%|███ | 1200/4000 [1:24:17<3:18:41, 4.26s/it]
{'loss': '0.5716', 'grad_norm': '0.1361', 'learning_rate': '0.0004244', 'epoch': '0.2908'}
30%|███ | 1200/4000 [1:24:17<3:18:41, 4.26s/it]
30%|███ | 1201/4000 [1:24:21<3:17:18, 4.23s/it]
30%|███ | 1202/4000 [1:24:25<3:16:15, 4.21s/it]
30%|███ | 1203/4000 [1:24:29<3:15:29, 4.19s/it]
30%|███ | 1204/4000 [1:24:33<3:14:50, 4.18s/it]
30%|███ | 1205/4000 [1:24:38<3:14:28, 4.17s/it]
30%|███ | 1206/4000 [1:24:42<3:19:28, 4.28s/it]
30%|███ | 1207/4000 [1:24:46<3:17:47, 4.25s/it]
30%|███ | 1208/4000 [1:24:51<3:16:19, 4.22s/it]
30%|███ | 1209/4000 [1:24:55<3:15:17, 4.20s/it]
30%|███ | 1210/4000 [1:24:59<3:14:36, 4.18s/it]
{'loss': '0.5381', 'grad_norm': '0.1753', 'learning_rate': '0.0004229', 'epoch': '0.2932'}
30%|███ | 1210/4000 [1:24:59<3:14:36, 4.18s/it]
30%|███ | 1211/4000 [1:25:03<3:14:05, 4.18s/it]
30%|███ | 1212/4000 [1:25:07<3:13:43, 4.17s/it]
30%|███ | 1213/4000 [1:25:12<3:18:58, 4.28s/it]
30%|███ | 1214/4000 [1:25:16<3:17:14, 4.25s/it]
30%|███ | 1215/4000 [1:25:20<3:15:55, 4.22s/it]
30%|███ | 1216/4000 [1:25:24<3:15:07, 4.21s/it]
30%|███ | 1217/4000 [1:25:28<3:14:18, 4.19s/it]
30%|███ | 1218/4000 [1:25:32<3:13:48, 4.18s/it]
30%|███ | 1219/4000 [1:25:37<3:16:05, 4.23s/it]
30%|███ | 1220/4000 [1:25:41<3:17:36, 4.27s/it]
{'loss': '0.5334', 'grad_norm': '0.1509', 'learning_rate': '0.0004214', 'epoch': '0.2956'}
30%|███ | 1220/4000 [1:25:41<3:17:36, 4.27s/it]
31%|███ | 1221/4000 [1:25:45<3:16:02, 4.23s/it]
31%|███ | 1222/4000 [1:25:50<3:15:23, 4.22s/it]
31%|███ | 1223/4000 [1:25:54<3:14:59, 4.21s/it]
31%|███ | 1224/4000 [1:25:58<3:13:58, 4.19s/it]
31%|███ | 1225/4000 [1:26:02<3:13:32, 4.18s/it]
31%|███ | 1226/4000 [1:26:07<3:18:29, 4.29s/it]
31%|███ | 1227/4000 [1:26:11<3:16:28, 4.25s/it]
31%|███ | 1228/4000 [1:26:15<3:15:00, 4.22s/it]
31%|███ | 1229/4000 [1:26:19<3:13:43, 4.19s/it]
31%|███ | 1230/4000 [1:26:23<3:12:57, 4.18s/it]
{'loss': '0.5645', 'grad_norm': '0.132', 'learning_rate': '0.0004198', 'epoch': '0.298'}
31%|███ | 1230/4000 [1:26:23<3:12:57, 4.18s/it]
31%|███ | 1231/4000 [1:26:27<3:12:27, 4.17s/it]
31%|███ | 1232/4000 [1:26:31<3:12:04, 4.16s/it]
31%|███ | 1233/4000 [1:26:36<3:16:45, 4.27s/it]
31%|███ | 1234/4000 [1:26:40<3:15:06, 4.23s/it]
31%|███ | 1235/4000 [1:26:44<3:13:55, 4.21s/it]
31%|███ | 1236/4000 [1:26:48<3:12:59, 4.19s/it]
31%|███ | 1237/4000 [1:26:53<3:12:23, 4.18s/it]
31%|███ | 1238/4000 [1:26:57<3:12:04, 4.17s/it]
31%|███ | 1239/4000 [1:27:01<3:11:49, 4.17s/it]
31%|███ | 1240/4000 [1:27:05<3:16:50, 4.28s/it]
{'loss': '0.5638', 'grad_norm': '0.1434', 'learning_rate': '0.0004183', 'epoch': '0.3005'}
31%|███ | 1240/4000 [1:27:05<3:16:50, 4.28s/it]
31%|███ | 1241/4000 [1:27:10<3:15:12, 4.25s/it]
31%|███ | 1242/4000 [1:27:14<3:14:02, 4.22s/it]
31%|███ | 1243/4000 [1:27:18<3:13:06, 4.20s/it]
31%|███ | 1244/4000 [1:27:22<3:12:21, 4.19s/it]
31%|███ | 1245/4000 [1:27:26<3:11:58, 4.18s/it]
31%|███ | 1246/4000 [1:27:31<3:14:23, 4.24s/it]
31%|███ | 1247/4000 [1:27:35<3:15:48, 4.27s/it]
31%|███ | 1248/4000 [1:27:39<3:14:10, 4.23s/it]
31%|███ | 1249/4000 [1:27:43<3:13:11, 4.21s/it]
31%|███▏ | 1250/4000 [1:27:47<3:12:14, 4.19s/it]
{'loss': '0.5903', 'grad_norm': '0.135', 'learning_rate': '0.0004168', 'epoch': '0.3029'}
31%|███▏ | 1250/4000 [1:27:47<3:12:14, 4.19s/it]
31%|███▏ | 1251/4000 [1:27:52<3:11:32, 4.18s/it]
31%|███▏ | 1252/4000 [1:27:56<3:11:20, 4.18s/it]
31%|███▏ | 1253/4000 [1:28:00<3:16:29, 4.29s/it]
31%|███▏ | 1254/4000 [1:28:04<3:14:34, 4.25s/it]
31%|███▏ | 1255/4000 [1:28:09<3:13:09, 4.22s/it]
31%|███▏ | 1256/4000 [1:28:13<3:12:15, 4.20s/it]
31%|███▏ | 1257/4000 [1:28:17<3:11:23, 4.19s/it]
31%|███▏ | 1258/4000 [1:28:21<3:10:46, 4.17s/it]
31%|███▏ | 1259/4000 [1:28:25<3:10:13, 4.16s/it]
32%|███▏ | 1260/4000 [1:28:30<3:15:17, 4.28s/it]
{'loss': '0.5562', 'grad_norm': '0.1464', 'learning_rate': '0.0004153', 'epoch': '0.3053'}
32%|███▏ | 1260/4000 [1:28:30<3:15:17, 4.28s/it]
32%|███▏ | 1261/4000 [1:28:34<3:13:37, 4.24s/it]
32%|███▏ | 1262/4000 [1:28:38<3:12:33, 4.22s/it]
32%|███▏ | 1263/4000 [1:28:42<3:11:38, 4.20s/it]
32%|███▏ | 1264/4000 [1:28:46<3:11:11, 4.19s/it]
32%|███▏ | 1265/4000 [1:28:51<3:10:40, 4.18s/it]
32%|███▏ | 1266/4000 [1:28:55<3:10:19, 4.18s/it]
32%|███▏ | 1267/4000 [1:28:59<3:15:34, 4.29s/it]
32%|███▏ | 1268/4000 [1:29:03<3:13:34, 4.25s/it]
32%|███▏ | 1269/4000 [1:29:08<3:11:59, 4.22s/it]
32%|███▏ | 1270/4000 [1:29:12<3:11:14, 4.20s/it]
{'loss': '0.5484', 'grad_norm': '0.1255', 'learning_rate': '0.0004138', 'epoch': '0.3077'}
32%|███▏ | 1270/4000 [1:29:12<3:11:14, 4.20s/it]
32%|███▏ | 1271/4000 [1:29:16<3:10:39, 4.19s/it]
32%|███▏ | 1272/4000 [1:29:20<3:10:05, 4.18s/it]
32%|███▏ | 1273/4000 [1:29:24<3:12:21, 4.23s/it]
32%|███▏ | 1274/4000 [1:29:29<3:13:37, 4.26s/it]
32%|███▏ | 1275/4000 [1:29:33<3:12:02, 4.23s/it]
32%|███▏ | 1276/4000 [1:29:37<3:10:52, 4.20s/it]
32%|███▏ | 1277/4000 [1:29:41<3:10:16, 4.19s/it]
32%|███▏ | 1278/4000 [1:29:45<3:09:38, 4.18s/it]
32%|███▏ | 1279/4000 [1:29:50<3:09:15, 4.17s/it]
32%|███▏ | 1280/4000 [1:29:54<3:14:27, 4.29s/it]
{'loss': '0.5281', 'grad_norm': '0.1772', 'learning_rate': '0.0004123', 'epoch': '0.3101'}
32%|███▏ | 1280/4000 [1:29:54<3:14:27, 4.29s/it]
32%|███▏ | 1281/4000 [1:29:58<3:12:35, 4.25s/it]
32%|███▏ | 1282/4000 [1:30:02<3:11:19, 4.22s/it]
32%|███▏ | 1283/4000 [1:30:07<3:10:09, 4.20s/it]
32%|███▏ | 1284/4000 [1:30:11<3:09:26, 4.18s/it]
32%|███▏ | 1285/4000 [1:30:15<3:09:02, 4.18s/it]
32%|███▏ | 1286/4000 [1:30:19<3:08:39, 4.17s/it]
32%|███▏ | 1287/4000 [1:30:24<3:13:15, 4.27s/it]
32%|███▏ | 1288/4000 [1:30:28<3:11:25, 4.24s/it]
32%|███▏ | 1289/4000 [1:30:32<3:10:37, 4.22s/it]
32%|███▏ | 1290/4000 [1:30:36<3:09:49, 4.20s/it]
{'loss': '0.5709', 'grad_norm': '0.1399', 'learning_rate': '0.0004108', 'epoch': '0.3126'}
32%|███▏ | 1290/4000 [1:30:36<3:09:49, 4.20s/it]
32%|███▏ | 1291/4000 [1:30:40<3:08:58, 4.19s/it]
32%|███▏ | 1292/4000 [1:30:44<3:08:50, 4.18s/it]
32%|███▏ | 1293/4000 [1:30:48<3:08:37, 4.18s/it]
32%|███▏ | 1294/4000 [1:30:53<3:13:00, 4.28s/it]
32%|███▏ | 1295/4000 [1:30:57<3:11:08, 4.24s/it]
32%|███▏ | 1296/4000 [1:31:01<3:09:49, 4.21s/it]
32%|███▏ | 1297/4000 [1:31:05<3:09:06, 4.20s/it]
32%|███▏ | 1298/4000 [1:31:10<3:08:33, 4.19s/it]
32%|███▏ | 1299/4000 [1:31:14<3:08:03, 4.18s/it]
32%|███▎ | 1300/4000 [1:31:18<3:13:12, 4.29s/it]
{'loss': '0.5659', 'grad_norm': '0.1292', 'learning_rate': '0.0004092', 'epoch': '0.315'}
32%|███▎ | 1300/4000 [1:31:18<3:13:12, 4.29s/it]
33%|███▎ | 1301/4000 [1:31:22<3:11:02, 4.25s/it]
33%|███▎ | 1302/4000 [1:31:27<3:09:43, 4.22s/it]
33%|███▎ | 1303/4000 [1:31:31<3:08:41, 4.20s/it]
33%|███▎ | 1304/4000 [1:31:35<3:07:56, 4.18s/it]
33%|███▎ | 1305/4000 [1:31:39<3:07:20, 4.17s/it]
33%|███▎ | 1306/4000 [1:31:43<3:07:05, 4.17s/it]
33%|███▎ | 1307/4000 [1:31:48<3:11:26, 4.27s/it]
33%|███▎ | 1308/4000 [1:31:52<3:09:45, 4.23s/it]
33%|███▎ | 1309/4000 [1:31:56<3:08:50, 4.21s/it]
33%|███▎ | 1310/4000 [1:32:00<3:08:12, 4.20s/it]
{'loss': '0.5598', 'grad_norm': '0.1478', 'learning_rate': '0.0004077', 'epoch': '0.3174'}
33%|███▎ | 1310/4000 [1:32:00<3:08:12, 4.20s/it]
33%|███▎ | 1311/4000 [1:32:04<3:07:36, 4.19s/it]
33%|███▎ | 1312/4000 [1:32:09<3:06:56, 4.17s/it]
33%|███▎ | 1313/4000 [1:32:13<3:06:32, 4.17s/it]
33%|███▎ | 1314/4000 [1:32:17<3:11:26, 4.28s/it]
33%|███▎ | 1315/4000 [1:32:21<3:09:48, 4.24s/it]
33%|███▎ | 1316/4000 [1:32:26<3:08:41, 4.22s/it]
33%|███▎ | 1317/4000 [1:32:30<3:08:08, 4.21s/it]
33%|███▎ | 1318/4000 [1:32:34<3:07:37, 4.20s/it]
33%|███▎ | 1319/4000 [1:32:38<3:07:28, 4.20s/it]
33%|███▎ | 1320/4000 [1:32:42<3:07:16, 4.19s/it]
{'loss': '0.5577', 'grad_norm': '0.1276', 'learning_rate': '0.0004062', 'epoch': '0.3198'}
33%|███▎ | 1320/4000 [1:32:42<3:07:16, 4.19s/it]
33%|███▎ | 1321/4000 [1:32:47<3:12:35, 4.31s/it]
33%|███▎ | 1322/4000 [1:32:51<3:10:50, 4.28s/it]
33%|███▎ | 1323/4000 [1:32:55<3:09:02, 4.24s/it]
33%|███▎ | 1324/4000 [1:32:59<3:08:02, 4.22s/it]
33%|███▎ | 1325/4000 [1:33:04<3:07:08, 4.20s/it]
33%|███▎ | 1326/4000 [1:33:08<3:06:27, 4.18s/it]
33%|███▎ | 1327/4000 [1:33:12<3:10:42, 4.28s/it]
33%|███▎ | 1328/4000 [1:33:16<3:08:56, 4.24s/it]
33%|███▎ | 1329/4000 [1:33:20<3:07:26, 4.21s/it]
33%|███▎ | 1330/4000 [1:33:25<3:06:31, 4.19s/it]
{'loss': '0.5051', 'grad_norm': '0.1239', 'learning_rate': '0.0004047', 'epoch': '0.3223'}
33%|███▎ | 1330/4000 [1:33:25<3:06:31, 4.19s/it]
33%|███▎ | 1331/4000 [1:33:29<3:05:53, 4.18s/it]
33%|███▎ | 1332/4000 [1:33:33<3:05:29, 4.17s/it]
33%|███▎ | 1333/4000 [1:33:37<3:05:08, 4.17s/it]
33%|███▎ | 1334/4000 [1:33:42<3:10:18, 4.28s/it]
33%|███▎ | 1335/4000 [1:33:46<3:08:34, 4.25s/it]
33%|███▎ | 1336/4000 [1:33:50<3:07:39, 4.23s/it]
33%|███▎ | 1337/4000 [1:33:54<3:06:46, 4.21s/it]
33%|███▎ | 1338/4000 [1:33:58<3:06:05, 4.19s/it]
33%|███▎ | 1339/4000 [1:34:02<3:05:37, 4.19s/it]
34%|███▎ | 1340/4000 [1:34:07<3:05:37, 4.19s/it]
{'loss': '0.5823', 'grad_norm': '0.1593', 'learning_rate': '0.0004032', 'epoch': '0.3247'}
34%|███▎ | 1340/4000 [1:34:07<3:05:37, 4.19s/it]
34%|███▎ | 1341/4000 [1:34:11<3:10:08, 4.29s/it]
34%|███▎ | 1342/4000 [1:34:15<3:08:29, 4.25s/it]
34%|███▎ | 1343/4000 [1:34:20<3:07:10, 4.23s/it]
34%|███▎ | 1344/4000 [1:34:24<3:06:25, 4.21s/it]
34%|███▎ | 1345/4000 [1:34:28<3:05:38, 4.20s/it]
34%|███▎ | 1346/4000 [1:34:32<3:04:57, 4.18s/it]
34%|███▎ | 1347/4000 [1:34:36<3:07:04, 4.23s/it]
34%|███▎ | 1348/4000 [1:34:41<3:08:25, 4.26s/it]
34%|███▎ | 1349/4000 [1:34:45<3:06:51, 4.23s/it]
34%|███▍ | 1350/4000 [1:34:49<3:05:57, 4.21s/it]
{'loss': '0.5374', 'grad_norm': '0.1937', 'learning_rate': '0.0004017', 'epoch': '0.3271'}
34%|███▍ | 1350/4000 [1:34:49<3:05:57, 4.21s/it]
34%|███▍ | 1351/4000 [1:34:53<3:05:16, 4.20s/it]
34%|███▍ | 1352/4000 [1:34:57<3:04:56, 4.19s/it]
34%|███▍ | 1353/4000 [1:35:01<3:04:29, 4.18s/it]
34%|███▍ | 1354/4000 [1:35:06<3:09:49, 4.30s/it]
34%|███▍ | 1355/4000 [1:35:10<3:08:05, 4.27s/it]
34%|███▍ | 1356/4000 [1:35:14<3:06:53, 4.24s/it]
34%|███▍ | 1357/4000 [1:35:19<3:05:48, 4.22s/it]
34%|███▍ | 1358/4000 [1:35:23<3:04:49, 4.20s/it]
34%|███▍ | 1359/4000 [1:35:27<3:04:28, 4.19s/it]
34%|███▍ | 1360/4000 [1:35:31<3:04:04, 4.18s/it]
{'loss': '0.5148', 'grad_norm': '0.1512', 'learning_rate': '0.0004002', 'epoch': '0.3295'}
34%|███▍ | 1360/4000 [1:35:31<3:04:04, 4.18s/it]
34%|███▍ | 1361/4000 [1:35:36<3:08:28, 4.29s/it]
34%|███▍ | 1362/4000 [1:35:40<3:06:54, 4.25s/it]
34%|███▍ | 1363/4000 [1:35:44<3:05:48, 4.23s/it]
34%|███▍ | 1364/4000 [1:35:48<3:04:53, 4.21s/it]
34%|███▍ | 1365/4000 [1:35:52<3:04:20, 4.20s/it]
34%|███▍ | 1366/4000 [1:35:56<3:04:03, 4.19s/it]
34%|███▍ | 1367/4000 [1:36:01<3:03:27, 4.18s/it]
34%|███▍ | 1368/4000 [1:36:05<3:08:30, 4.30s/it]
34%|███▍ | 1369/4000 [1:36:09<3:06:32, 4.25s/it]
34%|███▍ | 1370/4000 [1:36:14<3:05:21, 4.23s/it]
{'loss': '0.5666', 'grad_norm': '0.1293', 'learning_rate': '0.0003986', 'epoch': '0.332'}
34%|███▍ | 1370/4000 [1:36:14<3:05:21, 4.23s/it]
34%|███▍ | 1371/4000 [1:36:18<3:04:24, 4.21s/it]
34%|███▍ | 1372/4000 [1:36:22<3:03:47, 4.20s/it]
34%|███▍ | 1373/4000 [1:36:26<3:03:15, 4.19s/it]
34%|███▍ | 1374/4000 [1:36:30<3:05:20, 4.23s/it]
34%|███▍ | 1375/4000 [1:36:35<3:06:30, 4.26s/it]
34%|███▍ | 1376/4000 [1:36:39<3:05:02, 4.23s/it]
34%|███▍ | 1377/4000 [1:36:43<3:04:09, 4.21s/it]
34%|███▍ | 1378/4000 [1:36:47<3:03:36, 4.20s/it]
34%|███▍ | 1379/4000 [1:36:51<3:03:09, 4.19s/it]
34%|███▍ | 1380/4000 [1:36:56<3:02:41, 4.18s/it]
{'loss': '0.5444', 'grad_norm': '0.1459', 'learning_rate': '0.0003971', 'epoch': '0.3344'}
34%|███▍ | 1380/4000 [1:36:56<3:02:41, 4.18s/it]
35%|███▍ | 1381/4000 [1:37:00<3:07:06, 4.29s/it]
35%|███▍ | 1382/4000 [1:37:04<3:05:36, 4.25s/it]
35%|███▍ | 1383/4000 [1:37:08<3:04:19, 4.23s/it]
35%|███▍ | 1384/4000 [1:37:13<3:03:30, 4.21s/it]
35%|███▍ | 1385/4000 [1:37:17<3:02:49, 4.19s/it]
35%|███▍ | 1386/4000 [1:37:21<3:02:34, 4.19s/it]
35%|███▍ | 1387/4000 [1:37:25<3:02:11, 4.18s/it]
35%|███▍ | 1388/4000 [1:37:30<3:07:31, 4.31s/it]
35%|███▍ | 1389/4000 [1:37:34<3:05:43, 4.27s/it]
35%|███▍ | 1390/4000 [1:37:38<3:04:17, 4.24s/it]
{'loss': '0.5412', 'grad_norm': '0.1493', 'learning_rate': '0.0003956', 'epoch': '0.3368'}
35%|███▍ | 1390/4000 [1:37:38<3:04:17, 4.24s/it]
35%|███▍ | 1391/4000 [1:37:42<3:03:28, 4.22s/it]
35%|███▍ | 1392/4000 [1:37:46<3:02:50, 4.21s/it]
35%|███▍ | 1393/4000 [1:37:51<3:02:13, 4.19s/it]
35%|███▍ | 1394/4000 [1:37:55<3:01:49, 4.19s/it]
35%|███▍ | 1395/4000 [1:37:59<3:06:19, 4.29s/it]
35%|███▍ | 1396/4000 [1:38:03<3:04:32, 4.25s/it]
35%|███▍ | 1397/4000 [1:38:08<3:03:17, 4.23s/it]
35%|███▍ | 1398/4000 [1:38:12<3:03:07, 4.22s/it]
35%|███▍ | 1399/4000 [1:38:16<3:02:24, 4.21s/it]
35%|███▌ | 1400/4000 [1:38:20<3:01:40, 4.19s/it]
{'loss': '0.4633', 'grad_norm': '0.1313', 'learning_rate': '0.0003941', 'epoch': '0.3392'}
35%|███▌ | 1400/4000 [1:38:20<3:01:40, 4.19s/it]
35%|███▌ | 1401/4000 [1:38:24<3:03:57, 4.25s/it]
35%|███▌ | 1402/4000 [1:38:29<3:05:01, 4.27s/it]
35%|███▌ | 1403/4000 [1:38:33<3:03:36, 4.24s/it]
35%|███▌ | 1404/4000 [1:38:37<3:02:30, 4.22s/it]
35%|███▌ | 1405/4000 [1:38:41<3:01:36, 4.20s/it]
35%|███▌ | 1406/4000 [1:38:45<3:01:06, 4.19s/it]
35%|███▌ | 1407/4000 [1:38:50<3:00:54, 4.19s/it]
35%|███▌ | 1408/4000 [1:38:54<3:05:35, 4.30s/it]
35%|███▌ | 1409/4000 [1:38:58<3:03:52, 4.26s/it]
35%|███▌ | 1410/4000 [1:39:03<3:02:44, 4.23s/it]
{'loss': '0.5398', 'grad_norm': '0.1156', 'learning_rate': '0.0003926', 'epoch': '0.3416'}
35%|███▌ | 1410/4000 [1:39:03<3:02:44, 4.23s/it]
35%|███▌ | 1411/4000 [1:39:07<3:01:54, 4.22s/it]
35%|███▌ | 1412/4000 [1:39:11<3:01:19, 4.20s/it]
35%|███▌ | 1413/4000 [1:39:15<3:00:37, 4.19s/it]
35%|███▌ | 1414/4000 [1:39:19<3:00:07, 4.18s/it]
35%|███▌ | 1415/4000 [1:39:24<3:04:24, 4.28s/it]
35%|███▌ | 1416/4000 [1:39:28<3:03:04, 4.25s/it]
35%|███▌ | 1417/4000 [1:39:32<3:02:15, 4.23s/it]
35%|███▌ | 1418/4000 [1:39:36<3:01:25, 4.22s/it]
35%|███▌ | 1419/4000 [1:39:40<3:00:52, 4.20s/it]
36%|███▌ | 1420/4000 [1:39:45<3:00:18, 4.19s/it]
{'loss': '0.5687', 'grad_norm': '0.1501', 'learning_rate': '0.0003911', 'epoch': '0.3441'}
36%|███▌ | 1420/4000 [1:39:45<3:00:18, 4.19s/it]
36%|███▌ | 1421/4000 [1:39:49<2:59:46, 4.18s/it]
36%|███▌ | 1422/4000 [1:39:53<3:04:07, 4.29s/it]
36%|███▌ | 1423/4000 [1:39:57<3:02:43, 4.25s/it]
36%|███▌ | 1424/4000 [1:40:02<3:01:45, 4.23s/it]
36%|███▌ | 1425/4000 [1:40:06<3:00:46, 4.21s/it]
36%|███▌ | 1426/4000 [1:40:10<3:00:27, 4.21s/it]
36%|███▌ | 1427/4000 [1:40:14<2:59:49, 4.19s/it]
36%|███▌ | 1428/4000 [1:40:19<3:04:35, 4.31s/it]
36%|███▌ | 1429/4000 [1:40:23<3:03:11, 4.28s/it]
36%|███▌ | 1430/4000 [1:40:27<3:01:50, 4.25s/it]
{'loss': '0.5372', 'grad_norm': '0.2107', 'learning_rate': '0.0003895', 'epoch': '0.3465'}
36%|███▌ | 1430/4000 [1:40:27<3:01:50, 4.25s/it]
36%|███▌ | 1431/4000 [1:40:31<3:00:48, 4.22s/it]
36%|███▌ | 1432/4000 [1:40:35<3:00:05, 4.21s/it]
36%|███▌ | 1433/4000 [1:40:40<2:59:52, 4.20s/it]
36%|███▌ | 1434/4000 [1:40:44<2:59:05, 4.19s/it]
36%|███▌ | 1435/4000 [1:40:48<3:03:21, 4.29s/it]
36%|███▌ | 1436/4000 [1:40:53<3:01:49, 4.25s/it]
36%|███▌ | 1437/4000 [1:40:57<3:01:09, 4.24s/it]
36%|███▌ | 1438/4000 [1:41:01<3:00:21, 4.22s/it]
36%|███▌ | 1439/4000 [1:41:05<2:59:43, 4.21s/it]
36%|███▌ | 1440/4000 [1:41:09<2:59:14, 4.20s/it]
{'loss': '0.5215', 'grad_norm': '0.1397', 'learning_rate': '0.000388', 'epoch': '0.3489'}
36%|███▌ | 1440/4000 [1:41:09<2:59:14, 4.20s/it]
36%|███▌ | 1441/4000 [1:41:13<2:59:09, 4.20s/it]
36%|███▌ | 1442/4000 [1:41:18<3:03:28, 4.30s/it]
36%|███▌ | 1443/4000 [1:41:22<3:01:43, 4.26s/it]
36%|███▌ | 1444/4000 [1:41:26<3:00:29, 4.24s/it]
36%|███▌ | 1445/4000 [1:41:31<2:59:48, 4.22s/it]
36%|███▌ | 1446/4000 [1:41:35<2:59:05, 4.21s/it]
36%|███▌ | 1447/4000 [1:41:39<2:58:33, 4.20s/it]
36%|███▌ | 1448/4000 [1:41:43<2:57:53, 4.18s/it]
36%|███▌ | 1449/4000 [1:41:48<3:02:24, 4.29s/it]
36%|███▋ | 1450/4000 [1:41:52<3:00:33, 4.25s/it]
{'loss': '0.5467', 'grad_norm': '0.1459', 'learning_rate': '0.0003865', 'epoch': '0.3513'}
36%|███▋ | 1450/4000 [1:41:52<3:00:33, 4.25s/it]
36%|███▋ | 1451/4000 [1:41:56<2:59:14, 4.22s/it]
36%|███▋ | 1452/4000 [1:42:00<2:58:14, 4.20s/it]
36%|███▋ | 1453/4000 [1:42:04<2:58:03, 4.19s/it]
36%|███▋ | 1454/4000 [1:42:08<2:57:53, 4.19s/it]
36%|███▋ | 1455/4000 [1:42:13<3:03:02, 4.32s/it]
36%|███▋ | 1456/4000 [1:42:17<3:01:06, 4.27s/it]
36%|███▋ | 1457/4000 [1:42:21<2:59:43, 4.24s/it]
36%|███▋ | 1458/4000 [1:42:26<2:58:39, 4.22s/it]
36%|███▋ | 1459/4000 [1:42:30<2:57:46, 4.20s/it]
36%|███▋ | 1460/4000 [1:42:34<2:57:20, 4.19s/it]
{'loss': '0.533', 'grad_norm': '0.1424', 'learning_rate': '0.000385', 'epoch': '0.3538'}
36%|███▋ | 1460/4000 [1:42:34<2:57:20, 4.19s/it]
37%|███▋ | 1461/4000 [1:42:38<2:57:02, 4.18s/it]
37%|███▋ | 1462/4000 [1:42:42<3:00:43, 4.27s/it]
37%|███▋ | 1463/4000 [1:42:47<2:58:53, 4.23s/it]
37%|███▋ | 1464/4000 [1:42:51<2:57:42, 4.20s/it]
37%|███▋ | 1465/4000 [1:42:55<2:56:48, 4.18s/it]
37%|███▋ | 1466/4000 [1:42:59<2:56:23, 4.18s/it]
37%|███▋ | 1467/4000 [1:43:03<2:55:52, 4.17s/it]
37%|███▋ | 1468/4000 [1:43:07<2:55:33, 4.16s/it]
37%|███▋ | 1469/4000 [1:43:12<3:00:10, 4.27s/it]
37%|███▋ | 1470/4000 [1:43:16<2:58:41, 4.24s/it]
{'loss': '0.5525', 'grad_norm': '0.1409', 'learning_rate': '0.0003835', 'epoch': '0.3562'}
37%|███▋ | 1470/4000 [1:43:16<2:58:41, 4.24s/it]
37%|███▋ | 1471/4000 [1:43:20<2:57:29, 4.21s/it]
37%|███▋ | 1472/4000 [1:43:24<2:56:41, 4.19s/it]
37%|███▋ | 1473/4000 [1:43:28<2:56:03, 4.18s/it]
37%|███▋ | 1474/4000 [1:43:33<2:55:55, 4.18s/it]
37%|███▋ | 1475/4000 [1:43:37<2:58:04, 4.23s/it]
37%|███▋ | 1476/4000 [1:43:41<2:59:05, 4.26s/it]
37%|███▋ | 1477/4000 [1:43:45<2:57:52, 4.23s/it]
37%|███▋ | 1478/4000 [1:43:50<2:56:55, 4.21s/it]
37%|███▋ | 1479/4000 [1:43:54<2:56:22, 4.20s/it]
37%|███▋ | 1480/4000 [1:43:58<2:56:01, 4.19s/it]
{'loss': '0.5515', 'grad_norm': '0.1436', 'learning_rate': '0.000382', 'epoch': '0.3586'}
37%|███▋ | 1480/4000 [1:43:58<2:56:01, 4.19s/it]
37%|███▋ | 1481/4000 [1:44:02<2:55:44, 4.19s/it]
37%|███▋ | 1482/4000 [1:44:07<2:59:57, 4.29s/it]
37%|███▋ | 1483/4000 [1:44:11<2:58:25, 4.25s/it]
37%|███▋ | 1484/4000 [1:44:15<2:57:21, 4.23s/it]
37%|███▋ | 1485/4000 [1:44:19<2:56:15, 4.20s/it]
37%|███▋ | 1486/4000 [1:44:23<2:55:21, 4.18s/it]
37%|███▋ | 1487/4000 [1:44:27<2:54:59, 4.18s/it]
37%|███▋ | 1488/4000 [1:44:32<2:54:30, 4.17s/it]
37%|███▋ | 1489/4000 [1:44:36<2:58:47, 4.27s/it]
37%|███▋ | 1490/4000 [1:44:40<2:57:02, 4.23s/it]
{'loss': '0.499', 'grad_norm': '0.1194', 'learning_rate': '0.0003805', 'epoch': '0.361'}
37%|███▋ | 1490/4000 [1:44:40<2:57:02, 4.23s/it]
37%|███▋ | 1491/4000 [1:44:44<2:56:11, 4.21s/it]
37%|███▋ | 1492/4000 [1:44:49<2:55:21, 4.20s/it]
37%|███▋ | 1493/4000 [1:44:53<2:54:29, 4.18s/it]
37%|███▋ | 1494/4000 [1:44:57<2:54:00, 4.17s/it]
37%|███▋ | 1495/4000 [1:45:01<2:53:47, 4.16s/it]
37%|███▋ | 1496/4000 [1:45:06<2:58:02, 4.27s/it]
37%|███▋ | 1497/4000 [1:45:10<2:56:20, 4.23s/it]
37%|███▋ | 1498/4000 [1:45:14<2:55:11, 4.20s/it]
37%|███▋ | 1499/4000 [1:45:18<2:54:31, 4.19s/it]
38%|███▊ | 1500/4000 [1:45:22<2:53:50, 4.17s/it]
{'loss': '0.5304', 'grad_norm': '0.1319', 'learning_rate': '0.0003789', 'epoch': '0.3634'}
38%|███▊ | 1500/4000 [1:45:22<2:53:50, 4.17s/it]
38%|███▊ | 1501/4000 [1:45:26<2:53:21, 4.16s/it]
38%|███▊ | 1502/4000 [1:45:30<2:52:59, 4.16s/it]
38%|███▊ | 1503/4000 [1:45:35<2:57:05, 4.26s/it]
38%|███▊ | 1504/4000 [1:45:39<2:55:47, 4.23s/it]
38%|███▊ | 1505/4000 [1:45:43<2:54:44, 4.20s/it]
38%|███▊ | 1506/4000 [1:45:47<2:54:01, 4.19s/it]
38%|███▊ | 1507/4000 [1:45:51<2:53:26, 4.17s/it]
38%|███▊ | 1508/4000 [1:45:56<2:53:09, 4.17s/it]
38%|███▊ | 1509/4000 [1:46:00<2:57:26, 4.27s/it]
38%|███▊ | 1510/4000 [1:46:04<2:55:41, 4.23s/it]
{'loss': '0.4838', 'grad_norm': '0.1603', 'learning_rate': '0.0003774', 'epoch': '0.3659'}
38%|███▊ | 1510/4000 [1:46:04<2:55:41, 4.23s/it]
38%|███▊ | 1511/4000 [1:46:08<2:54:32, 4.21s/it]
38%|███▊ | 1512/4000 [1:46:13<2:53:58, 4.20s/it]
38%|███▊ | 1513/4000 [1:46:17<2:53:09, 4.18s/it]
38%|███▊ | 1514/4000 [1:46:21<2:52:39, 4.17s/it]
38%|███▊ | 1515/4000 [1:46:25<2:52:20, 4.16s/it]
38%|███▊ | 1516/4000 [1:46:30<2:56:48, 4.27s/it]
38%|███▊ | 1517/4000 [1:46:34<2:54:57, 4.23s/it]
38%|███▊ | 1518/4000 [1:46:38<2:53:46, 4.20s/it]
38%|███▊ | 1519/4000 [1:46:42<2:52:58, 4.18s/it]
38%|███▊ | 1520/4000 [1:46:46<2:52:23, 4.17s/it]
{'loss': '0.526', 'grad_norm': '0.1302', 'learning_rate': '0.0003759', 'epoch': '0.3683'}
38%|███▊ | 1520/4000 [1:46:46<2:52:23, 4.17s/it]
38%|███▊ | 1521/4000 [1:46:50<2:52:09, 4.17s/it]
38%|███▊ | 1522/4000 [1:46:54<2:51:36, 4.15s/it]
38%|███▊ | 1523/4000 [1:46:59<2:55:43, 4.26s/it]
38%|███▊ | 1524/4000 [1:47:03<2:54:07, 4.22s/it]
38%|███▊ | 1525/4000 [1:47:07<2:53:19, 4.20s/it]
38%|███▊ | 1526/4000 [1:47:11<2:52:28, 4.18s/it]
38%|███▊ | 1527/4000 [1:47:15<2:51:56, 4.17s/it]
38%|███▊ | 1528/4000 [1:47:20<2:51:35, 4.16s/it]
38%|███▊ | 1529/4000 [1:47:24<2:53:40, 4.22s/it]
38%|███▊ | 1530/4000 [1:47:28<2:55:05, 4.25s/it]
{'loss': '0.5107', 'grad_norm': '0.1193', 'learning_rate': '0.0003744', 'epoch': '0.3707'}
38%|███▊ | 1530/4000 [1:47:28<2:55:05, 4.25s/it]
38%|███▊ | 1531/4000 [1:47:32<2:53:45, 4.22s/it]
38%|███▊ | 1532/4000 [1:47:37<2:52:52, 4.20s/it]
38%|███▊ | 1533/4000 [1:47:41<2:52:19, 4.19s/it]
38%|███▊ | 1534/4000 [1:47:45<2:51:37, 4.18s/it]
38%|███▊ | 1535/4000 [1:47:49<2:51:21, 4.17s/it]
38%|███▊ | 1536/4000 [1:47:54<2:55:36, 4.28s/it]
38%|███▊ | 1537/4000 [1:47:58<2:54:31, 4.25s/it]
38%|███▊ | 1538/4000 [1:48:02<2:53:15, 4.22s/it]
38%|███▊ | 1539/4000 [1:48:06<2:52:28, 4.21s/it]
38%|███▊ | 1540/4000 [1:48:10<2:51:50, 4.19s/it]
{'loss': '0.4817', 'grad_norm': '0.1222', 'learning_rate': '0.0003729', 'epoch': '0.3731'}
38%|███▊ | 1540/4000 [1:48:10<2:51:50, 4.19s/it]
39%|███▊ | 1541/4000 [1:48:14<2:51:21, 4.18s/it]
39%|███▊ | 1542/4000 [1:48:19<2:51:08, 4.18s/it]
39%|███▊ | 1543/4000 [1:48:23<2:55:14, 4.28s/it]
39%|███▊ | 1544/4000 [1:48:27<2:53:45, 4.24s/it]
39%|███▊ | 1545/4000 [1:48:31<2:52:31, 4.22s/it]
39%|███▊ | 1546/4000 [1:48:36<2:51:49, 4.20s/it]
39%|███▊ | 1547/4000 [1:48:40<2:51:25, 4.19s/it]
39%|███▊ | 1548/4000 [1:48:44<2:50:41, 4.18s/it]
39%|███▊ | 1549/4000 [1:48:48<2:50:12, 4.17s/it]
39%|███▉ | 1550/4000 [1:48:53<2:54:56, 4.28s/it]
{'loss': '0.4882', 'grad_norm': '0.1181', 'learning_rate': '0.0003714', 'epoch': '0.3756'}
39%|███▉ | 1550/4000 [1:48:53<2:54:56, 4.28s/it]
39%|███▉ | 1551/4000 [1:48:57<2:53:26, 4.25s/it]
39%|███▉ | 1552/4000 [1:49:01<2:52:13, 4.22s/it]
39%|███▉ | 1553/4000 [1:49:05<2:51:08, 4.20s/it]
39%|███▉ | 1554/4000 [1:49:09<2:50:39, 4.19s/it]
39%|███▉ | 1555/4000 [1:49:13<2:50:09, 4.18s/it]
39%|███▉ | 1556/4000 [1:49:18<2:54:35, 4.29s/it]
39%|███▉ | 1557/4000 [1:49:22<2:53:04, 4.25s/it]
39%|███▉ | 1558/4000 [1:49:26<2:51:54, 4.22s/it]
39%|███▉ | 1559/4000 [1:49:30<2:51:07, 4.21s/it]
39%|███▉ | 1560/4000 [1:49:35<2:50:37, 4.20s/it]
{'loss': '0.5387', 'grad_norm': '0.1452', 'learning_rate': '0.0003698', 'epoch': '0.378'}
39%|███▉ | 1560/4000 [1:49:35<2:50:37, 4.20s/it]
39%|███▉ | 1561/4000 [1:49:39<2:50:13, 4.19s/it]
39%|███▉ | 1562/4000 [1:49:43<2:49:47, 4.18s/it]
39%|███▉ | 1563/4000 [1:49:47<2:53:03, 4.26s/it]
39%|███▉ | 1564/4000 [1:49:52<2:51:45, 4.23s/it]
39%|███▉ | 1565/4000 [1:49:56<2:50:53, 4.21s/it]
39%|███▉ | 1566/4000 [1:50:00<2:50:10, 4.20s/it]
39%|███▉ | 1567/4000 [1:50:04<2:49:34, 4.18s/it]
39%|███▉ | 1568/4000 [1:50:08<2:49:00, 4.17s/it]
39%|███▉ | 1569/4000 [1:50:12<2:48:35, 4.16s/it]
39%|███▉ | 1570/4000 [1:50:17<2:53:18, 4.28s/it]
{'loss': '0.5409', 'grad_norm': '0.146', 'learning_rate': '0.0003683', 'epoch': '0.3804'}
39%|███▉ | 1570/4000 [1:50:17<2:53:18, 4.28s/it]
39%|███▉ | 1571/4000 [1:50:21<2:51:44, 4.24s/it]
39%|███▉ | 1572/4000 [1:50:25<2:50:34, 4.22s/it]
39%|███▉ | 1573/4000 [1:50:29<2:49:43, 4.20s/it]
39%|███▉ | 1574/4000 [1:50:33<2:49:01, 4.18s/it]
39%|███▉ | 1575/4000 [1:50:38<2:48:29, 4.17s/it]
39%|███▉ | 1576/4000 [1:50:42<2:48:14, 4.16s/it]
39%|███▉ | 1577/4000 [1:50:46<2:52:17, 4.27s/it]
39%|███▉ | 1578/4000 [1:50:50<2:50:40, 4.23s/it]
39%|███▉ | 1579/4000 [1:50:55<2:49:38, 4.20s/it]
40%|███▉ | 1580/4000 [1:50:59<2:49:27, 4.20s/it]
{'loss': '0.5167', 'grad_norm': '0.1351', 'learning_rate': '0.0003668', 'epoch': '0.3828'}
40%|███▉ | 1580/4000 [1:50:59<2:49:27, 4.20s/it]
40%|███▉ | 1581/4000 [1:51:03<2:48:52, 4.19s/it]
40%|███▉ | 1582/4000 [1:51:07<2:48:43, 4.19s/it]
40%|███▉ | 1583/4000 [1:51:11<2:50:52, 4.24s/it]
40%|███▉ | 1584/4000 [1:51:16<2:51:54, 4.27s/it]
40%|███▉ | 1585/4000 [1:51:20<2:50:23, 4.23s/it]
40%|███▉ | 1586/4000 [1:51:24<2:49:28, 4.21s/it]
40%|███▉ | 1587/4000 [1:51:28<2:48:58, 4.20s/it]
40%|███▉ | 1588/4000 [1:51:32<2:48:34, 4.19s/it]
40%|███▉ | 1589/4000 [1:51:37<2:48:10, 4.19s/it]
40%|███▉ | 1590/4000 [1:51:41<2:52:44, 4.30s/it]
{'loss': '0.5061', 'grad_norm': '0.1514', 'learning_rate': '0.0003653', 'epoch': '0.3853'}
40%|███▉ | 1590/4000 [1:51:41<2:52:44, 4.30s/it]
40%|███▉ | 1591/4000 [1:51:45<2:51:08, 4.26s/it]
40%|███▉ | 1592/4000 [1:51:50<2:50:03, 4.24s/it]
40%|███▉ | 1593/4000 [1:51:54<2:49:09, 4.22s/it]
40%|███▉ | 1594/4000 [1:51:58<2:48:26, 4.20s/it]
40%|███▉ | 1595/4000 [1:52:02<2:47:52, 4.19s/it]
40%|███▉ | 1596/4000 [1:52:06<2:47:36, 4.18s/it]
40%|███▉ | 1597/4000 [1:52:11<2:51:32, 4.28s/it]
40%|███▉ | 1598/4000 [1:52:15<2:50:16, 4.25s/it]
40%|███▉ | 1599/4000 [1:52:19<2:49:06, 4.23s/it]
40%|████ | 1600/4000 [1:52:23<2:48:45, 4.22s/it]
{'loss': '0.4952', 'grad_norm': '0.1838', 'learning_rate': '0.0003638', 'epoch': '0.3877'}
40%|████ | 1600/4000 [1:52:23<2:48:45, 4.22s/it]
40%|████ | 1601/4000 [1:52:27<2:48:25, 4.21s/it]
40%|████ | 1602/4000 [1:52:32<2:47:50, 4.20s/it]
40%|████ | 1603/4000 [1:52:36<2:47:32, 4.19s/it]
40%|████ | 1604/4000 [1:52:40<2:51:40, 4.30s/it]
40%|████ | 1605/4000 [1:52:45<2:49:58, 4.26s/it]
40%|████ | 1606/4000 [1:52:49<2:48:46, 4.23s/it]
40%|████ | 1607/4000 [1:52:53<2:48:03, 4.21s/it]
40%|████ | 1608/4000 [1:52:57<2:47:17, 4.20s/it]
40%|████ | 1609/4000 [1:53:01<2:46:55, 4.19s/it]
40%|████ | 1610/4000 [1:53:06<2:51:21, 4.30s/it]
{'loss': '0.5098', 'grad_norm': '0.1327', 'learning_rate': '0.0003623', 'epoch': '0.3901'}
40%|████ | 1610/4000 [1:53:06<2:51:21, 4.30s/it]
40%|████ | 1611/4000 [1:53:10<2:49:40, 4.26s/it]
40%|████ | 1612/4000 [1:53:14<2:48:28, 4.23s/it]
40%|████ | 1613/4000 [1:53:18<2:47:42, 4.22s/it]
40%|████ | 1614/4000 [1:53:22<2:47:12, 4.20s/it]
40%|████ | 1615/4000 [1:53:27<2:46:36, 4.19s/it]
40%|████ | 1616/4000 [1:53:31<2:46:16, 4.18s/it]
40%|████ | 1617/4000 [1:53:35<2:50:41, 4.30s/it]
40%|████ | 1618/4000 [1:53:39<2:49:09, 4.26s/it]
40%|████ | 1619/4000 [1:53:44<2:47:56, 4.23s/it]
40%|████ | 1620/4000 [1:53:48<2:47:01, 4.21s/it]
{'loss': '0.4661', 'grad_norm': '0.1156', 'learning_rate': '0.0003608', 'epoch': '0.3925'}
40%|████ | 1620/4000 [1:53:48<2:47:01, 4.21s/it]
41%|████ | 1621/4000 [1:53:52<2:46:22, 4.20s/it]
41%|████ | 1622/4000 [1:53:56<2:46:04, 4.19s/it]
41%|████ | 1623/4000 [1:54:00<2:45:44, 4.18s/it]
41%|████ | 1624/4000 [1:54:05<2:49:56, 4.29s/it]
41%|████ | 1625/4000 [1:54:09<2:48:28, 4.26s/it]
41%|████ | 1626/4000 [1:54:13<2:47:26, 4.23s/it]
41%|████ | 1627/4000 [1:54:17<2:46:54, 4.22s/it]
41%|████ | 1628/4000 [1:54:22<2:46:15, 4.21s/it]
41%|████ | 1629/4000 [1:54:26<2:45:42, 4.19s/it]
41%|████ | 1630/4000 [1:54:30<2:47:24, 4.24s/it]
{'loss': '0.4574', 'grad_norm': '0.1199', 'learning_rate': '0.0003592', 'epoch': '0.3949'}
41%|████ | 1630/4000 [1:54:30<2:47:24, 4.24s/it]
41%|████ | 1631/4000 [1:54:34<2:48:32, 4.27s/it]
41%|████ | 1632/4000 [1:54:39<2:47:20, 4.24s/it]
41%|████ | 1633/4000 [1:54:43<2:46:25, 4.22s/it]
41%|████ | 1634/4000 [1:54:47<2:45:50, 4.21s/it]
41%|████ | 1635/4000 [1:54:51<2:45:24, 4.20s/it]
41%|████ | 1636/4000 [1:54:55<2:44:59, 4.19s/it]
41%|████ | 1637/4000 [1:55:00<2:49:52, 4.31s/it]
41%|████ | 1638/4000 [1:55:04<2:48:14, 4.27s/it]
41%|████ | 1639/4000 [1:55:08<2:47:02, 4.25s/it]
41%|████ | 1640/4000 [1:55:12<2:46:20, 4.23s/it]
{'loss': '0.4998', 'grad_norm': '0.2282', 'learning_rate': '0.0003577', 'epoch': '0.3974'}
41%|████ | 1640/4000 [1:55:12<2:46:20, 4.23s/it]
41%|████ | 1641/4000 [1:55:17<2:45:41, 4.21s/it]
41%|████ | 1642/4000 [1:55:21<2:45:08, 4.20s/it]
41%|████ | 1643/4000 [1:55:25<2:44:36, 4.19s/it]
41%|████ | 1644/4000 [1:55:29<2:48:21, 4.29s/it]
41%|████ | 1645/4000 [1:55:34<2:47:04, 4.26s/it]
41%|████ | 1646/4000 [1:55:38<2:45:59, 4.23s/it]
41%|████ | 1647/4000 [1:55:42<2:45:12, 4.21s/it]
41%|████ | 1648/4000 [1:55:46<2:44:26, 4.19s/it]
41%|████ | 1649/4000 [1:55:50<2:43:55, 4.18s/it]
41%|████▏ | 1650/4000 [1:55:54<2:43:36, 4.18s/it]
{'loss': '0.5091', 'grad_norm': '0.1964', 'learning_rate': '0.0003562', 'epoch': '0.3998'}
41%|████▏ | 1650/4000 [1:55:54<2:43:36, 4.18s/it]
41%|████▏ | 1651/4000 [1:55:59<2:47:47, 4.29s/it]
41%|████▏ | 1652/4000 [1:56:03<2:46:11, 4.25s/it]
41%|████▏ | 1653/4000 [1:56:07<2:45:02, 4.22s/it]
41%|████▏ | 1654/4000 [1:56:11<2:44:23, 4.20s/it]
41%|████▏ | 1655/4000 [1:56:16<2:44:10, 4.20s/it]
41%|████▏ | 1656/4000 [1:56:20<2:44:14, 4.20s/it]
41%|████▏ | 1657/4000 [1:56:24<2:46:26, 4.26s/it]
41%|████▏ | 1658/4000 [1:56:29<2:47:59, 4.30s/it]
41%|████▏ | 1659/4000 [1:56:33<2:46:20, 4.26s/it]
42%|████▏ | 1660/4000 [1:56:37<2:45:39, 4.25s/it]
{'loss': '0.483', 'grad_norm': '0.1286', 'learning_rate': '0.0003547', 'epoch': '0.4022'}
42%|████▏ | 1660/4000 [1:56:37<2:45:39, 4.25s/it]
42%|████▏ | 1661/4000 [1:56:41<2:44:39, 4.22s/it]
42%|████▏ | 1662/4000 [1:56:45<2:44:12, 4.21s/it]
42%|████▏ | 1663/4000 [1:56:50<2:43:37, 4.20s/it]
42%|████▏ | 1664/4000 [1:56:54<2:47:16, 4.30s/it]
42%|████▏ | 1665/4000 [1:56:58<2:45:41, 4.26s/it]
42%|████▏ | 1666/4000 [1:57:02<2:44:28, 4.23s/it]
42%|████▏ | 1667/4000 [1:57:07<2:43:47, 4.21s/it]
42%|████▏ | 1668/4000 [1:57:11<2:43:24, 4.20s/it]
42%|████▏ | 1669/4000 [1:57:15<2:43:14, 4.20s/it]
42%|████▏ | 1670/4000 [1:57:19<2:43:14, 4.20s/it]
{'loss': '0.4756', 'grad_norm': '0.1426', 'learning_rate': '0.0003532', 'epoch': '0.4046'}
42%|████▏ | 1670/4000 [1:57:19<2:43:14, 4.20s/it]
42%|████▏ | 1671/4000 [1:57:24<2:47:13, 4.31s/it]
42%|████▏ | 1672/4000 [1:57:28<2:45:51, 4.27s/it]
42%|████▏ | 1673/4000 [1:57:32<2:44:47, 4.25s/it]
42%|████▏ | 1674/4000 [1:57:36<2:44:10, 4.23s/it]
42%|████▏ | 1675/4000 [1:57:41<2:43:14, 4.21s/it]
42%|████▏ | 1676/4000 [1:57:45<2:42:46, 4.20s/it]
42%|████▏ | 1677/4000 [1:57:49<2:42:16, 4.19s/it]
42%|████▏ | 1678/4000 [1:57:53<2:46:19, 4.30s/it]
42%|████▏ | 1679/4000 [1:57:58<2:44:33, 4.25s/it]
42%|████▏ | 1680/4000 [1:58:02<2:43:17, 4.22s/it]
{'loss': '0.5059', 'grad_norm': '0.1299', 'learning_rate': '0.0003517', 'epoch': '0.4071'}
42%|████▏ | 1680/4000 [1:58:02<2:43:17, 4.22s/it]
42%|████▏ | 1681/4000 [1:58:06<2:42:37, 4.21s/it]
42%|████▏ | 1682/4000 [1:58:10<2:42:04, 4.20s/it]
42%|████▏ | 1683/4000 [1:58:14<2:41:25, 4.18s/it]
42%|████▏ | 1684/4000 [1:58:19<2:43:05, 4.23s/it]
42%|████▏ | 1685/4000 [1:58:23<2:44:37, 4.27s/it]
42%|████▏ | 1686/4000 [1:58:27<2:43:12, 4.23s/it]
42%|████▏ | 1687/4000 [1:58:31<2:42:22, 4.21s/it]
42%|████▏ | 1688/4000 [1:58:35<2:41:53, 4.20s/it]
42%|████▏ | 1689/4000 [1:58:40<2:41:32, 4.19s/it]
42%|████▏ | 1690/4000 [1:58:44<2:41:12, 4.19s/it]
{'loss': '0.5031', 'grad_norm': '0.1176', 'learning_rate': '0.0003502', 'epoch': '0.4095'}
42%|████▏ | 1690/4000 [1:58:44<2:41:12, 4.19s/it]
42%|████▏ | 1691/4000 [1:58:48<2:45:12, 4.29s/it]
42%|████▏ | 1692/4000 [1:58:52<2:43:40, 4.25s/it]
42%|████▏ | 1693/4000 [1:58:57<2:42:20, 4.22s/it]
42%|████▏ | 1694/4000 [1:59:01<2:41:36, 4.21s/it]
42%|████▏ | 1695/4000 [1:59:05<2:41:00, 4.19s/it]
42%|████▏ | 1696/4000 [1:59:09<2:40:41, 4.18s/it]
42%|████▏ | 1697/4000 [1:59:13<2:40:38, 4.19s/it]
42%|████▏ | 1698/4000 [1:59:18<2:44:54, 4.30s/it]
42%|████▏ | 1699/4000 [1:59:22<2:43:25, 4.26s/it]
42%|████▎ | 1700/4000 [1:59:26<2:42:22, 4.24s/it]
{'loss': '0.4895', 'grad_norm': '0.1596', 'learning_rate': '0.0003486', 'epoch': '0.4119'}
42%|████▎ | 1700/4000 [1:59:26<2:42:22, 4.24s/it]
43%|████▎ | 1701/4000 [1:59:30<2:41:51, 4.22s/it]
43%|████▎ | 1702/4000 [1:59:35<2:41:48, 4.22s/it]
43%|████▎ | 1703/4000 [1:59:39<2:41:19, 4.21s/it]
43%|████▎ | 1704/4000 [1:59:43<2:40:46, 4.20s/it]
43%|████▎ | 1705/4000 [1:59:47<2:44:36, 4.30s/it]
43%|████▎ | 1706/4000 [1:59:52<2:43:00, 4.26s/it]
43%|████▎ | 1707/4000 [1:59:56<2:42:04, 4.24s/it]
43%|████▎ | 1708/4000 [2:00:00<2:41:44, 4.23s/it]
43%|████▎ | 1709/4000 [2:00:04<2:40:57, 4.22s/it]
43%|████▎ | 1710/4000 [2:00:08<2:40:21, 4.20s/it]
{'loss': '0.5079', 'grad_norm': '0.122', 'learning_rate': '0.0003471', 'epoch': '0.4143'}
43%|████▎ | 1710/4000 [2:00:08<2:40:21, 4.20s/it]
43%|████▎ | 1711/4000 [2:00:13<2:43:37, 4.29s/it]
43%|████▎ | 1712/4000 [2:00:17<2:42:24, 4.26s/it]
43%|████▎ | 1713/4000 [2:00:21<2:41:18, 4.23s/it]
43%|████▎ | 1714/4000 [2:00:25<2:40:47, 4.22s/it]
43%|████▎ | 1715/4000 [2:00:30<2:40:00, 4.20s/it]
43%|████▎ | 1716/4000 [2:00:34<2:39:21, 4.19s/it]
43%|████▎ | 1717/4000 [2:00:38<2:39:27, 4.19s/it]
43%|████▎ | 1718/4000 [2:00:43<2:43:37, 4.30s/it]
43%|████▎ | 1719/4000 [2:00:47<2:41:57, 4.26s/it]
43%|████▎ | 1720/4000 [2:00:51<2:40:46, 4.23s/it]
{'loss': '0.5074', 'grad_norm': '0.1177', 'learning_rate': '0.0003456', 'epoch': '0.4168'}
43%|████▎ | 1720/4000 [2:00:51<2:40:46, 4.23s/it]
43%|████▎ | 1721/4000 [2:00:55<2:40:08, 4.22s/it]
43%|████▎ | 1722/4000 [2:00:59<2:39:50, 4.21s/it]
43%|████▎ | 1723/4000 [2:01:03<2:39:39, 4.21s/it]
43%|████▎ | 1724/4000 [2:01:08<2:39:32, 4.21s/it]
43%|████▎ | 1725/4000 [2:01:12<2:43:34, 4.31s/it]
43%|████▎ | 1726/4000 [2:01:16<2:41:50, 4.27s/it]
43%|████▎ | 1727/4000 [2:01:21<2:40:29, 4.24s/it]
43%|████▎ | 1728/4000 [2:01:25<2:39:27, 4.21s/it]
43%|████▎ | 1729/4000 [2:01:29<2:38:48, 4.20s/it]
43%|████▎ | 1730/4000 [2:01:33<2:38:21, 4.19s/it]
{'loss': '0.5266', 'grad_norm': '0.1315', 'learning_rate': '0.0003441', 'epoch': '0.4192'}
43%|████▎ | 1730/4000 [2:01:33<2:38:21, 4.19s/it]
43%|████▎ | 1731/4000 [2:01:37<2:38:00, 4.18s/it]
43%|████▎ | 1732/4000 [2:01:42<2:41:57, 4.28s/it]
43%|████▎ | 1733/4000 [2:01:46<2:40:43, 4.25s/it]
43%|████▎ | 1734/4000 [2:01:50<2:39:53, 4.23s/it]
43%|████▎ | 1735/4000 [2:01:54<2:39:15, 4.22s/it]
43%|████▎ | 1736/4000 [2:01:58<2:38:55, 4.21s/it]
43%|████▎ | 1737/4000 [2:02:03<2:38:15, 4.20s/it]
43%|████▎ | 1738/4000 [2:02:07<2:42:17, 4.31s/it]
43%|████▎ | 1739/4000 [2:02:11<2:40:44, 4.27s/it]
44%|████▎ | 1740/4000 [2:02:16<2:39:53, 4.24s/it]
{'loss': '0.5235', 'grad_norm': '0.1266', 'learning_rate': '0.0003426', 'epoch': '0.4216'}
44%|████▎ | 1740/4000 [2:02:16<2:39:53, 4.24s/it]
44%|████▎ | 1741/4000 [2:02:20<2:38:56, 4.22s/it]
44%|████▎ | 1742/4000 [2:02:24<2:38:27, 4.21s/it]
44%|████▎ | 1743/4000 [2:02:28<2:37:55, 4.20s/it]
44%|████▎ | 1744/4000 [2:02:32<2:37:33, 4.19s/it]
44%|████▎ | 1745/4000 [2:02:37<2:41:18, 4.29s/it]
44%|████▎ | 1746/4000 [2:02:41<2:39:51, 4.26s/it]
44%|████▎ | 1747/4000 [2:02:45<2:38:47, 4.23s/it]
44%|████▎ | 1748/4000 [2:02:49<2:38:06, 4.21s/it]
44%|████▎ | 1749/4000 [2:02:53<2:37:37, 4.20s/it]
44%|████▍ | 1750/4000 [2:02:58<2:37:12, 4.19s/it]
{'loss': '0.489', 'grad_norm': '0.1482', 'learning_rate': '0.0003411', 'epoch': '0.424'}
44%|████▍ | 1750/4000 [2:02:58<2:37:12, 4.19s/it]
44%|████▍ | 1751/4000 [2:03:02<2:36:47, 4.18s/it]
44%|████▍ | 1752/4000 [2:03:06<2:40:49, 4.29s/it]
44%|████▍ | 1753/4000 [2:03:10<2:39:27, 4.26s/it]
44%|████▍ | 1754/4000 [2:03:15<2:38:25, 4.23s/it]
44%|████▍ | 1755/4000 [2:03:19<2:37:37, 4.21s/it]
44%|████▍ | 1756/4000 [2:03:23<2:37:22, 4.21s/it]
44%|████▍ | 1757/4000 [2:03:27<2:36:51, 4.20s/it]
44%|████▍ | 1758/4000 [2:03:32<2:38:31, 4.24s/it]
44%|████▍ | 1759/4000 [2:03:36<2:40:17, 4.29s/it]
44%|████▍ | 1760/4000 [2:03:40<2:39:01, 4.26s/it]
{'loss': '0.529', 'grad_norm': '0.1371', 'learning_rate': '0.0003395', 'epoch': '0.4264'}
44%|████▍ | 1760/4000 [2:03:40<2:39:01, 4.26s/it]
44%|████▍ | 1761/4000 [2:03:44<2:38:16, 4.24s/it]
44%|████▍ | 1762/4000 [2:03:48<2:37:08, 4.21s/it]
44%|████▍ | 1763/4000 [2:03:53<2:36:45, 4.20s/it]
44%|████▍ | 1764/4000 [2:03:57<2:36:14, 4.19s/it]
44%|████▍ | 1765/4000 [2:04:01<2:39:59, 4.29s/it]
44%|████▍ | 1766/4000 [2:04:06<2:38:28, 4.26s/it]
44%|████▍ | 1767/4000 [2:04:10<2:37:19, 4.23s/it]
44%|████▍ | 1768/4000 [2:04:14<2:36:47, 4.21s/it]
44%|████▍ | 1769/4000 [2:04:18<2:36:14, 4.20s/it]
44%|████▍ | 1770/4000 [2:04:22<2:35:43, 4.19s/it]
{'loss': '0.4912', 'grad_norm': '0.1379', 'learning_rate': '0.000338', 'epoch': '0.4289'}
44%|████▍ | 1770/4000 [2:04:22<2:35:43, 4.19s/it]
44%|████▍ | 1771/4000 [2:04:26<2:35:28, 4.19s/it]
44%|████▍ | 1772/4000 [2:04:31<2:39:20, 4.29s/it]
44%|████▍ | 1773/4000 [2:04:35<2:37:50, 4.25s/it]
44%|████▍ | 1774/4000 [2:04:39<2:36:54, 4.23s/it]
44%|████▍ | 1775/4000 [2:04:43<2:36:04, 4.21s/it]
44%|████▍ | 1776/4000 [2:04:48<2:35:31, 4.20s/it]
44%|████▍ | 1777/4000 [2:04:52<2:35:03, 4.19s/it]
44%|████▍ | 1778/4000 [2:04:56<2:34:51, 4.18s/it]
44%|████▍ | 1779/4000 [2:05:00<2:38:48, 4.29s/it]
44%|████▍ | 1780/4000 [2:05:05<2:37:28, 4.26s/it]
{'loss': '0.4496', 'grad_norm': '0.1109', 'learning_rate': '0.0003365', 'epoch': '0.4313'}
44%|████▍ | 1780/4000 [2:05:05<2:37:28, 4.26s/it]
45%|████▍ | 1781/4000 [2:05:09<2:36:28, 4.23s/it]
45%|████▍ | 1782/4000 [2:05:13<2:35:42, 4.21s/it]
45%|████▍ | 1783/4000 [2:05:17<2:35:08, 4.20s/it]
45%|████▍ | 1784/4000 [2:05:21<2:34:35, 4.19s/it]
45%|████▍ | 1785/4000 [2:05:26<2:36:35, 4.24s/it]
45%|████▍ | 1786/4000 [2:05:30<2:37:37, 4.27s/it]
45%|████▍ | 1787/4000 [2:05:34<2:36:31, 4.24s/it]
45%|████▍ | 1788/4000 [2:05:38<2:35:36, 4.22s/it]
45%|████▍ | 1789/4000 [2:05:43<2:34:55, 4.20s/it]
45%|████▍ | 1790/4000 [2:05:47<2:34:27, 4.19s/it]
{'loss': '0.5288', 'grad_norm': '0.1151', 'learning_rate': '0.000335', 'epoch': '0.4337'}
45%|████▍ | 1790/4000 [2:05:47<2:34:27, 4.19s/it]
45%|████▍ | 1791/4000 [2:05:51<2:34:11, 4.19s/it]
45%|████▍ | 1792/4000 [2:05:55<2:37:52, 4.29s/it]
45%|████▍ | 1793/4000 [2:06:00<2:36:10, 4.25s/it]
45%|████▍ | 1794/4000 [2:06:04<2:35:14, 4.22s/it]
45%|████▍ | 1795/4000 [2:06:08<2:34:30, 4.20s/it]
45%|████▍ | 1796/4000 [2:06:12<2:34:07, 4.20s/it]
45%|████▍ | 1797/4000 [2:06:16<2:33:37, 4.18s/it]
45%|████▍ | 1798/4000 [2:06:20<2:33:11, 4.17s/it]
45%|████▍ | 1799/4000 [2:06:25<2:36:48, 4.27s/it]
45%|████▌ | 1800/4000 [2:06:29<2:35:40, 4.25s/it]
{'loss': '0.4617', 'grad_norm': '0.1257', 'learning_rate': '0.0003335', 'epoch': '0.4361'}
45%|████▌ | 1800/4000 [2:06:29<2:35:40, 4.25s/it]
45%|████▌ | 1801/4000 [2:06:33<2:34:39, 4.22s/it]
45%|████▌ | 1802/4000 [2:06:37<2:33:57, 4.20s/it]
45%|████▌ | 1803/4000 [2:06:42<2:33:33, 4.19s/it]
45%|████▌ | 1804/4000 [2:06:46<2:33:06, 4.18s/it]
45%|████▌ | 1805/4000 [2:06:50<2:32:48, 4.18s/it]
45%|████▌ | 1806/4000 [2:06:54<2:36:57, 4.29s/it]
45%|████▌ | 1807/4000 [2:06:59<2:35:36, 4.26s/it]
45%|████▌ | 1808/4000 [2:07:03<2:34:28, 4.23s/it]
45%|████▌ | 1809/4000 [2:07:07<2:33:44, 4.21s/it]
45%|████▌ | 1810/4000 [2:07:11<2:33:10, 4.20s/it]
{'loss': '0.4481', 'grad_norm': '0.1269', 'learning_rate': '0.000332', 'epoch': '0.4386'}
45%|████▌ | 1810/4000 [2:07:11<2:33:10, 4.20s/it]
45%|████▌ | 1811/4000 [2:07:15<2:32:58, 4.19s/it]
45%|████▌ | 1812/4000 [2:07:20<2:35:27, 4.26s/it]
45%|████▌ | 1813/4000 [2:07:24<2:37:04, 4.31s/it]
45%|████▌ | 1814/4000 [2:07:28<2:35:20, 4.26s/it]
45%|████▌ | 1815/4000 [2:07:32<2:34:13, 4.23s/it]
45%|████▌ | 1816/4000 [2:07:37<2:33:32, 4.22s/it]
45%|████▌ | 1817/4000 [2:07:41<2:33:09, 4.21s/it]
45%|████▌ | 1818/4000 [2:07:45<2:32:50, 4.20s/it]
45%|████▌ | 1819/4000 [2:07:50<2:36:24, 4.30s/it]
46%|████▌ | 1820/4000 [2:07:54<2:34:54, 4.26s/it]
{'loss': '0.5216', 'grad_norm': '0.2422', 'learning_rate': '0.0003305', 'epoch': '0.441'}
46%|████▌ | 1820/4000 [2:07:54<2:34:54, 4.26s/it]
46%|████▌ | 1821/4000 [2:07:58<2:33:55, 4.24s/it]
46%|████▌ | 1822/4000 [2:08:02<2:33:05, 4.22s/it]
46%|████▌ | 1823/4000 [2:08:06<2:32:44, 4.21s/it]
46%|████▌ | 1824/4000 [2:08:10<2:32:19, 4.20s/it]
46%|████▌ | 1825/4000 [2:08:15<2:31:57, 4.19s/it]
46%|████▌ | 1826/4000 [2:08:19<2:35:52, 4.30s/it]
46%|████▌ | 1827/4000 [2:08:23<2:34:10, 4.26s/it]
46%|████▌ | 1828/4000 [2:08:27<2:33:01, 4.23s/it]
46%|████▌ | 1829/4000 [2:08:32<2:32:10, 4.21s/it]
46%|████▌ | 1830/4000 [2:08:36<2:31:34, 4.19s/it]
{'loss': '0.4807', 'grad_norm': '0.1571', 'learning_rate': '0.0003289', 'epoch': '0.4434'}
46%|████▌ | 1830/4000 [2:08:36<2:31:34, 4.19s/it]
46%|████▌ | 1831/4000 [2:08:40<2:31:13, 4.18s/it]
46%|████▌ | 1832/4000 [2:08:44<2:30:57, 4.18s/it]
46%|████▌ | 1833/4000 [2:08:49<2:34:34, 4.28s/it]
46%|████▌ | 1834/4000 [2:08:53<2:33:13, 4.24s/it]
46%|████▌ | 1835/4000 [2:08:57<2:32:19, 4.22s/it]
46%|████▌ | 1836/4000 [2:09:01<2:31:43, 4.21s/it]
46%|████▌ | 1837/4000 [2:09:05<2:31:09, 4.19s/it]
46%|████▌ | 1838/4000 [2:09:09<2:30:36, 4.18s/it]
46%|████▌ | 1839/4000 [2:09:14<2:34:00, 4.28s/it]
46%|████▌ | 1840/4000 [2:09:18<2:32:30, 4.24s/it]
{'loss': '0.4902', 'grad_norm': '0.1436', 'learning_rate': '0.0003274', 'epoch': '0.4458'}
46%|████▌ | 1840/4000 [2:09:18<2:32:30, 4.24s/it]
46%|████▌ | 1841/4000 [2:09:22<2:31:22, 4.21s/it]
46%|████▌ | 1842/4000 [2:09:26<2:30:43, 4.19s/it]
46%|████▌ | 1843/4000 [2:09:31<2:30:12, 4.18s/it]
46%|████▌ | 1844/4000 [2:09:35<2:29:56, 4.17s/it]
46%|████▌ | 1845/4000 [2:09:39<2:29:36, 4.17s/it]
46%|████▌ | 1846/4000 [2:09:43<2:33:11, 4.27s/it]
46%|████▌ | 1847/4000 [2:09:47<2:31:45, 4.23s/it]
46%|████▌ | 1848/4000 [2:09:52<2:30:51, 4.21s/it]
46%|████▌ | 1849/4000 [2:09:56<2:30:29, 4.20s/it]
46%|████▋ | 1850/4000 [2:10:00<2:30:05, 4.19s/it]
{'loss': '0.4978', 'grad_norm': '0.1289', 'learning_rate': '0.0003259', 'epoch': '0.4483'}
46%|████▋ | 1850/4000 [2:10:00<2:30:05, 4.19s/it]
46%|████▋ | 1851/4000 [2:10:04<2:29:37, 4.18s/it]
46%|████▋ | 1852/4000 [2:10:08<2:29:21, 4.17s/it]
46%|████▋ | 1853/4000 [2:10:13<2:33:21, 4.29s/it]
46%|████▋ | 1854/4000 [2:10:17<2:32:15, 4.26s/it]
46%|████▋ | 1855/4000 [2:10:21<2:31:19, 4.23s/it]
46%|████▋ | 1856/4000 [2:10:25<2:30:43, 4.22s/it]
46%|████▋ | 1857/4000 [2:10:30<2:30:04, 4.20s/it]
46%|████▋ | 1858/4000 [2:10:34<2:29:42, 4.19s/it]
46%|████▋ | 1859/4000 [2:10:38<2:29:22, 4.19s/it]
46%|████▋ | 1860/4000 [2:10:42<2:33:13, 4.30s/it]
{'loss': '0.4719', 'grad_norm': '0.1304', 'learning_rate': '0.0003244', 'epoch': '0.4507'}
46%|████▋ | 1860/4000 [2:10:42<2:33:13, 4.30s/it]
47%|████▋ | 1861/4000 [2:10:47<2:31:36, 4.25s/it]
47%|████▋ | 1862/4000 [2:10:51<2:30:27, 4.22s/it]
47%|████▋ | 1863/4000 [2:10:55<2:29:31, 4.20s/it]
47%|████▋ | 1864/4000 [2:10:59<2:29:39, 4.20s/it]
47%|████▋ | 1865/4000 [2:11:03<2:29:29, 4.20s/it]
47%|████▋ | 1866/4000 [2:11:08<2:32:51, 4.30s/it]
47%|████▋ | 1867/4000 [2:11:12<2:31:21, 4.26s/it]
47%|████▋ | 1868/4000 [2:11:16<2:30:14, 4.23s/it]
47%|████▋ | 1869/4000 [2:11:20<2:29:24, 4.21s/it]
47%|████▋ | 1870/4000 [2:11:24<2:28:58, 4.20s/it]
{'loss': '0.4726', 'grad_norm': '0.1169', 'learning_rate': '0.0003229', 'epoch': '0.4531'}
47%|████▋ | 1870/4000 [2:11:24<2:28:58, 4.20s/it]
47%|████▋ | 1871/4000 [2:11:29<2:28:34, 4.19s/it]
47%|████▋ | 1872/4000 [2:11:33<2:28:10, 4.18s/it]
47%|████▋ | 1873/4000 [2:11:37<2:32:10, 4.29s/it]
47%|████▋ | 1874/4000 [2:11:42<2:30:55, 4.26s/it]
47%|████▋ | 1875/4000 [2:11:46<2:30:06, 4.24s/it]
47%|████▋ | 1876/4000 [2:11:50<2:29:40, 4.23s/it]
47%|████▋ | 1877/4000 [2:11:54<2:28:59, 4.21s/it]
47%|████▋ | 1878/4000 [2:11:58<2:28:31, 4.20s/it]
47%|████▋ | 1879/4000 [2:12:02<2:28:00, 4.19s/it]
47%|████▋ | 1880/4000 [2:12:07<2:31:18, 4.28s/it]
{'loss': '0.487', 'grad_norm': '0.1263', 'learning_rate': '0.0003214', 'epoch': '0.4555'}
47%|████▋ | 1880/4000 [2:12:07<2:31:18, 4.28s/it]
47%|████▋ | 1881/4000 [2:12:11<2:29:54, 4.24s/it]
47%|████▋ | 1882/4000 [2:12:15<2:28:57, 4.22s/it]
47%|████▋ | 1883/4000 [2:12:19<2:28:24, 4.21s/it]
47%|████▋ | 1884/4000 [2:12:24<2:27:59, 4.20s/it]
47%|████▋ | 1885/4000 [2:12:28<2:27:46, 4.19s/it]
47%|████▋ | 1886/4000 [2:12:32<2:29:19, 4.24s/it]
47%|████▋ | 1887/4000 [2:12:37<2:30:36, 4.28s/it]
47%|████▋ | 1888/4000 [2:12:41<2:29:18, 4.24s/it]
47%|████▋ | 1889/4000 [2:12:45<2:28:20, 4.22s/it]
47%|████▋ | 1890/4000 [2:12:49<2:27:50, 4.20s/it]
{'loss': '0.455', 'grad_norm': '0.134', 'learning_rate': '0.0003198', 'epoch': '0.4579'}
47%|████▋ | 1890/4000 [2:12:49<2:27:50, 4.20s/it]
47%|████▋ | 1891/4000 [2:12:53<2:27:27, 4.19s/it]
47%|████▋ | 1892/4000 [2:12:57<2:27:14, 4.19s/it]
47%|████▋ | 1893/4000 [2:13:02<2:30:55, 4.30s/it]
47%|████▋ | 1894/4000 [2:13:06<2:29:18, 4.25s/it]
47%|████▋ | 1895/4000 [2:13:10<2:28:03, 4.22s/it]
47%|████▋ | 1896/4000 [2:13:14<2:27:11, 4.20s/it]
47%|████▋ | 1897/4000 [2:13:19<2:26:42, 4.19s/it]
47%|████▋ | 1898/4000 [2:13:23<2:26:12, 4.17s/it]
47%|████▋ | 1899/4000 [2:13:27<2:25:52, 4.17s/it]
48%|████▊ | 1900/4000 [2:13:31<2:29:31, 4.27s/it]
{'loss': '0.4551', 'grad_norm': '0.119', 'learning_rate': '0.0003183', 'epoch': '0.4604'}
48%|████▊ | 1900/4000 [2:13:31<2:29:31, 4.27s/it]
48%|████▊ | 1901/4000 [2:13:35<2:28:23, 4.24s/it]
48%|████▊ | 1902/4000 [2:13:40<2:27:21, 4.21s/it]
48%|████▊ | 1903/4000 [2:13:44<2:26:29, 4.19s/it]
48%|████▊ | 1904/4000 [2:13:48<2:25:58, 4.18s/it]
48%|████▊ | 1905/4000 [2:13:52<2:25:38, 4.17s/it]
48%|████▊ | 1906/4000 [2:13:56<2:25:11, 4.16s/it]
48%|████▊ | 1907/4000 [2:14:01<2:29:11, 4.28s/it]
48%|████▊ | 1908/4000 [2:14:05<2:27:59, 4.24s/it]
48%|████▊ | 1909/4000 [2:14:09<2:26:58, 4.22s/it]
48%|████▊ | 1910/4000 [2:14:13<2:26:10, 4.20s/it]
{'loss': '0.4717', 'grad_norm': '0.1591', 'learning_rate': '0.0003168', 'epoch': '0.4628'}
48%|████▊ | 1910/4000 [2:14:13<2:26:10, 4.20s/it]
48%|████▊ | 1911/4000 [2:14:17<2:25:39, 4.18s/it]
48%|████▊ | 1912/4000 [2:14:22<2:25:19, 4.18s/it]
48%|████▊ | 1913/4000 [2:14:26<2:27:07, 4.23s/it]
48%|████▊ | 1914/4000 [2:14:30<2:27:59, 4.26s/it]
48%|████▊ | 1915/4000 [2:14:34<2:26:57, 4.23s/it]
48%|████▊ | 1916/4000 [2:14:39<2:26:15, 4.21s/it]
48%|████▊ | 1917/4000 [2:14:43<2:25:38, 4.20s/it]
48%|████▊ | 1918/4000 [2:14:47<2:25:20, 4.19s/it]
48%|████▊ | 1919/4000 [2:14:51<2:24:57, 4.18s/it]
48%|████▊ | 1920/4000 [2:14:56<2:28:28, 4.28s/it]
{'loss': '0.4751', 'grad_norm': '0.163', 'learning_rate': '0.0003153', 'epoch': '0.4652'}
48%|████▊ | 1920/4000 [2:14:56<2:28:28, 4.28s/it]
48%|████▊ | 1921/4000 [2:15:00<2:27:08, 4.25s/it]
48%|████▊ | 1922/4000 [2:15:04<2:26:15, 4.22s/it]
48%|████▊ | 1923/4000 [2:15:08<2:25:35, 4.21s/it]
48%|████▊ | 1924/4000 [2:15:12<2:25:04, 4.19s/it]
48%|████▊ | 1925/4000 [2:15:16<2:24:48, 4.19s/it]
48%|████▊ | 1926/4000 [2:15:21<2:24:35, 4.18s/it]
48%|████▊ | 1927/4000 [2:15:25<2:28:08, 4.29s/it]
48%|████▊ | 1928/4000 [2:15:29<2:27:01, 4.26s/it]
48%|████▊ | 1929/4000 [2:15:33<2:26:10, 4.24s/it]
48%|████▊ | 1930/4000 [2:15:38<2:25:20, 4.21s/it]
{'loss': '0.4428', 'grad_norm': '0.1237', 'learning_rate': '0.0003138', 'epoch': '0.4676'}
48%|████▊ | 1930/4000 [2:15:38<2:25:20, 4.21s/it]
48%|████▊ | 1931/4000 [2:15:42<2:24:51, 4.20s/it]
48%|████▊ | 1932/4000 [2:15:46<2:24:35, 4.20s/it]
48%|████▊ | 1933/4000 [2:15:50<2:24:15, 4.19s/it]
48%|████▊ | 1934/4000 [2:15:55<2:27:44, 4.29s/it]
48%|████▊ | 1935/4000 [2:15:59<2:26:21, 4.25s/it]
48%|████▊ | 1936/4000 [2:16:03<2:25:27, 4.23s/it]
48%|████▊ | 1937/4000 [2:16:07<2:24:44, 4.21s/it]
48%|████▊ | 1938/4000 [2:16:11<2:24:14, 4.20s/it]
48%|████▊ | 1939/4000 [2:16:16<2:23:58, 4.19s/it]
48%|████▊ | 1940/4000 [2:16:20<2:26:01, 4.25s/it]
{'loss': '0.4963', 'grad_norm': '0.1213', 'learning_rate': '0.0003123', 'epoch': '0.4701'}
48%|████▊ | 1940/4000 [2:16:20<2:26:01, 4.25s/it]
49%|████▊ | 1941/4000 [2:16:24<2:26:57, 4.28s/it]
49%|████▊ | 1942/4000 [2:16:28<2:25:40, 4.25s/it]
49%|████▊ | 1943/4000 [2:16:33<2:24:55, 4.23s/it]
49%|████▊ | 1944/4000 [2:16:37<2:24:06, 4.21s/it]
49%|████▊ | 1945/4000 [2:16:41<2:23:32, 4.19s/it]
49%|████▊ | 1946/4000 [2:16:45<2:23:04, 4.18s/it]
49%|████▊ | 1947/4000 [2:16:50<2:26:05, 4.27s/it]
49%|████▊ | 1948/4000 [2:16:54<2:24:46, 4.23s/it]
49%|████▊ | 1949/4000 [2:16:58<2:23:48, 4.21s/it]
49%|████▉ | 1950/4000 [2:17:02<2:23:19, 4.19s/it]
{'loss': '0.4976', 'grad_norm': '0.2503', 'learning_rate': '0.0003108', 'epoch': '0.4725'}
49%|████▉ | 1950/4000 [2:17:02<2:23:19, 4.19s/it]
49%|████▉ | 1951/4000 [2:17:06<2:22:55, 4.19s/it]
49%|████▉ | 1952/4000 [2:17:10<2:22:36, 4.18s/it]
49%|████▉ | 1953/4000 [2:17:15<2:22:25, 4.17s/it]
49%|████▉ | 1954/4000 [2:17:19<2:25:57, 4.28s/it]
49%|████▉ | 1955/4000 [2:17:23<2:24:32, 4.24s/it]
49%|████▉ | 1956/4000 [2:17:27<2:23:40, 4.22s/it]
49%|████▉ | 1957/4000 [2:17:32<2:23:03, 4.20s/it]
49%|████▉ | 1958/4000 [2:17:36<2:22:32, 4.19s/it]
49%|████▉ | 1959/4000 [2:17:40<2:22:14, 4.18s/it]
49%|████▉ | 1960/4000 [2:17:44<2:21:52, 4.17s/it]
{'loss': '0.4479', 'grad_norm': '0.1186', 'learning_rate': '0.0003092', 'epoch': '0.4749'}
49%|████▉ | 1960/4000 [2:17:44<2:21:52, 4.17s/it]
49%|████▉ | 1961/4000 [2:17:49<2:25:28, 4.28s/it]
49%|████▉ | 1962/4000 [2:17:53<2:24:02, 4.24s/it]
49%|████▉ | 1963/4000 [2:17:57<2:23:08, 4.22s/it]
49%|████▉ | 1964/4000 [2:18:01<2:22:20, 4.19s/it]
49%|████▉ | 1965/4000 [2:18:05<2:21:46, 4.18s/it]
49%|████▉ | 1966/4000 [2:18:09<2:21:34, 4.18s/it]
49%|████▉ | 1967/4000 [2:18:14<2:24:47, 4.27s/it]
49%|████▉ | 1968/4000 [2:18:18<2:23:21, 4.23s/it]
49%|████▉ | 1969/4000 [2:18:22<2:22:02, 4.20s/it]
49%|████▉ | 1970/4000 [2:18:26<2:21:23, 4.18s/it]
{'loss': '0.4695', 'grad_norm': '0.1176', 'learning_rate': '0.0003077', 'epoch': '0.4773'}
49%|████▉ | 1970/4000 [2:18:26<2:21:23, 4.18s/it]
49%|████▉ | 1971/4000 [2:18:30<2:20:32, 4.16s/it]
49%|████▉ | 1972/4000 [2:18:34<2:19:54, 4.14s/it]
49%|████▉ | 1973/4000 [2:18:38<2:19:27, 4.13s/it]
49%|████▉ | 1974/4000 [2:18:43<2:22:44, 4.23s/it]
49%|████▉ | 1975/4000 [2:18:47<2:21:23, 4.19s/it]
49%|████▉ | 1976/4000 [2:18:51<2:20:19, 4.16s/it]
49%|████▉ | 1977/4000 [2:18:55<2:19:39, 4.14s/it]
49%|████▉ | 1978/4000 [2:18:59<2:19:16, 4.13s/it]
49%|████▉ | 1979/4000 [2:19:03<2:18:56, 4.13s/it]
50%|████▉ | 1980/4000 [2:19:08<2:18:42, 4.12s/it]
{'loss': '0.4677', 'grad_norm': '0.127', 'learning_rate': '0.0003062', 'epoch': '0.4798'}
50%|████▉ | 1980/4000 [2:19:08<2:18:42, 4.12s/it]
50%|████▉ | 1981/4000 [2:19:12<2:21:39, 4.21s/it]
50%|████▉ | 1982/4000 [2:19:16<2:20:26, 4.18s/it]
50%|████▉ | 1983/4000 [2:19:20<2:19:40, 4.16s/it]
50%|████▉ | 1984/4000 [2:19:24<2:19:04, 4.14s/it]
50%|████▉ | 1985/4000 [2:19:28<2:18:40, 4.13s/it]
50%|████▉ | 1986/4000 [2:19:33<2:18:19, 4.12s/it]
50%|████▉ | 1987/4000 [2:19:37<2:18:04, 4.12s/it]
50%|████▉ | 1988/4000 [2:19:41<2:21:35, 4.22s/it]
50%|████▉ | 1989/4000 [2:19:45<2:20:17, 4.19s/it]
50%|████▉ | 1990/4000 [2:19:49<2:19:18, 4.16s/it]
{'loss': '0.4632', 'grad_norm': '0.1225', 'learning_rate': '0.0003047', 'epoch': '0.4822'}
50%|████▉ | 1990/4000 [2:19:49<2:19:18, 4.16s/it]
50%|████▉ | 1991/4000 [2:19:53<2:18:39, 4.14s/it]
50%|████▉ | 1992/4000 [2:19:57<2:18:14, 4.13s/it]
50%|████▉ | 1993/4000 [2:20:02<2:17:57, 4.12s/it]
50%|████▉ | 1994/4000 [2:20:06<2:21:14, 4.22s/it]
50%|████▉ | 1995/4000 [2:20:10<2:19:52, 4.19s/it]
50%|████▉ | 1996/4000 [2:20:14<2:18:54, 4.16s/it]
50%|████▉ | 1997/4000 [2:20:18<2:19:11, 4.17s/it]
50%|████▉ | 1998/4000 [2:20:23<2:19:32, 4.18s/it]
50%|████▉ | 1999/4000 [2:20:27<2:18:38, 4.16s/it]
50%|█████ | 2000/4000 [2:20:31<2:17:53, 4.14s/it]
{'loss': '0.4511', 'grad_norm': '0.1182', 'learning_rate': '0.0003032', 'epoch': '0.4846'}
50%|█████ | 2000/4000 [2:20:31<2:17:53, 4.14s/it]
0%| | 0/2 [00:00<?, ?it/s][A
100%|██████████| 2/2 [00:00<00:00, 4.28it/s][A
[A{'eval_loss': '2.541', 'eval_runtime': '0.9973', 'eval_samples_per_second': '22.06', 'eval_steps_per_second': '2.005', 'epoch': '0.4846'}
50%|█████ | 2000/4000 [2:20:32<2:17:53, 4.14s/it]
100%|██████████| 2/2 [00:00<00:00, 4.28it/s][A
[A[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
- If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
- If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
- If you are not the owner of the model architecture class, please contact the model code owner to update it.
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s][A
Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 3.64it/s][A
Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 3.63it/s]
50%|█████ | 2001/4000 [2:20:40<3:10:18, 5.71s/it]
50%|█████ | 2002/4000 [2:20:44<2:54:03, 5.23s/it]
50%|█████ | 2003/4000 [2:20:48<2:42:39, 4.89s/it]
50%|█████ | 2004/4000 [2:20:53<2:34:43, 4.65s/it]
50%|█████ | 2005/4000 [2:20:57<2:29:05, 4.48s/it]
50%|█████ | 2006/4000 [2:21:01<2:25:06, 4.37s/it]
50%|█████ | 2007/4000 [2:21:05<2:22:25, 4.29s/it]
50%|█████ | 2008/4000 [2:21:09<2:23:45, 4.33s/it]
50%|█████ | 2009/4000 [2:21:13<2:21:25, 4.26s/it]
50%|█████ | 2010/4000 [2:21:17<2:19:44, 4.21s/it]
{'loss': '0.4894', 'grad_norm': '0.1521', 'learning_rate': '0.0003017', 'epoch': '0.487'}
50%|█████ | 2010/4000 [2:21:17<2:19:44, 4.21s/it]
50%|█████ | 2011/4000 [2:21:22<2:18:32, 4.18s/it]
50%|█████ | 2012/4000 [2:21:26<2:17:42, 4.16s/it]
50%|█████ | 2013/4000 [2:21:30<2:17:01, 4.14s/it]
50%|█████ | 2014/4000 [2:21:34<2:16:29, 4.12s/it]
50%|█████ | 2015/4000 [2:21:38<2:19:32, 4.22s/it]
50%|█████ | 2016/4000 [2:21:42<2:18:23, 4.19s/it]
50%|█████ | 2017/4000 [2:21:46<2:17:23, 4.16s/it]
50%|█████ | 2018/4000 [2:21:51<2:16:41, 4.14s/it]
50%|█████ | 2019/4000 [2:21:55<2:16:13, 4.13s/it]
50%|█████ | 2020/4000 [2:21:59<2:15:48, 4.12s/it]
{'loss': '0.4608', 'grad_norm': '0.1268', 'learning_rate': '0.0003002', 'epoch': '0.4894'}
50%|█████ | 2020/4000 [2:21:59<2:15:48, 4.12s/it]
51%|█████ | 2021/4000 [2:22:03<2:18:48, 4.21s/it]
51%|█████ | 2022/4000 [2:22:07<2:17:42, 4.18s/it]
51%|█████ | 2023/4000 [2:22:11<2:16:53, 4.15s/it]
51%|█████ | 2024/4000 [2:22:15<2:16:13, 4.14s/it]
51%|█████ | 2025/4000 [2:22:20<2:15:48, 4.13s/it]
51%|█████ | 2026/4000 [2:22:24<2:15:33, 4.12s/it]
51%|█████ | 2027/4000 [2:22:28<2:15:14, 4.11s/it]
51%|█████ | 2028/4000 [2:22:32<2:18:18, 4.21s/it]
51%|█████ | 2029/4000 [2:22:36<2:17:07, 4.17s/it]
51%|█████ | 2030/4000 [2:22:40<2:16:16, 4.15s/it]
{'loss': '0.4211', 'grad_norm': '0.119', 'learning_rate': '0.0002986', 'epoch': '0.4919'}
51%|█████ | 2030/4000 [2:22:40<2:16:16, 4.15s/it]
51%|█████ | 2031/4000 [2:22:44<2:15:39, 4.13s/it]
51%|█████ | 2032/4000 [2:22:49<2:15:20, 4.13s/it]
51%|█████ | 2033/4000 [2:22:53<2:15:00, 4.12s/it]
51%|█████ | 2034/4000 [2:22:57<2:14:50, 4.12s/it]
51%|█████ | 2035/4000 [2:23:01<2:18:11, 4.22s/it]
51%|█████ | 2036/4000 [2:23:05<2:16:55, 4.18s/it]
51%|█████ | 2037/4000 [2:23:09<2:15:58, 4.16s/it]
51%|█████ | 2038/4000 [2:23:14<2:15:19, 4.14s/it]
51%|█████ | 2039/4000 [2:23:18<2:14:53, 4.13s/it]
51%|█████ | 2040/4000 [2:23:22<2:14:31, 4.12s/it]
{'loss': '0.4982', 'grad_norm': '0.1192', 'learning_rate': '0.0002971', 'epoch': '0.4943'}
51%|█████ | 2040/4000 [2:23:22<2:14:31, 4.12s/it]
51%|█████ | 2041/4000 [2:23:26<2:15:41, 4.16s/it]
51%|█████ | 2042/4000 [2:23:30<2:16:26, 4.18s/it]
51%|█████ | 2043/4000 [2:23:34<2:15:33, 4.16s/it]
51%|█████ | 2044/4000 [2:23:38<2:14:56, 4.14s/it]
51%|█████ | 2045/4000 [2:23:43<2:14:34, 4.13s/it]
51%|█████ | 2046/4000 [2:23:47<2:14:13, 4.12s/it]
51%|█████ | 2047/4000 [2:23:51<2:13:52, 4.11s/it]
51%|█████ | 2048/4000 [2:23:55<2:16:58, 4.21s/it]
51%|█████ | 2049/4000 [2:23:59<2:15:51, 4.18s/it]
51%|█████▏ | 2050/4000 [2:24:03<2:14:56, 4.15s/it]
{'loss': '0.4563', 'grad_norm': '0.1189', 'learning_rate': '0.0002956', 'epoch': '0.4967'}
51%|█████▏ | 2050/4000 [2:24:03<2:14:56, 4.15s/it]
51%|█████▏ | 2051/4000 [2:24:07<2:14:24, 4.14s/it]
51%|█████▏ | 2052/4000 [2:24:12<2:14:01, 4.13s/it]
51%|█████▏ | 2053/4000 [2:24:16<2:13:39, 4.12s/it]
51%|█████▏ | 2054/4000 [2:24:20<2:13:24, 4.11s/it]
51%|█████▏ | 2055/4000 [2:24:24<2:16:40, 4.22s/it]
51%|█████▏ | 2056/4000 [2:24:28<2:15:30, 4.18s/it]
51%|█████▏ | 2057/4000 [2:24:32<2:14:39, 4.16s/it]
51%|█████▏ | 2058/4000 [2:24:37<2:13:54, 4.14s/it]
51%|█████▏ | 2059/4000 [2:24:41<2:13:26, 4.13s/it]
52%|█████▏ | 2060/4000 [2:24:45<2:13:06, 4.12s/it]
{'loss': '0.4767', 'grad_norm': '0.1314', 'learning_rate': '0.0002941', 'epoch': '0.4991'}
52%|█████▏ | 2060/4000 [2:24:45<2:13:06, 4.12s/it]
52%|█████▏ | 2061/4000 [2:24:49<2:12:54, 4.11s/it]
52%|█████▏ | 2062/4000 [2:24:53<2:16:11, 4.22s/it]
52%|█████▏ | 2063/4000 [2:24:57<2:15:02, 4.18s/it]
52%|█████▏ | 2064/4000 [2:25:01<2:14:07, 4.16s/it]
52%|█████▏ | 2065/4000 [2:25:06<2:13:31, 4.14s/it]
52%|█████▏ | 2066/4000 [2:25:10<2:13:12, 4.13s/it]
52%|█████▏ | 2067/4000 [2:25:14<2:12:54, 4.13s/it]
52%|█████▏ | 2068/4000 [2:25:18<2:14:18, 4.17s/it]
52%|█████▏ | 2069/4000 [2:25:22<2:15:11, 4.20s/it]
52%|█████▏ | 2070/4000 [2:25:26<2:14:15, 4.17s/it]
{'loss': '0.465', 'grad_norm': '0.1344', 'learning_rate': '0.0002926', 'epoch': '0.5016'}
52%|█████▏ | 2070/4000 [2:25:26<2:14:15, 4.17s/it]
52%|█████▏ | 2071/4000 [2:25:31<2:13:33, 4.15s/it]
52%|█████▏ | 2072/4000 [2:25:35<2:14:04, 4.17s/it]
52%|█████▏ | 2073/4000 [2:25:39<2:14:45, 4.20s/it]
52%|█████▏ | 2074/4000 [2:25:43<2:13:56, 4.17s/it]
52%|█████▏ | 2075/4000 [2:25:48<2:16:25, 4.25s/it]
52%|█████▏ | 2076/4000 [2:25:52<2:15:09, 4.21s/it]
52%|█████▏ | 2077/4000 [2:25:56<2:14:09, 4.19s/it]
52%|█████▏ | 2078/4000 [2:26:00<2:13:26, 4.17s/it]
52%|█████▏ | 2079/4000 [2:26:04<2:12:50, 4.15s/it]
52%|█████▏ | 2080/4000 [2:26:08<2:12:21, 4.14s/it]
{'loss': '0.4363', 'grad_norm': '0.1398', 'learning_rate': '0.0002911', 'epoch': '0.504'}
52%|█████▏ | 2080/4000 [2:26:08<2:12:21, 4.14s/it]
52%|█████▏ | 2081/4000 [2:26:12<2:12:03, 4.13s/it]
52%|█████▏ | 2082/4000 [2:26:17<2:15:02, 4.22s/it]
52%|█████▏ | 2083/4000 [2:26:21<2:13:42, 4.19s/it]
52%|█████▏ | 2084/4000 [2:26:25<2:13:13, 4.17s/it]
52%|█████▏ | 2085/4000 [2:26:29<2:12:37, 4.16s/it]
52%|█████▏ | 2086/4000 [2:26:33<2:12:04, 4.14s/it]
52%|█████▏ | 2087/4000 [2:26:37<2:11:43, 4.13s/it]
52%|█████▏ | 2088/4000 [2:26:41<2:11:30, 4.13s/it]
52%|█████▏ | 2089/4000 [2:26:46<2:14:28, 4.22s/it]
52%|█████▏ | 2090/4000 [2:26:50<2:13:24, 4.19s/it]
{'loss': '0.4345', 'grad_norm': '0.131', 'learning_rate': '0.0002895', 'epoch': '0.5064'}
52%|█████▏ | 2090/4000 [2:26:50<2:13:24, 4.19s/it]
52%|█████▏ | 2091/4000 [2:26:54<2:12:36, 4.17s/it]
52%|█████▏ | 2092/4000 [2:26:58<2:12:11, 4.16s/it]
52%|█████▏ | 2093/4000 [2:27:02<2:11:47, 4.15s/it]
52%|█████▏ | 2094/4000 [2:27:06<2:11:30, 4.14s/it]
52%|█████▏ | 2095/4000 [2:27:11<2:12:59, 4.19s/it]
52%|█████▏ | 2096/4000 [2:27:15<2:13:58, 4.22s/it]
52%|█████▏ | 2097/4000 [2:27:19<2:12:50, 4.19s/it]
52%|█████▏ | 2098/4000 [2:27:23<2:12:04, 4.17s/it]
52%|█████▏ | 2099/4000 [2:27:27<2:11:34, 4.15s/it]
52%|█████▎ | 2100/4000 [2:27:32<2:11:26, 4.15s/it]
{'loss': '0.4314', 'grad_norm': '0.1013', 'learning_rate': '0.000288', 'epoch': '0.5088'}
52%|█████▎ | 2100/4000 [2:27:32<2:11:26, 4.15s/it]
53%|█████▎ | 2101/4000 [2:27:36<2:11:06, 4.14s/it]
53%|█████▎ | 2102/4000 [2:27:40<2:14:07, 4.24s/it]
53%|█████▎ | 2103/4000 [2:27:44<2:12:49, 4.20s/it]
53%|█████▎ | 2104/4000 [2:27:48<2:11:48, 4.17s/it]
53%|█████▎ | 2105/4000 [2:27:52<2:11:01, 4.15s/it]
53%|█████▎ | 2106/4000 [2:27:57<2:10:28, 4.13s/it]
53%|█████▎ | 2107/4000 [2:28:01<2:10:05, 4.12s/it]
53%|█████▎ | 2108/4000 [2:28:05<2:09:51, 4.12s/it]
53%|█████▎ | 2109/4000 [2:28:09<2:12:32, 4.21s/it]
53%|█████▎ | 2110/4000 [2:28:13<2:11:33, 4.18s/it]
{'loss': '0.4538', 'grad_norm': '0.1193', 'learning_rate': '0.0002865', 'epoch': '0.5113'}
53%|█████▎ | 2110/4000 [2:28:13<2:11:33, 4.18s/it]
53%|█████▎ | 2111/4000 [2:28:17<2:10:45, 4.15s/it]
53%|█████▎ | 2112/4000 [2:28:21<2:10:14, 4.14s/it]
53%|█████▎ | 2113/4000 [2:28:26<2:09:50, 4.13s/it]
53%|█████▎ | 2114/4000 [2:28:30<2:09:32, 4.12s/it]
53%|█████▎ | 2115/4000 [2:28:34<2:09:21, 4.12s/it]
53%|█████▎ | 2116/4000 [2:28:38<2:12:42, 4.23s/it]
53%|█████▎ | 2117/4000 [2:28:42<2:11:28, 4.19s/it]
53%|█████▎ | 2118/4000 [2:28:47<2:10:39, 4.17s/it]
53%|█████▎ | 2119/4000 [2:28:51<2:10:21, 4.16s/it]
53%|█████▎ | 2120/4000 [2:28:55<2:09:49, 4.14s/it]
{'loss': '0.447', 'grad_norm': '0.1392', 'learning_rate': '0.000285', 'epoch': '0.5137'}
53%|█████▎ | 2120/4000 [2:28:55<2:09:49, 4.14s/it]
53%|█████▎ | 2121/4000 [2:28:59<2:09:23, 4.13s/it]
53%|█████▎ | 2122/4000 [2:29:03<2:12:34, 4.24s/it]
53%|█████▎ | 2123/4000 [2:29:07<2:11:17, 4.20s/it]
53%|█████▎ | 2124/4000 [2:29:12<2:10:22, 4.17s/it]
53%|█████▎ | 2125/4000 [2:29:16<2:09:45, 4.15s/it]
53%|█████▎ | 2126/4000 [2:29:20<2:09:17, 4.14s/it]
53%|█████▎ | 2127/4000 [2:29:24<2:08:59, 4.13s/it]
53%|█████▎ | 2128/4000 [2:29:28<2:08:42, 4.13s/it]
53%|█████▎ | 2129/4000 [2:29:32<2:11:42, 4.22s/it]
53%|█████▎ | 2130/4000 [2:29:37<2:10:41, 4.19s/it]
{'loss': '0.4419', 'grad_norm': '0.2505', 'learning_rate': '0.0002835', 'epoch': '0.5161'}
53%|█████▎ | 2130/4000 [2:29:37<2:10:41, 4.19s/it]
53%|█████▎ | 2131/4000 [2:29:41<2:09:53, 4.17s/it]
53%|█████▎ | 2132/4000 [2:29:45<2:09:17, 4.15s/it]
53%|█████▎ | 2133/4000 [2:29:49<2:08:50, 4.14s/it]
53%|█████▎ | 2134/4000 [2:29:53<2:08:31, 4.13s/it]
53%|█████▎ | 2135/4000 [2:29:57<2:08:17, 4.13s/it]
53%|█████▎ | 2136/4000 [2:30:02<2:11:14, 4.22s/it]
53%|█████▎ | 2137/4000 [2:30:06<2:10:10, 4.19s/it]
53%|█████▎ | 2138/4000 [2:30:10<2:09:21, 4.17s/it]
53%|█████▎ | 2139/4000 [2:30:14<2:08:51, 4.15s/it]
54%|█████▎ | 2140/4000 [2:30:18<2:08:25, 4.14s/it]
{'loss': '0.465', 'grad_norm': '0.1463', 'learning_rate': '0.000282', 'epoch': '0.5185'}
54%|█████▎ | 2140/4000 [2:30:18<2:08:25, 4.14s/it]
54%|█████▎ | 2141/4000 [2:30:22<2:08:01, 4.13s/it]
54%|█████▎ | 2142/4000 [2:30:26<2:07:45, 4.13s/it]
54%|█████▎ | 2143/4000 [2:30:31<2:10:59, 4.23s/it]
54%|█████▎ | 2144/4000 [2:30:35<2:09:50, 4.20s/it]
54%|█████▎ | 2145/4000 [2:30:39<2:08:58, 4.17s/it]
54%|█████▎ | 2146/4000 [2:30:43<2:08:26, 4.16s/it]
54%|█████▎ | 2147/4000 [2:30:47<2:08:03, 4.15s/it]
54%|█████▎ | 2148/4000 [2:30:51<2:07:40, 4.14s/it]
54%|█████▎ | 2149/4000 [2:30:56<2:09:08, 4.19s/it]
54%|█████▍ | 2150/4000 [2:31:00<2:09:56, 4.21s/it]
{'loss': '0.4351', 'grad_norm': '0.1265', 'learning_rate': '0.0002805', 'epoch': '0.5209'}
54%|█████▍ | 2150/4000 [2:31:00<2:09:56, 4.21s/it]
54%|█████▍ | 2151/4000 [2:31:04<2:08:55, 4.18s/it]
54%|█████▍ | 2152/4000 [2:31:08<2:08:15, 4.16s/it]
54%|█████▍ | 2153/4000 [2:31:12<2:07:38, 4.15s/it]
54%|█████▍ | 2154/4000 [2:31:16<2:07:15, 4.14s/it]
54%|█████▍ | 2155/4000 [2:31:20<2:06:57, 4.13s/it]
54%|█████▍ | 2156/4000 [2:31:25<2:09:52, 4.23s/it]
54%|█████▍ | 2157/4000 [2:31:29<2:08:50, 4.19s/it]
54%|█████▍ | 2158/4000 [2:31:33<2:08:04, 4.17s/it]
54%|█████▍ | 2159/4000 [2:31:37<2:07:31, 4.16s/it]
54%|█████▍ | 2160/4000 [2:31:41<2:07:17, 4.15s/it]
{'loss': '0.45', 'grad_norm': '0.1343', 'learning_rate': '0.0002789', 'epoch': '0.5234'}
54%|█████▍ | 2160/4000 [2:31:41<2:07:17, 4.15s/it]
54%|█████▍ | 2161/4000 [2:31:46<2:06:58, 4.14s/it]
54%|█████▍ | 2162/4000 [2:31:50<2:06:35, 4.13s/it]
54%|█████▍ | 2163/4000 [2:31:54<2:09:52, 4.24s/it]
54%|█████▍ | 2164/4000 [2:31:58<2:08:40, 4.21s/it]
54%|█████▍ | 2165/4000 [2:32:02<2:07:44, 4.18s/it]
54%|█████▍ | 2166/4000 [2:32:07<2:07:13, 4.16s/it]
54%|█████▍ | 2167/4000 [2:32:11<2:06:55, 4.15s/it]
54%|█████▍ | 2168/4000 [2:32:15<2:06:27, 4.14s/it]
54%|█████▍ | 2169/4000 [2:32:19<2:07:51, 4.19s/it]
54%|█████▍ | 2170/4000 [2:32:23<2:09:12, 4.24s/it]
{'loss': '0.4569', 'grad_norm': '0.1064', 'learning_rate': '0.0002774', 'epoch': '0.5258'}
54%|█████▍ | 2170/4000 [2:32:23<2:09:12, 4.24s/it]
54%|█████▍ | 2171/4000 [2:32:28<2:08:02, 4.20s/it]
54%|█████▍ | 2172/4000 [2:32:32<2:07:05, 4.17s/it]
54%|█████▍ | 2173/4000 [2:32:36<2:06:23, 4.15s/it]
54%|█████▍ | 2174/4000 [2:32:40<2:06:08, 4.14s/it]
54%|█████▍ | 2175/4000 [2:32:44<2:05:44, 4.13s/it]
54%|█████▍ | 2176/4000 [2:32:48<2:07:51, 4.21s/it]
54%|█████▍ | 2177/4000 [2:32:52<2:06:50, 4.17s/it]
54%|█████▍ | 2178/4000 [2:32:57<2:06:05, 4.15s/it]
54%|█████▍ | 2179/4000 [2:33:01<2:05:34, 4.14s/it]
55%|█████▍ | 2180/4000 [2:33:05<2:05:29, 4.14s/it]
{'loss': '0.4537', 'grad_norm': '0.1231', 'learning_rate': '0.0002759', 'epoch': '0.5282'}
55%|█████▍ | 2180/4000 [2:33:05<2:05:29, 4.14s/it]
55%|█████▍ | 2181/4000 [2:33:09<2:06:05, 4.16s/it]
55%|█████▍ | 2182/4000 [2:33:13<2:05:43, 4.15s/it]
55%|█████▍ | 2183/4000 [2:33:18<2:08:49, 4.25s/it]
55%|█████▍ | 2184/4000 [2:33:22<2:07:31, 4.21s/it]
55%|█████▍ | 2185/4000 [2:33:26<2:06:36, 4.19s/it]
55%|█████▍ | 2186/4000 [2:33:30<2:05:57, 4.17s/it]
55%|█████▍ | 2187/4000 [2:33:34<2:05:30, 4.15s/it]
55%|█████▍ | 2188/4000 [2:33:38<2:05:05, 4.14s/it]
55%|█████▍ | 2189/4000 [2:33:42<2:04:49, 4.14s/it]
55%|█████▍ | 2190/4000 [2:33:47<2:07:54, 4.24s/it]
{'loss': '0.4676', 'grad_norm': '0.122', 'learning_rate': '0.0002744', 'epoch': '0.5306'}
55%|█████▍ | 2190/4000 [2:33:47<2:07:54, 4.24s/it]
55%|█████▍ | 2191/4000 [2:33:51<2:06:44, 4.20s/it]
55%|█████▍ | 2192/4000 [2:33:55<2:05:58, 4.18s/it]
55%|█████▍ | 2193/4000 [2:33:59<2:05:21, 4.16s/it]
55%|█████▍ | 2194/4000 [2:34:03<2:04:53, 4.15s/it]
55%|█████▍ | 2195/4000 [2:34:07<2:04:34, 4.14s/it]
55%|█████▍ | 2196/4000 [2:34:12<2:05:59, 4.19s/it]
55%|█████▍ | 2197/4000 [2:34:16<2:06:42, 4.22s/it]
55%|█████▍ | 2198/4000 [2:34:20<2:07:15, 4.24s/it]
55%|█████▍ | 2199/4000 [2:34:25<2:07:32, 4.25s/it]
55%|█████▌ | 2200/4000 [2:34:29<2:07:43, 4.26s/it]
{'loss': '0.4642', 'grad_norm': '0.1354', 'learning_rate': '0.0002729', 'epoch': '0.5331'}
55%|█████▌ | 2200/4000 [2:34:29<2:07:43, 4.26s/it]
55%|█████▌ | 2201/4000 [2:34:33<2:07:44, 4.26s/it]
55%|█████▌ | 2202/4000 [2:34:37<2:07:49, 4.27s/it]
55%|█████▌ | 2203/4000 [2:34:42<2:11:41, 4.40s/it]
55%|█████▌ | 2204/4000 [2:34:46<2:10:26, 4.36s/it]
55%|█████▌ | 2205/4000 [2:34:51<2:08:24, 4.29s/it]
55%|█████▌ | 2206/4000 [2:34:55<2:06:47, 4.24s/it]
55%|█████▌ | 2207/4000 [2:34:59<2:05:43, 4.21s/it]
55%|█████▌ | 2208/4000 [2:35:03<2:04:59, 4.19s/it]
55%|█████▌ | 2209/4000 [2:35:07<2:04:30, 4.17s/it]
55%|█████▌ | 2210/4000 [2:35:12<2:07:09, 4.26s/it]
{'loss': '0.4432', 'grad_norm': '0.1118', 'learning_rate': '0.0002714', 'epoch': '0.5355'}
55%|█████▌ | 2210/4000 [2:35:12<2:07:09, 4.26s/it]
55%|█████▌ | 2211/4000 [2:35:16<2:05:51, 4.22s/it]
55%|█████▌ | 2212/4000 [2:35:20<2:04:55, 4.19s/it]
55%|█████▌ | 2213/4000 [2:35:24<2:04:16, 4.17s/it]
55%|█████▌ | 2214/4000 [2:35:28<2:03:46, 4.16s/it]
55%|█████▌ | 2215/4000 [2:35:32<2:03:26, 4.15s/it]
55%|█████▌ | 2216/4000 [2:35:36<2:03:08, 4.14s/it]
55%|█████▌ | 2217/4000 [2:35:41<2:06:14, 4.25s/it]
55%|█████▌ | 2218/4000 [2:35:45<2:05:13, 4.22s/it]
55%|█████▌ | 2219/4000 [2:35:49<2:04:28, 4.19s/it]
56%|█████▌ | 2220/4000 [2:35:53<2:03:51, 4.17s/it]
{'loss': '0.4497', 'grad_norm': '0.13', 'learning_rate': '0.0002698', 'epoch': '0.5379'}
56%|█████▌ | 2220/4000 [2:35:53<2:03:51, 4.17s/it]
56%|█████▌ | 2221/4000 [2:35:58<2:05:38, 4.24s/it]
56%|█████▌ | 2222/4000 [2:36:02<2:06:07, 4.26s/it]
56%|█████▌ | 2223/4000 [2:36:06<2:08:18, 4.33s/it]
56%|█████▌ | 2224/4000 [2:36:11<2:09:42, 4.38s/it]
56%|█████▌ | 2225/4000 [2:36:15<2:08:49, 4.35s/it]
56%|█████▌ | 2226/4000 [2:36:19<2:08:05, 4.33s/it]
56%|█████▌ | 2227/4000 [2:36:24<2:06:21, 4.28s/it]
56%|█████▌ | 2228/4000 [2:36:28<2:05:07, 4.24s/it]
56%|█████▌ | 2229/4000 [2:36:32<2:04:08, 4.21s/it]
56%|█████▌ | 2230/4000 [2:36:36<2:07:08, 4.31s/it]
{'loss': '0.4474', 'grad_norm': '0.121', 'learning_rate': '0.0002683', 'epoch': '0.5403'}
56%|█████▌ | 2230/4000 [2:36:36<2:07:08, 4.31s/it]
56%|█████▌ | 2231/4000 [2:36:41<2:05:27, 4.26s/it]
56%|█████▌ | 2232/4000 [2:36:45<2:04:24, 4.22s/it]
56%|█████▌ | 2233/4000 [2:36:49<2:03:36, 4.20s/it]
56%|█████▌ | 2234/4000 [2:36:53<2:03:08, 4.18s/it]
56%|█████▌ | 2235/4000 [2:36:57<2:02:42, 4.17s/it]
56%|█████▌ | 2236/4000 [2:37:01<2:02:18, 4.16s/it]
56%|█████▌ | 2237/4000 [2:37:06<2:05:42, 4.28s/it]
56%|█████▌ | 2238/4000 [2:37:10<2:04:27, 4.24s/it]
56%|█████▌ | 2239/4000 [2:37:14<2:04:04, 4.23s/it]
56%|█████▌ | 2240/4000 [2:37:18<2:04:03, 4.23s/it]
{'loss': '0.4517', 'grad_norm': '0.122', 'learning_rate': '0.0002668', 'epoch': '0.5428'}
56%|█████▌ | 2240/4000 [2:37:18<2:04:03, 4.23s/it]
56%|█████▌ | 2241/4000 [2:37:23<2:03:09, 4.20s/it]
56%|█████▌ | 2242/4000 [2:37:27<2:05:13, 4.27s/it]
56%|█████▌ | 2243/4000 [2:37:31<2:05:35, 4.29s/it]
56%|█████▌ | 2244/4000 [2:37:36<2:08:48, 4.40s/it]
56%|█████▌ | 2245/4000 [2:37:40<2:07:41, 4.37s/it]
56%|█████▌ | 2246/4000 [2:37:44<2:06:02, 4.31s/it]
56%|█████▌ | 2247/4000 [2:37:49<2:04:24, 4.26s/it]
56%|█████▌ | 2248/4000 [2:37:53<2:03:14, 4.22s/it]
56%|█████▌ | 2249/4000 [2:37:57<2:02:25, 4.20s/it]
56%|█████▋ | 2250/4000 [2:38:01<2:03:37, 4.24s/it]
{'loss': '0.4452', 'grad_norm': '0.1265', 'learning_rate': '0.0002653', 'epoch': '0.5452'}
56%|█████▋ | 2250/4000 [2:38:01<2:03:37, 4.24s/it]
56%|█████▋ | 2251/4000 [2:38:05<2:04:18, 4.26s/it]
56%|█████▋ | 2252/4000 [2:38:10<2:03:03, 4.22s/it]
56%|█████▋ | 2253/4000 [2:38:14<2:02:14, 4.20s/it]
56%|█████▋ | 2254/4000 [2:38:18<2:01:33, 4.18s/it]
56%|█████▋ | 2255/4000 [2:38:22<2:01:08, 4.17s/it]
56%|█████▋ | 2256/4000 [2:38:26<2:03:03, 4.23s/it]
56%|█████▋ | 2257/4000 [2:38:31<2:05:38, 4.33s/it]
56%|█████▋ | 2258/4000 [2:38:35<2:04:13, 4.28s/it]
56%|█████▋ | 2259/4000 [2:38:39<2:02:57, 4.24s/it]
56%|█████▋ | 2260/4000 [2:38:43<2:02:00, 4.21s/it]
{'loss': '0.4498', 'grad_norm': '0.09977', 'learning_rate': '0.0002638', 'epoch': '0.5476'}
56%|█████▋ | 2260/4000 [2:38:43<2:02:00, 4.21s/it]
57%|█████▋ | 2261/4000 [2:38:48<2:02:10, 4.22s/it]
57%|█████▋ | 2262/4000 [2:38:52<2:02:06, 4.22s/it]
57%|█████▋ | 2263/4000 [2:38:56<2:02:28, 4.23s/it]
57%|█████▋ | 2264/4000 [2:39:01<2:06:50, 4.38s/it]
57%|█████▋ | 2265/4000 [2:39:05<2:04:48, 4.32s/it]
57%|█████▋ | 2266/4000 [2:39:09<2:03:07, 4.26s/it]
57%|█████▋ | 2267/4000 [2:39:13<2:01:59, 4.22s/it]
57%|█████▋ | 2268/4000 [2:39:17<2:01:08, 4.20s/it]
57%|█████▋ | 2269/4000 [2:39:22<2:00:29, 4.18s/it]
57%|█████▋ | 2270/4000 [2:39:26<1:59:53, 4.16s/it]
{'loss': '0.4221', 'grad_norm': '0.1154', 'learning_rate': '0.0002623', 'epoch': '0.55'}
57%|█████▋ | 2270/4000 [2:39:26<1:59:53, 4.16s/it]
57%|█████▋ | 2271/4000 [2:39:30<2:02:57, 4.27s/it]
57%|█████▋ | 2272/4000 [2:39:34<2:01:41, 4.23s/it]
57%|█████▋ | 2273/4000 [2:39:39<2:01:51, 4.23s/it]
57%|█████▋ | 2274/4000 [2:39:43<2:00:57, 4.20s/it]
57%|█████▋ | 2275/4000 [2:39:47<2:00:53, 4.21s/it]
57%|█████▋ | 2276/4000 [2:39:51<2:00:09, 4.18s/it]
57%|█████▋ | 2277/4000 [2:39:55<2:02:27, 4.26s/it]
57%|█████▋ | 2278/4000 [2:40:00<2:02:20, 4.26s/it]
57%|█████▋ | 2279/4000 [2:40:04<2:01:22, 4.23s/it]
57%|█████▋ | 2280/4000 [2:40:08<2:00:30, 4.20s/it]
{'loss': '0.4633', 'grad_norm': '0.1137', 'learning_rate': '0.0002608', 'epoch': '0.5524'}
57%|█████▋ | 2280/4000 [2:40:08<2:00:30, 4.20s/it]
57%|█████▋ | 2281/4000 [2:40:12<2:00:46, 4.22s/it]
57%|█████▋ | 2282/4000 [2:40:17<2:01:20, 4.24s/it]
57%|█████▋ | 2283/4000 [2:40:21<2:01:50, 4.26s/it]
57%|█████▋ | 2284/4000 [2:40:25<2:04:02, 4.34s/it]
57%|█████▋ | 2285/4000 [2:40:30<2:02:03, 4.27s/it]
57%|█████▋ | 2286/4000 [2:40:34<2:00:51, 4.23s/it]
57%|█████▋ | 2287/4000 [2:40:38<1:59:55, 4.20s/it]
57%|█████▋ | 2288/4000 [2:40:42<1:59:19, 4.18s/it]
57%|█████▋ | 2289/4000 [2:40:46<1:58:47, 4.17s/it]
57%|█████▋ | 2290/4000 [2:40:50<1:59:17, 4.19s/it]
{'loss': '0.4553', 'grad_norm': '0.1379', 'learning_rate': '0.0002592', 'epoch': '0.5549'}
57%|█████▋ | 2290/4000 [2:40:50<1:59:17, 4.19s/it]
57%|█████▋ | 2291/4000 [2:40:55<2:04:00, 4.35s/it]
57%|█████▋ | 2292/4000 [2:40:59<2:02:23, 4.30s/it]
57%|█████▋ | 2293/4000 [2:41:03<2:00:57, 4.25s/it]
57%|█████▋ | 2294/4000 [2:41:08<2:00:01, 4.22s/it]
57%|█████▋ | 2295/4000 [2:41:12<1:59:28, 4.20s/it]
57%|█████▋ | 2296/4000 [2:41:16<2:00:48, 4.25s/it]
57%|█████▋ | 2297/4000 [2:41:20<2:01:37, 4.28s/it]
57%|█████▋ | 2298/4000 [2:41:25<2:02:06, 4.30s/it]
57%|█████▋ | 2299/4000 [2:41:29<2:00:39, 4.26s/it]
57%|█████▊ | 2300/4000 [2:41:33<2:00:28, 4.25s/it]
{'loss': '0.4624', 'grad_norm': '0.1325', 'learning_rate': '0.0002577', 'epoch': '0.5573'}
57%|█████▊ | 2300/4000 [2:41:33<2:00:28, 4.25s/it]
58%|█████▊ | 2301/4000 [2:41:37<2:00:56, 4.27s/it]
58%|█████▊ | 2302/4000 [2:41:42<2:00:21, 4.25s/it]
58%|█████▊ | 2303/4000 [2:41:46<1:59:26, 4.22s/it]
58%|█████▊ | 2304/4000 [2:41:50<2:01:47, 4.31s/it]
58%|█████▊ | 2305/4000 [2:41:54<2:00:21, 4.26s/it]
58%|█████▊ | 2306/4000 [2:41:59<1:59:20, 4.23s/it]
58%|█████▊ | 2307/4000 [2:42:03<2:01:00, 4.29s/it]
58%|█████▊ | 2308/4000 [2:42:07<2:02:12, 4.33s/it]
58%|█████▊ | 2309/4000 [2:42:12<2:00:36, 4.28s/it]
58%|█████▊ | 2310/4000 [2:42:16<1:59:26, 4.24s/it]
{'loss': '0.4456', 'grad_norm': '0.1388', 'learning_rate': '0.0002562', 'epoch': '0.5597'}
58%|█████▊ | 2310/4000 [2:42:16<1:59:26, 4.24s/it]
58%|█████▊ | 2311/4000 [2:42:20<2:01:33, 4.32s/it]
58%|█████▊ | 2312/4000 [2:42:24<2:00:18, 4.28s/it]
58%|█████▊ | 2313/4000 [2:42:29<2:00:26, 4.28s/it]
58%|█████▊ | 2314/4000 [2:42:33<1:59:20, 4.25s/it]
58%|█████▊ | 2315/4000 [2:42:37<1:58:25, 4.22s/it]
58%|█████▊ | 2316/4000 [2:42:41<1:57:52, 4.20s/it]
58%|█████▊ | 2317/4000 [2:42:45<1:57:21, 4.18s/it]
58%|█████▊ | 2318/4000 [2:42:50<2:00:28, 4.30s/it]
58%|█████▊ | 2319/4000 [2:42:54<2:00:14, 4.29s/it]
58%|█████▊ | 2320/4000 [2:42:59<2:00:22, 4.30s/it]
{'loss': '0.4758', 'grad_norm': '0.1284', 'learning_rate': '0.0002547', 'epoch': '0.5621'}
58%|█████▊ | 2320/4000 [2:42:59<2:00:22, 4.30s/it]
58%|█████▊ | 2321/4000 [2:43:03<1:59:24, 4.27s/it]
58%|█████▊ | 2322/4000 [2:43:07<1:58:19, 4.23s/it]
58%|█████▊ | 2323/4000 [2:43:11<1:58:17, 4.23s/it]
58%|█████▊ | 2324/4000 [2:43:15<1:59:27, 4.28s/it]
58%|█████▊ | 2325/4000 [2:43:20<2:01:58, 4.37s/it]
58%|█████▊ | 2326/4000 [2:43:24<2:00:06, 4.30s/it]
58%|█████▊ | 2327/4000 [2:43:28<1:58:44, 4.26s/it]
58%|█████▊ | 2328/4000 [2:43:33<1:57:42, 4.22s/it]
58%|█████▊ | 2329/4000 [2:43:37<1:56:54, 4.20s/it]
58%|█████▊ | 2330/4000 [2:43:41<1:58:19, 4.25s/it]
{'loss': '0.4533', 'grad_norm': '0.1235', 'learning_rate': '0.0002532', 'epoch': '0.5646'}
58%|█████▊ | 2330/4000 [2:43:41<1:58:19, 4.25s/it]
58%|█████▊ | 2331/4000 [2:43:46<2:00:57, 4.35s/it]
58%|█████▊ | 2332/4000 [2:43:50<1:59:11, 4.29s/it]
58%|█████▊ | 2333/4000 [2:43:54<1:58:02, 4.25s/it]
58%|█████▊ | 2334/4000 [2:43:58<1:57:06, 4.22s/it]
58%|█████▊ | 2335/4000 [2:44:02<1:56:24, 4.19s/it]
58%|█████▊ | 2336/4000 [2:44:06<1:56:00, 4.18s/it]
58%|█████▊ | 2337/4000 [2:44:11<1:56:49, 4.21s/it]
58%|█████▊ | 2338/4000 [2:44:15<2:01:03, 4.37s/it]
58%|█████▊ | 2339/4000 [2:44:20<2:00:27, 4.35s/it]
58%|█████▊ | 2340/4000 [2:44:24<1:59:03, 4.30s/it]
{'loss': '0.4112', 'grad_norm': '0.1257', 'learning_rate': '0.0002517', 'epoch': '0.567'}
58%|█████▊ | 2340/4000 [2:44:24<1:59:03, 4.30s/it]
59%|█████▊ | 2341/4000 [2:44:28<1:57:51, 4.26s/it]
59%|█████▊ | 2342/4000 [2:44:32<1:58:49, 4.30s/it]
59%|█████▊ | 2343/4000 [2:44:37<1:57:59, 4.27s/it]
59%|█████▊ | 2344/4000 [2:44:41<1:56:50, 4.23s/it]
59%|█████▊ | 2345/4000 [2:44:45<1:58:50, 4.31s/it]
59%|█████▊ | 2346/4000 [2:44:49<1:57:22, 4.26s/it]
59%|█████▊ | 2347/4000 [2:44:54<1:57:16, 4.26s/it]
59%|█████▊ | 2348/4000 [2:44:58<1:56:16, 4.22s/it]
59%|█████▊ | 2349/4000 [2:45:02<1:55:28, 4.20s/it]
59%|█████▉ | 2350/4000 [2:45:06<1:54:50, 4.18s/it]
{'loss': '0.423', 'grad_norm': '0.1292', 'learning_rate': '0.0002502', 'epoch': '0.5694'}
59%|█████▉ | 2350/4000 [2:45:06<1:54:50, 4.18s/it]
59%|█████▉ | 2351/4000 [2:45:10<1:56:10, 4.23s/it]
59%|█████▉ | 2352/4000 [2:45:15<1:56:42, 4.25s/it]
59%|█████▉ | 2353/4000 [2:45:19<1:55:44, 4.22s/it]
59%|█████▉ | 2354/4000 [2:45:23<1:55:01, 4.19s/it]
59%|█████▉ | 2355/4000 [2:45:27<1:55:57, 4.23s/it]
59%|█████▉ | 2356/4000 [2:45:32<1:57:34, 4.29s/it]
59%|█████▉ | 2357/4000 [2:45:36<1:57:30, 4.29s/it]
59%|█████▉ | 2358/4000 [2:45:41<1:59:24, 4.36s/it]
59%|█████▉ | 2359/4000 [2:45:45<2:00:18, 4.40s/it]
59%|█████▉ | 2360/4000 [2:45:49<1:58:09, 4.32s/it]
{'loss': '0.4463', 'grad_norm': '0.1273', 'learning_rate': '0.0002486', 'epoch': '0.5718'}
59%|█████▉ | 2360/4000 [2:45:49<1:58:09, 4.32s/it]
59%|█████▉ | 2361/4000 [2:45:53<1:56:35, 4.27s/it]
59%|█████▉ | 2362/4000 [2:45:57<1:55:34, 4.23s/it]
59%|█████▉ | 2363/4000 [2:46:02<1:55:32, 4.23s/it]
59%|█████▉ | 2364/4000 [2:46:06<1:56:15, 4.26s/it]
59%|█████▉ | 2365/4000 [2:46:11<1:58:41, 4.36s/it]
59%|█████▉ | 2366/4000 [2:46:15<1:56:58, 4.30s/it]
59%|█████▉ | 2367/4000 [2:46:19<1:55:36, 4.25s/it]
59%|█████▉ | 2368/4000 [2:46:23<1:54:42, 4.22s/it]
59%|█████▉ | 2369/4000 [2:46:27<1:54:02, 4.20s/it]
59%|█████▉ | 2370/4000 [2:46:31<1:53:30, 4.18s/it]
{'loss': '0.4244', 'grad_norm': '0.1066', 'learning_rate': '0.0002471', 'epoch': '0.5742'}
59%|█████▉ | 2370/4000 [2:46:31<1:53:30, 4.18s/it]
59%|█████▉ | 2371/4000 [2:46:36<1:54:09, 4.20s/it]
59%|█████▉ | 2372/4000 [2:46:40<1:56:54, 4.31s/it]
59%|█████▉ | 2373/4000 [2:46:44<1:56:13, 4.29s/it]
59%|█████▉ | 2374/4000 [2:46:49<1:56:39, 4.31s/it]
59%|█████▉ | 2375/4000 [2:46:53<1:56:48, 4.31s/it]
59%|█████▉ | 2376/4000 [2:46:57<1:56:34, 4.31s/it]
59%|█████▉ | 2377/4000 [2:47:02<1:55:09, 4.26s/it]
59%|█████▉ | 2378/4000 [2:47:06<1:55:37, 4.28s/it]
59%|█████▉ | 2379/4000 [2:47:10<1:55:54, 4.29s/it]
60%|█████▉ | 2380/4000 [2:47:14<1:56:10, 4.30s/it]
{'loss': '0.4588', 'grad_norm': '0.1298', 'learning_rate': '0.0002456', 'epoch': '0.5767'}
60%|█████▉ | 2380/4000 [2:47:14<1:56:10, 4.30s/it]
60%|█████▉ | 2381/4000 [2:47:19<1:54:52, 4.26s/it]
60%|█████▉ | 2382/4000 [2:47:23<1:53:47, 4.22s/it]
60%|█████▉ | 2383/4000 [2:47:27<1:52:57, 4.19s/it]
60%|█████▉ | 2384/4000 [2:47:31<1:52:29, 4.18s/it]
60%|█████▉ | 2385/4000 [2:47:36<1:55:16, 4.28s/it]
60%|█████▉ | 2386/4000 [2:47:40<1:54:02, 4.24s/it]
60%|█████▉ | 2387/4000 [2:47:44<1:53:19, 4.22s/it]
60%|█████▉ | 2388/4000 [2:47:48<1:54:41, 4.27s/it]
60%|█████▉ | 2389/4000 [2:47:52<1:53:32, 4.23s/it]
60%|█████▉ | 2390/4000 [2:47:57<1:52:47, 4.20s/it]
{'loss': '0.422', 'grad_norm': '0.1311', 'learning_rate': '0.0002441', 'epoch': '0.5791'}
60%|█████▉ | 2390/4000 [2:47:57<1:52:47, 4.20s/it]
60%|█████▉ | 2391/4000 [2:48:01<1:54:43, 4.28s/it]
60%|█████▉ | 2392/4000 [2:48:06<1:58:38, 4.43s/it]
60%|█████▉ | 2393/4000 [2:48:10<1:58:17, 4.42s/it]
60%|█████▉ | 2394/4000 [2:48:14<1:55:59, 4.33s/it]
60%|█████▉ | 2395/4000 [2:48:18<1:54:16, 4.27s/it]
60%|█████▉ | 2396/4000 [2:48:23<1:53:07, 4.23s/it]
60%|█████▉ | 2397/4000 [2:48:27<1:53:45, 4.26s/it]
60%|█████▉ | 2398/4000 [2:48:31<1:52:43, 4.22s/it]
60%|█████▉ | 2399/4000 [2:48:36<1:54:56, 4.31s/it]
60%|██████ | 2400/4000 [2:48:40<1:53:28, 4.26s/it]
{'loss': '0.4471', 'grad_norm': '0.1121', 'learning_rate': '0.0002426', 'epoch': '0.5815'}
60%|██████ | 2400/4000 [2:48:40<1:53:28, 4.26s/it]
60%|██████ | 2401/4000 [2:48:44<1:52:23, 4.22s/it]
60%|██████ | 2402/4000 [2:48:48<1:51:42, 4.19s/it]
60%|██████ | 2403/4000 [2:48:52<1:51:15, 4.18s/it]
60%|██████ | 2404/4000 [2:48:57<1:53:17, 4.26s/it]
60%|██████ | 2405/4000 [2:49:01<1:56:21, 4.38s/it]
60%|██████ | 2406/4000 [2:49:05<1:54:19, 4.30s/it]
60%|██████ | 2407/4000 [2:49:09<1:53:17, 4.27s/it]
60%|██████ | 2408/4000 [2:49:14<1:53:14, 4.27s/it]
60%|██████ | 2409/4000 [2:49:18<1:52:50, 4.26s/it]
60%|██████ | 2410/4000 [2:49:22<1:53:57, 4.30s/it]
{'loss': '0.4311', 'grad_norm': '0.113', 'learning_rate': '0.0002411', 'epoch': '0.5839'}
60%|██████ | 2410/4000 [2:49:22<1:53:57, 4.30s/it]
60%|██████ | 2411/4000 [2:49:27<1:52:47, 4.26s/it]
60%|██████ | 2412/4000 [2:49:31<1:54:37, 4.33s/it]
60%|██████ | 2413/4000 [2:49:35<1:54:35, 4.33s/it]
60%|██████ | 2414/4000 [2:49:40<1:53:27, 4.29s/it]
60%|██████ | 2415/4000 [2:49:44<1:52:06, 4.24s/it]
60%|██████ | 2416/4000 [2:49:48<1:51:10, 4.21s/it]
60%|██████ | 2417/4000 [2:49:52<1:50:25, 4.19s/it]
60%|██████ | 2418/4000 [2:49:56<1:50:00, 4.17s/it]
60%|██████ | 2419/4000 [2:50:01<1:52:31, 4.27s/it]
60%|██████ | 2420/4000 [2:50:05<1:51:25, 4.23s/it]
{'loss': '0.4425', 'grad_norm': '0.1242', 'learning_rate': '0.0002395', 'epoch': '0.5864'}
60%|██████ | 2420/4000 [2:50:05<1:51:25, 4.23s/it]
61%|██████ | 2421/4000 [2:50:09<1:53:25, 4.31s/it]
61%|██████ | 2422/4000 [2:50:13<1:52:06, 4.26s/it]
61%|██████ | 2423/4000 [2:50:18<1:51:00, 4.22s/it]
61%|██████ | 2424/4000 [2:50:22<1:50:52, 4.22s/it]
61%|██████ | 2425/4000 [2:50:26<1:52:16, 4.28s/it]
61%|██████ | 2426/4000 [2:50:30<1:52:29, 4.29s/it]
61%|██████ | 2427/4000 [2:50:35<1:53:50, 4.34s/it]
61%|██████ | 2428/4000 [2:50:39<1:53:05, 4.32s/it]
61%|██████ | 2429/4000 [2:50:43<1:51:28, 4.26s/it]
61%|██████ | 2430/4000 [2:50:48<1:52:24, 4.30s/it]
{'loss': '0.4242', 'grad_norm': '0.1184', 'learning_rate': '0.000238', 'epoch': '0.5888'}
61%|██████ | 2430/4000 [2:50:48<1:52:24, 4.30s/it]
61%|██████ | 2431/4000 [2:50:52<1:51:03, 4.25s/it]
61%|██████ | 2432/4000 [2:50:56<1:53:09, 4.33s/it]
61%|██████ | 2433/4000 [2:51:00<1:51:29, 4.27s/it]
61%|██████ | 2434/4000 [2:51:05<1:50:18, 4.23s/it]
61%|██████ | 2435/4000 [2:51:09<1:49:30, 4.20s/it]
61%|██████ | 2436/4000 [2:51:13<1:48:50, 4.18s/it]
61%|██████ | 2437/4000 [2:51:17<1:48:20, 4.16s/it]
61%|██████ | 2438/4000 [2:51:21<1:50:03, 4.23s/it]
61%|██████ | 2439/4000 [2:51:26<1:52:16, 4.32s/it]
61%|██████ | 2440/4000 [2:51:30<1:50:47, 4.26s/it]
{'loss': '0.3936', 'grad_norm': '0.1247', 'learning_rate': '0.0002365', 'epoch': '0.5912'}
61%|██████ | 2440/4000 [2:51:30<1:50:47, 4.26s/it]
61%|██████ | 2441/4000 [2:51:34<1:50:02, 4.24s/it]
61%|██████ | 2442/4000 [2:51:38<1:49:45, 4.23s/it]
61%|██████ | 2443/4000 [2:51:43<1:49:25, 4.22s/it]
61%|██████ | 2444/4000 [2:51:47<1:49:40, 4.23s/it]
61%|██████ | 2445/4000 [2:51:51<1:50:47, 4.28s/it]
61%|██████ | 2446/4000 [2:51:56<1:52:22, 4.34s/it]
61%|██████ | 2447/4000 [2:52:00<1:52:48, 4.36s/it]
61%|██████ | 2448/4000 [2:52:04<1:51:06, 4.30s/it]
61%|██████ | 2449/4000 [2:52:08<1:49:49, 4.25s/it]
61%|██████▏ | 2450/4000 [2:52:13<1:48:49, 4.21s/it]
{'loss': '0.4206', 'grad_norm': '0.117', 'learning_rate': '0.000235', 'epoch': '0.5936'}
61%|██████▏ | 2450/4000 [2:52:13<1:48:49, 4.21s/it]
61%|██████▏ | 2451/4000 [2:52:17<1:48:07, 4.19s/it]
61%|██████▏ | 2452/4000 [2:52:21<1:47:41, 4.17s/it]
61%|██████▏ | 2453/4000 [2:52:25<1:50:24, 4.28s/it]
61%|██████▏ | 2454/4000 [2:52:30<1:50:43, 4.30s/it]
61%|██████▏ | 2455/4000 [2:52:34<1:49:38, 4.26s/it]
61%|██████▏ | 2456/4000 [2:52:38<1:48:37, 4.22s/it]
61%|██████▏ | 2457/4000 [2:52:42<1:47:55, 4.20s/it]
61%|██████▏ | 2458/4000 [2:52:46<1:48:26, 4.22s/it]
61%|██████▏ | 2459/4000 [2:52:51<1:51:03, 4.32s/it]
62%|██████▏ | 2460/4000 [2:52:55<1:49:57, 4.28s/it]
{'loss': '0.4193', 'grad_norm': '0.1188', 'learning_rate': '0.0002335', 'epoch': '0.5961'}
62%|██████▏ | 2460/4000 [2:52:55<1:49:57, 4.28s/it]
62%|██████▏ | 2461/4000 [2:52:59<1:48:43, 4.24s/it]
62%|██████▏ | 2462/4000 [2:53:04<1:51:30, 4.35s/it]
62%|██████▏ | 2463/4000 [2:53:08<1:49:59, 4.29s/it]
62%|██████▏ | 2464/4000 [2:53:13<1:52:08, 4.38s/it]
62%|██████▏ | 2465/4000 [2:53:17<1:50:10, 4.31s/it]
62%|██████▏ | 2466/4000 [2:53:21<1:51:44, 4.37s/it]
62%|██████▏ | 2467/4000 [2:53:25<1:49:51, 4.30s/it]
62%|██████▏ | 2468/4000 [2:53:30<1:48:30, 4.25s/it]
62%|██████▏ | 2469/4000 [2:53:34<1:47:28, 4.21s/it]
62%|██████▏ | 2470/4000 [2:53:38<1:46:48, 4.19s/it]
{'loss': '0.4233', 'grad_norm': '0.1265', 'learning_rate': '0.000232', 'epoch': '0.5985'}
62%|██████▏ | 2470/4000 [2:53:38<1:46:48, 4.19s/it]
62%|██████▏ | 2471/4000 [2:53:42<1:47:20, 4.21s/it]
62%|██████▏ | 2472/4000 [2:53:46<1:46:41, 4.19s/it]
62%|██████▏ | 2473/4000 [2:53:51<1:49:12, 4.29s/it]
62%|██████▏ | 2474/4000 [2:53:55<1:47:53, 4.24s/it]
62%|██████▏ | 2475/4000 [2:53:59<1:48:20, 4.26s/it]
62%|██████▏ | 2476/4000 [2:54:03<1:48:01, 4.25s/it]
62%|██████▏ | 2477/4000 [2:54:08<1:47:24, 4.23s/it]
62%|██████▏ | 2478/4000 [2:54:12<1:46:30, 4.20s/it]
62%|██████▏ | 2479/4000 [2:54:17<1:51:39, 4.40s/it]
62%|██████▏ | 2480/4000 [2:54:21<1:52:25, 4.44s/it]
{'loss': '0.429', 'grad_norm': '0.1131', 'learning_rate': '0.0002305', 'epoch': '0.6009'}
62%|██████▏ | 2480/4000 [2:54:21<1:52:25, 4.44s/it]
62%|██████▏ | 2481/4000 [2:54:25<1:50:26, 4.36s/it]
62%|██████▏ | 2482/4000 [2:54:29<1:48:41, 4.30s/it]
62%|██████▏ | 2483/4000 [2:54:34<1:47:24, 4.25s/it]
62%|██████▏ | 2484/4000 [2:54:38<1:46:31, 4.22s/it]
62%|██████▏ | 2485/4000 [2:54:42<1:45:52, 4.19s/it]
62%|██████▏ | 2486/4000 [2:54:46<1:48:26, 4.30s/it]
62%|██████▏ | 2487/4000 [2:54:51<1:47:06, 4.25s/it]
62%|██████▏ | 2488/4000 [2:54:55<1:46:58, 4.25s/it]
62%|██████▏ | 2489/4000 [2:54:59<1:46:07, 4.21s/it]
62%|██████▏ | 2490/4000 [2:55:03<1:45:30, 4.19s/it]
{'loss': '0.4325', 'grad_norm': '0.1439', 'learning_rate': '0.0002289', 'epoch': '0.6033'}
62%|██████▏ | 2490/4000 [2:55:03<1:45:30, 4.19s/it]
62%|██████▏ | 2491/4000 [2:55:07<1:44:59, 4.17s/it]
62%|██████▏ | 2492/4000 [2:55:11<1:45:22, 4.19s/it]
62%|██████▏ | 2493/4000 [2:55:16<1:49:33, 4.36s/it]
62%|██████▏ | 2494/4000 [2:55:20<1:47:50, 4.30s/it]
62%|██████▏ | 2495/4000 [2:55:24<1:46:36, 4.25s/it]
62%|██████▏ | 2496/4000 [2:55:29<1:46:47, 4.26s/it]
62%|██████▏ | 2497/4000 [2:55:33<1:48:39, 4.34s/it]
62%|██████▏ | 2498/4000 [2:55:37<1:47:03, 4.28s/it]
62%|██████▏ | 2499/4000 [2:55:42<1:45:47, 4.23s/it]
62%|██████▎ | 2500/4000 [2:55:46<1:47:57, 4.32s/it]
{'loss': '0.4406', 'grad_norm': '0.1617', 'learning_rate': '0.0002274', 'epoch': '0.6057'}
62%|██████▎ | 2500/4000 [2:55:46<1:47:57, 4.32s/it]
63%|██████▎ | 2501/4000 [2:55:50<1:46:30, 4.26s/it]
63%|██████▎ | 2502/4000 [2:55:54<1:45:27, 4.22s/it]
63%|██████▎ | 2503/4000 [2:55:58<1:44:43, 4.20s/it]
63%|██████▎ | 2504/4000 [2:56:03<1:44:23, 4.19s/it]
63%|██████▎ | 2505/4000 [2:56:07<1:44:50, 4.21s/it]
63%|██████▎ | 2506/4000 [2:56:11<1:45:29, 4.24s/it]
63%|██████▎ | 2507/4000 [2:56:16<1:46:05, 4.26s/it]
63%|██████▎ | 2508/4000 [2:56:20<1:45:10, 4.23s/it]
63%|██████▎ | 2509/4000 [2:56:24<1:45:34, 4.25s/it]
63%|██████▎ | 2510/4000 [2:56:28<1:45:25, 4.25s/it]
{'loss': '0.4191', 'grad_norm': '0.1201', 'learning_rate': '0.0002259', 'epoch': '0.6082'}
63%|██████▎ | 2510/4000 [2:56:28<1:45:25, 4.25s/it]
63%|██████▎ | 2511/4000 [2:56:32<1:44:47, 4.22s/it]
63%|██████▎ | 2512/4000 [2:56:37<1:44:07, 4.20s/it]
63%|██████▎ | 2513/4000 [2:56:41<1:47:24, 4.33s/it]
63%|██████▎ | 2514/4000 [2:56:45<1:46:45, 4.31s/it]
63%|██████▎ | 2515/4000 [2:56:50<1:46:30, 4.30s/it]
63%|██████▎ | 2516/4000 [2:56:54<1:45:13, 4.25s/it]
63%|██████▎ | 2517/4000 [2:56:58<1:44:18, 4.22s/it]
63%|██████▎ | 2518/4000 [2:57:02<1:43:40, 4.20s/it]
63%|██████▎ | 2519/4000 [2:57:06<1:43:17, 4.18s/it]
63%|██████▎ | 2520/4000 [2:57:11<1:46:15, 4.31s/it]
{'loss': '0.4117', 'grad_norm': '0.1111', 'learning_rate': '0.0002244', 'epoch': '0.6106'}
63%|██████▎ | 2520/4000 [2:57:11<1:46:15, 4.31s/it]
63%|██████▎ | 2521/4000 [2:57:15<1:45:54, 4.30s/it]
63%|██████▎ | 2522/4000 [2:57:19<1:44:48, 4.25s/it]
63%|██████▎ | 2523/4000 [2:57:23<1:43:56, 4.22s/it]
63%|██████▎ | 2524/4000 [2:57:28<1:43:16, 4.20s/it]
63%|██████▎ | 2525/4000 [2:57:32<1:42:46, 4.18s/it]
63%|██████▎ | 2526/4000 [2:57:36<1:43:57, 4.23s/it]
63%|██████▎ | 2527/4000 [2:57:41<1:46:40, 4.35s/it]
63%|██████▎ | 2528/4000 [2:57:45<1:45:12, 4.29s/it]
63%|██████▎ | 2529/4000 [2:57:49<1:44:06, 4.25s/it]
63%|██████▎ | 2530/4000 [2:57:53<1:44:24, 4.26s/it]
{'loss': '0.4041', 'grad_norm': '0.1393', 'learning_rate': '0.0002229', 'epoch': '0.613'}
63%|██████▎ | 2530/4000 [2:57:53<1:44:24, 4.26s/it]
63%|██████▎ | 2531/4000 [2:57:58<1:44:38, 4.27s/it]
63%|██████▎ | 2532/4000 [2:58:02<1:46:55, 4.37s/it]
63%|██████▎ | 2533/4000 [2:58:07<1:50:52, 4.53s/it]
63%|██████▎ | 2534/4000 [2:58:11<1:48:02, 4.42s/it]
63%|██████▎ | 2535/4000 [2:58:15<1:45:53, 4.34s/it]
63%|██████▎ | 2536/4000 [2:58:20<1:44:27, 4.28s/it]
63%|██████▎ | 2537/4000 [2:58:24<1:43:19, 4.24s/it]
63%|██████▎ | 2538/4000 [2:58:28<1:43:57, 4.27s/it]
63%|██████▎ | 2539/4000 [2:58:32<1:42:58, 4.23s/it]
64%|██████▎ | 2540/4000 [2:58:37<1:45:09, 4.32s/it]
{'loss': '0.4366', 'grad_norm': '0.1341', 'learning_rate': '0.0002214', 'epoch': '0.6154'}
64%|██████▎ | 2540/4000 [2:58:37<1:45:09, 4.32s/it]
64%|██████▎ | 2541/4000 [2:58:41<1:43:42, 4.26s/it]
64%|██████▎ | 2542/4000 [2:58:45<1:43:55, 4.28s/it]
64%|██████▎ | 2543/4000 [2:58:49<1:43:19, 4.25s/it]
64%|██████▎ | 2544/4000 [2:58:54<1:44:31, 4.31s/it]
64%|██████▎ | 2545/4000 [2:58:58<1:43:27, 4.27s/it]
64%|██████▎ | 2546/4000 [2:59:02<1:42:26, 4.23s/it]
64%|██████▎ | 2547/4000 [2:59:07<1:45:54, 4.37s/it]
64%|██████▎ | 2548/4000 [2:59:11<1:45:45, 4.37s/it]
64%|██████▎ | 2549/4000 [2:59:15<1:45:03, 4.34s/it]
64%|██████▍ | 2550/4000 [2:59:20<1:43:30, 4.28s/it]
{'loss': '0.4377', 'grad_norm': '0.1391', 'learning_rate': '0.0002198', 'epoch': '0.6179'}
64%|██████▍ | 2550/4000 [2:59:20<1:43:30, 4.28s/it]
64%|██████▍ | 2551/4000 [2:59:24<1:42:21, 4.24s/it]
64%|██████▍ | 2552/4000 [2:59:28<1:41:28, 4.20s/it]
64%|██████▍ | 2553/4000 [2:59:32<1:40:55, 4.18s/it]
64%|██████▍ | 2554/4000 [2:59:37<1:43:50, 4.31s/it]
64%|██████▍ | 2555/4000 [2:59:41<1:43:09, 4.28s/it]
64%|██████▍ | 2556/4000 [2:59:45<1:42:00, 4.24s/it]
64%|██████▍ | 2557/4000 [2:59:49<1:41:07, 4.20s/it]
64%|██████▍ | 2558/4000 [2:59:53<1:40:31, 4.18s/it]
64%|██████▍ | 2559/4000 [2:59:57<1:40:42, 4.19s/it]
64%|██████▍ | 2560/4000 [3:00:02<1:42:58, 4.29s/it]
{'loss': '0.3938', 'grad_norm': '0.1115', 'learning_rate': '0.0002183', 'epoch': '0.6203'}
64%|██████▍ | 2560/4000 [3:00:02<1:42:58, 4.29s/it]
64%|██████▍ | 2561/4000 [3:00:06<1:43:31, 4.32s/it]
64%|██████▍ | 2562/4000 [3:00:10<1:42:09, 4.26s/it]
64%|██████▍ | 2563/4000 [3:00:15<1:41:08, 4.22s/it]
64%|██████▍ | 2564/4000 [3:00:19<1:42:10, 4.27s/it]
64%|██████▍ | 2565/4000 [3:00:23<1:42:16, 4.28s/it]
64%|██████▍ | 2566/4000 [3:00:27<1:41:24, 4.24s/it]
64%|██████▍ | 2567/4000 [3:00:32<1:45:10, 4.40s/it]
64%|██████▍ | 2568/4000 [3:00:36<1:43:39, 4.34s/it]
64%|██████▍ | 2569/4000 [3:00:41<1:42:01, 4.28s/it]
64%|██████▍ | 2570/4000 [3:00:45<1:40:53, 4.23s/it]
{'loss': '0.3954', 'grad_norm': '0.1285', 'learning_rate': '0.0002168', 'epoch': '0.6227'}
64%|██████▍ | 2570/4000 [3:00:45<1:40:53, 4.23s/it]
64%|██████▍ | 2571/4000 [3:00:49<1:41:10, 4.25s/it]
64%|██████▍ | 2572/4000 [3:00:53<1:40:17, 4.21s/it]
64%|██████▍ | 2573/4000 [3:00:57<1:39:36, 4.19s/it]
64%|██████▍ | 2574/4000 [3:01:02<1:41:48, 4.28s/it]
64%|██████▍ | 2575/4000 [3:01:06<1:40:34, 4.23s/it]
64%|██████▍ | 2576/4000 [3:01:10<1:41:39, 4.28s/it]
64%|██████▍ | 2577/4000 [3:01:14<1:40:43, 4.25s/it]
64%|██████▍ | 2578/4000 [3:01:19<1:40:31, 4.24s/it]
64%|██████▍ | 2579/4000 [3:01:23<1:39:46, 4.21s/it]
64%|██████▍ | 2580/4000 [3:01:27<1:40:28, 4.25s/it]
{'loss': '0.4072', 'grad_norm': '0.1221', 'learning_rate': '0.0002153', 'epoch': '0.6251'}
64%|██████▍ | 2580/4000 [3:01:27<1:40:28, 4.25s/it]
65%|██████▍ | 2581/4000 [3:01:32<1:42:35, 4.34s/it]
65%|██████▍ | 2582/4000 [3:01:36<1:41:55, 4.31s/it]
65%|██████▍ | 2583/4000 [3:01:40<1:40:40, 4.26s/it]
65%|██████▍ | 2584/4000 [3:01:44<1:40:22, 4.25s/it]
65%|██████▍ | 2585/4000 [3:01:48<1:39:34, 4.22s/it]
65%|██████▍ | 2586/4000 [3:01:53<1:38:49, 4.19s/it]
65%|██████▍ | 2587/4000 [3:01:57<1:40:58, 4.29s/it]
65%|██████▍ | 2588/4000 [3:02:01<1:41:18, 4.31s/it]
65%|██████▍ | 2589/4000 [3:02:06<1:40:11, 4.26s/it]
65%|██████▍ | 2590/4000 [3:02:10<1:39:23, 4.23s/it]
{'loss': '0.3849', 'grad_norm': '0.1156', 'learning_rate': '0.0002138', 'epoch': '0.6276'}
65%|██████▍ | 2590/4000 [3:02:10<1:39:23, 4.23s/it]
65%|██████▍ | 2591/4000 [3:02:14<1:38:40, 4.20s/it]
65%|██████▍ | 2592/4000 [3:02:18<1:38:13, 4.19s/it]
65%|██████▍ | 2593/4000 [3:02:22<1:38:26, 4.20s/it]
65%|██████▍ | 2594/4000 [3:02:27<1:40:37, 4.29s/it]
65%|██████▍ | 2595/4000 [3:02:31<1:41:11, 4.32s/it]
65%|██████▍ | 2596/4000 [3:02:35<1:39:52, 4.27s/it]
65%|██████▍ | 2597/4000 [3:02:39<1:38:56, 4.23s/it]
65%|██████▍ | 2598/4000 [3:02:44<1:39:18, 4.25s/it]
65%|██████▍ | 2599/4000 [3:02:48<1:39:57, 4.28s/it]
65%|██████▌ | 2600/4000 [3:02:52<1:38:56, 4.24s/it]
{'loss': '0.4373', 'grad_norm': '0.1099', 'learning_rate': '0.0002123', 'epoch': '0.63'}
65%|██████▌ | 2600/4000 [3:02:52<1:38:56, 4.24s/it]
65%|██████▌ | 2601/4000 [3:02:57<1:42:09, 4.38s/it]
65%|██████▌ | 2602/4000 [3:03:01<1:40:28, 4.31s/it]
65%|██████▌ | 2603/4000 [3:03:05<1:39:10, 4.26s/it]
65%|██████▌ | 2604/4000 [3:03:09<1:39:17, 4.27s/it]
65%|██████▌ | 2605/4000 [3:03:14<1:38:25, 4.23s/it]
65%|██████▌ | 2606/4000 [3:03:18<1:37:39, 4.20s/it]
65%|██████▌ | 2607/4000 [3:03:22<1:38:30, 4.24s/it]
65%|██████▌ | 2608/4000 [3:03:26<1:38:43, 4.26s/it]
65%|██████▌ | 2609/4000 [3:03:31<1:38:11, 4.24s/it]
65%|██████▌ | 2610/4000 [3:03:35<1:37:49, 4.22s/it]
{'loss': '0.4537', 'grad_norm': '0.1235', 'learning_rate': '0.0002108', 'epoch': '0.6324'}
65%|██████▌ | 2610/4000 [3:03:35<1:37:49, 4.22s/it]
65%|██████▌ | 2611/4000 [3:03:39<1:37:15, 4.20s/it]
65%|██████▌ | 2612/4000 [3:03:43<1:37:30, 4.22s/it]
65%|██████▌ | 2613/4000 [3:03:47<1:36:57, 4.19s/it]
65%|██████▌ | 2614/4000 [3:03:52<1:38:25, 4.26s/it]
65%|██████▌ | 2615/4000 [3:03:56<1:39:04, 4.29s/it]
65%|██████▌ | 2616/4000 [3:04:00<1:38:51, 4.29s/it]
65%|██████▌ | 2617/4000 [3:04:05<1:37:55, 4.25s/it]
65%|██████▌ | 2618/4000 [3:04:09<1:37:21, 4.23s/it]
65%|██████▌ | 2619/4000 [3:04:13<1:38:52, 4.30s/it]
66%|██████▌ | 2620/4000 [3:04:17<1:37:45, 4.25s/it]
{'loss': '0.4175', 'grad_norm': '0.112', 'learning_rate': '0.0002092', 'epoch': '0.6348'}
66%|██████▌ | 2620/4000 [3:04:17<1:37:45, 4.25s/it]
66%|██████▌ | 2621/4000 [3:04:22<1:40:50, 4.39s/it]
66%|██████▌ | 2622/4000 [3:04:26<1:39:04, 4.31s/it]
66%|██████▌ | 2623/4000 [3:04:30<1:37:48, 4.26s/it]
66%|██████▌ | 2624/4000 [3:04:34<1:36:53, 4.22s/it]
66%|██████▌ | 2625/4000 [3:04:39<1:36:14, 4.20s/it]
66%|██████▌ | 2626/4000 [3:04:43<1:36:15, 4.20s/it]
66%|██████▌ | 2627/4000 [3:04:47<1:36:05, 4.20s/it]
66%|██████▌ | 2628/4000 [3:04:52<1:39:13, 4.34s/it]
66%|██████▌ | 2629/4000 [3:04:56<1:37:48, 4.28s/it]
66%|██████▌ | 2630/4000 [3:05:00<1:36:46, 4.24s/it]
{'loss': '0.4294', 'grad_norm': '0.1219', 'learning_rate': '0.0002077', 'epoch': '0.6372'}
66%|██████▌ | 2630/4000 [3:05:00<1:36:46, 4.24s/it]
66%|██████▌ | 2631/4000 [3:05:04<1:36:05, 4.21s/it]
66%|██████▌ | 2632/4000 [3:05:08<1:37:02, 4.26s/it]
66%|██████▌ | 2633/4000 [3:05:13<1:36:53, 4.25s/it]
66%|██████▌ | 2634/4000 [3:05:17<1:37:30, 4.28s/it]
66%|██████▌ | 2635/4000 [3:05:21<1:37:42, 4.29s/it]
66%|██████▌ | 2636/4000 [3:05:26<1:38:52, 4.35s/it]
66%|██████▌ | 2637/4000 [3:05:30<1:37:28, 4.29s/it]
66%|██████▌ | 2638/4000 [3:05:34<1:37:24, 4.29s/it]
66%|██████▌ | 2639/4000 [3:05:38<1:36:15, 4.24s/it]
66%|██████▌ | 2640/4000 [3:05:43<1:35:22, 4.21s/it]
{'loss': '0.4103', 'grad_norm': '0.1417', 'learning_rate': '0.0002062', 'epoch': '0.6397'}
66%|██████▌ | 2640/4000 [3:05:43<1:35:22, 4.21s/it]
66%|██████▌ | 2641/4000 [3:05:47<1:37:07, 4.29s/it]
66%|██████▌ | 2642/4000 [3:05:51<1:35:59, 4.24s/it]
66%|██████▌ | 2643/4000 [3:05:56<1:37:02, 4.29s/it]
66%|██████▌ | 2644/4000 [3:06:00<1:36:10, 4.26s/it]
66%|██████▌ | 2645/4000 [3:06:04<1:36:26, 4.27s/it]
66%|██████▌ | 2646/4000 [3:06:08<1:35:29, 4.23s/it]
66%|██████▌ | 2647/4000 [3:06:12<1:34:47, 4.20s/it]
66%|██████▌ | 2648/4000 [3:06:17<1:36:25, 4.28s/it]
66%|██████▌ | 2649/4000 [3:06:21<1:36:07, 4.27s/it]
66%|██████▋ | 2650/4000 [3:06:25<1:36:27, 4.29s/it]
{'loss': '0.4134', 'grad_norm': '0.1227', 'learning_rate': '0.0002047', 'epoch': '0.6421'}
66%|██████▋ | 2650/4000 [3:06:25<1:36:27, 4.29s/it]
66%|██████▋ | 2651/4000 [3:06:29<1:35:25, 4.24s/it]
66%|██████▋ | 2652/4000 [3:06:34<1:34:38, 4.21s/it]
66%|██████▋ | 2653/4000 [3:06:38<1:36:21, 4.29s/it]
66%|██████▋ | 2654/4000 [3:06:42<1:35:23, 4.25s/it]
66%|██████▋ | 2655/4000 [3:06:47<1:38:17, 4.39s/it]
66%|██████▋ | 2656/4000 [3:06:51<1:36:32, 4.31s/it]
66%|██████▋ | 2657/4000 [3:06:55<1:35:17, 4.26s/it]
66%|██████▋ | 2658/4000 [3:06:59<1:34:20, 4.22s/it]
66%|██████▋ | 2659/4000 [3:07:03<1:33:42, 4.19s/it]
66%|██████▋ | 2660/4000 [3:07:08<1:34:13, 4.22s/it]
{'loss': '0.4317', 'grad_norm': '0.1114', 'learning_rate': '0.0002032', 'epoch': '0.6445'}
66%|██████▋ | 2660/4000 [3:07:08<1:34:13, 4.22s/it]
67%|██████▋ | 2661/4000 [3:07:12<1:37:05, 4.35s/it]
67%|██████▋ | 2662/4000 [3:07:17<1:35:47, 4.30s/it]
67%|██████▋ | 2663/4000 [3:07:21<1:34:36, 4.25s/it]
67%|██████▋ | 2664/4000 [3:07:25<1:33:44, 4.21s/it]
67%|██████▋ | 2665/4000 [3:07:29<1:33:10, 4.19s/it]
67%|██████▋ | 2666/4000 [3:07:33<1:32:48, 4.17s/it]
67%|██████▋ | 2667/4000 [3:07:37<1:33:35, 4.21s/it]
67%|██████▋ | 2668/4000 [3:07:42<1:35:29, 4.30s/it]
67%|██████▋ | 2669/4000 [3:07:46<1:34:16, 4.25s/it]
67%|██████▋ | 2670/4000 [3:07:50<1:34:11, 4.25s/it]
{'loss': '0.4512', 'grad_norm': '0.1369', 'learning_rate': '0.0002017', 'epoch': '0.6469'}
67%|██████▋ | 2670/4000 [3:07:50<1:34:11, 4.25s/it]
67%|██████▋ | 2671/4000 [3:07:55<1:33:51, 4.24s/it]
67%|██████▋ | 2672/4000 [3:07:59<1:34:02, 4.25s/it]
67%|██████▋ | 2673/4000 [3:08:03<1:33:11, 4.21s/it]
67%|██████▋ | 2674/4000 [3:08:07<1:32:28, 4.18s/it]
67%|██████▋ | 2675/4000 [3:08:12<1:34:23, 4.27s/it]
67%|██████▋ | 2676/4000 [3:08:16<1:33:32, 4.24s/it]
67%|██████▋ | 2677/4000 [3:08:20<1:33:32, 4.24s/it]
67%|██████▋ | 2678/4000 [3:08:24<1:34:04, 4.27s/it]
67%|██████▋ | 2679/4000 [3:08:28<1:33:14, 4.23s/it]
67%|██████▋ | 2680/4000 [3:08:33<1:32:27, 4.20s/it]
{'loss': '0.4546', 'grad_norm': '0.1234', 'learning_rate': '0.0002002', 'epoch': '0.6494'}
67%|██████▋ | 2680/4000 [3:08:33<1:32:27, 4.20s/it]
67%|██████▋ | 2681/4000 [3:08:37<1:31:57, 4.18s/it]
67%|██████▋ | 2682/4000 [3:08:41<1:33:39, 4.26s/it]
67%|██████▋ | 2683/4000 [3:08:45<1:32:38, 4.22s/it]
67%|██████▋ | 2684/4000 [3:08:50<1:33:16, 4.25s/it]
67%|██████▋ | 2685/4000 [3:08:54<1:32:32, 4.22s/it]
67%|██████▋ | 2686/4000 [3:08:58<1:31:50, 4.19s/it]
67%|██████▋ | 2687/4000 [3:09:02<1:31:19, 4.17s/it]
67%|██████▋ | 2688/4000 [3:09:07<1:33:58, 4.30s/it]
67%|██████▋ | 2689/4000 [3:09:11<1:34:22, 4.32s/it]
67%|██████▋ | 2690/4000 [3:09:15<1:33:09, 4.27s/it]
{'loss': '0.4204', 'grad_norm': '0.1305', 'learning_rate': '0.0001986', 'epoch': '0.6518'}
67%|██████▋ | 2690/4000 [3:09:15<1:33:09, 4.27s/it]
67%|██████▋ | 2691/4000 [3:09:19<1:32:07, 4.22s/it]
67%|██████▋ | 2692/4000 [3:09:23<1:31:26, 4.19s/it]
67%|██████▋ | 2693/4000 [3:09:28<1:32:03, 4.23s/it]
67%|██████▋ | 2694/4000 [3:09:32<1:31:25, 4.20s/it]
67%|██████▋ | 2695/4000 [3:09:36<1:34:17, 4.34s/it]
67%|██████▋ | 2696/4000 [3:09:41<1:32:53, 4.27s/it]
67%|██████▋ | 2697/4000 [3:09:45<1:31:51, 4.23s/it]
67%|██████▋ | 2698/4000 [3:09:49<1:31:08, 4.20s/it]
67%|██████▋ | 2699/4000 [3:09:53<1:30:36, 4.18s/it]
68%|██████▊ | 2700/4000 [3:09:57<1:30:14, 4.16s/it]
{'loss': '0.4062', 'grad_norm': '0.1601', 'learning_rate': '0.0001971', 'epoch': '0.6542'}
68%|██████▊ | 2700/4000 [3:09:57<1:30:14, 4.16s/it]
68%|██████▊ | 2701/4000 [3:10:01<1:30:36, 4.18s/it]
68%|██████▊ | 2702/4000 [3:10:06<1:33:30, 4.32s/it]
68%|██████▊ | 2703/4000 [3:10:10<1:32:10, 4.26s/it]
68%|██████▊ | 2704/4000 [3:10:14<1:31:12, 4.22s/it]
68%|██████▊ | 2705/4000 [3:10:19<1:31:47, 4.25s/it]
68%|██████▊ | 2706/4000 [3:10:23<1:33:23, 4.33s/it]
68%|██████▊ | 2707/4000 [3:10:27<1:32:09, 4.28s/it]
68%|██████▊ | 2708/4000 [3:10:32<1:32:24, 4.29s/it]
68%|██████▊ | 2709/4000 [3:10:36<1:32:15, 4.29s/it]
68%|██████▊ | 2710/4000 [3:10:40<1:32:29, 4.30s/it]
{'loss': '0.4095', 'grad_norm': '0.121', 'learning_rate': '0.0001956', 'epoch': '0.6566'}
68%|██████▊ | 2710/4000 [3:10:40<1:32:29, 4.30s/it]
68%|██████▊ | 2711/4000 [3:10:44<1:32:10, 4.29s/it]
68%|██████▊ | 2712/4000 [3:10:49<1:31:05, 4.24s/it]
68%|██████▊ | 2713/4000 [3:10:53<1:30:17, 4.21s/it]
68%|██████▊ | 2714/4000 [3:10:57<1:29:50, 4.19s/it]
68%|██████▊ | 2715/4000 [3:11:01<1:31:23, 4.27s/it]
68%|██████▊ | 2716/4000 [3:11:05<1:30:28, 4.23s/it]
68%|██████▊ | 2717/4000 [3:11:10<1:29:47, 4.20s/it]
68%|██████▊ | 2718/4000 [3:11:14<1:30:01, 4.21s/it]
68%|██████▊ | 2719/4000 [3:11:18<1:30:28, 4.24s/it]
68%|██████▊ | 2720/4000 [3:11:22<1:29:42, 4.20s/it]
{'loss': '0.4102', 'grad_norm': '0.1308', 'learning_rate': '0.0001941', 'epoch': '0.6591'}
68%|██████▊ | 2720/4000 [3:11:22<1:29:42, 4.20s/it]
68%|██████▊ | 2721/4000 [3:11:26<1:29:06, 4.18s/it]
68%|██████▊ | 2722/4000 [3:11:31<1:32:08, 4.33s/it]
68%|██████▊ | 2723/4000 [3:11:35<1:32:38, 4.35s/it]
68%|██████▊ | 2724/4000 [3:11:40<1:31:07, 4.28s/it]
68%|██████▊ | 2725/4000 [3:11:44<1:30:04, 4.24s/it]
68%|██████▊ | 2726/4000 [3:11:48<1:29:15, 4.20s/it]
68%|██████▊ | 2727/4000 [3:11:52<1:29:54, 4.24s/it]
68%|██████▊ | 2728/4000 [3:11:56<1:30:29, 4.27s/it]
68%|██████▊ | 2729/4000 [3:12:01<1:31:53, 4.34s/it]
68%|██████▊ | 2730/4000 [3:12:05<1:30:30, 4.28s/it]
{'loss': '0.4362', 'grad_norm': '0.1407', 'learning_rate': '0.0001926', 'epoch': '0.6615'}
68%|██████▊ | 2730/4000 [3:12:05<1:30:30, 4.28s/it]
68%|██████▊ | 2731/4000 [3:12:09<1:29:29, 4.23s/it]
68%|██████▊ | 2732/4000 [3:12:13<1:28:50, 4.20s/it]
68%|██████▊ | 2733/4000 [3:12:17<1:28:20, 4.18s/it]
68%|██████▊ | 2734/4000 [3:12:22<1:28:02, 4.17s/it]
68%|██████▊ | 2735/4000 [3:12:26<1:29:52, 4.26s/it]
68%|██████▊ | 2736/4000 [3:12:31<1:31:09, 4.33s/it]
68%|██████▊ | 2737/4000 [3:12:35<1:29:48, 4.27s/it]
68%|██████▊ | 2738/4000 [3:12:39<1:28:56, 4.23s/it]
68%|██████▊ | 2739/4000 [3:12:43<1:29:35, 4.26s/it]
68%|██████▊ | 2740/4000 [3:12:48<1:30:01, 4.29s/it]
{'loss': '0.4305', 'grad_norm': '0.1318', 'learning_rate': '0.0001911', 'epoch': '0.6639'}
68%|██████▊ | 2740/4000 [3:12:48<1:30:01, 4.29s/it]
69%|██████▊ | 2741/4000 [3:12:52<1:28:59, 4.24s/it]
69%|██████▊ | 2742/4000 [3:12:56<1:30:24, 4.31s/it]
69%|██████▊ | 2743/4000 [3:13:00<1:29:12, 4.26s/it]
69%|██████▊ | 2744/4000 [3:13:05<1:29:44, 4.29s/it]
69%|██████▊ | 2745/4000 [3:13:09<1:29:05, 4.26s/it]
69%|██████▊ | 2746/4000 [3:13:13<1:28:14, 4.22s/it]
69%|██████▊ | 2747/4000 [3:13:17<1:27:31, 4.19s/it]
69%|██████▊ | 2748/4000 [3:13:21<1:27:05, 4.17s/it]
69%|██████▊ | 2749/4000 [3:13:26<1:28:44, 4.26s/it]
69%|██████▉ | 2750/4000 [3:13:30<1:27:51, 4.22s/it]
{'loss': '0.4596', 'grad_norm': '0.127', 'learning_rate': '0.0001895', 'epoch': '0.6663'}
69%|██████▉ | 2750/4000 [3:13:30<1:27:51, 4.22s/it]
69%|██████▉ | 2751/4000 [3:13:34<1:27:17, 4.19s/it]
69%|██████▉ | 2752/4000 [3:13:38<1:27:21, 4.20s/it]
69%|██████▉ | 2753/4000 [3:13:42<1:27:40, 4.22s/it]
69%|██████▉ | 2754/4000 [3:13:47<1:27:11, 4.20s/it]
69%|██████▉ | 2755/4000 [3:13:51<1:26:38, 4.18s/it]
69%|██████▉ | 2756/4000 [3:13:55<1:29:07, 4.30s/it]
69%|██████▉ | 2757/4000 [3:13:59<1:28:37, 4.28s/it]
69%|██████▉ | 2758/4000 [3:14:04<1:27:52, 4.25s/it]
69%|██████▉ | 2759/4000 [3:14:08<1:27:07, 4.21s/it]
69%|██████▉ | 2760/4000 [3:14:12<1:26:30, 4.19s/it]
{'loss': '0.401', 'grad_norm': '0.1253', 'learning_rate': '0.000188', 'epoch': '0.6687'}
69%|██████▉ | 2760/4000 [3:14:12<1:26:30, 4.19s/it]
69%|██████▉ | 2761/4000 [3:14:17<1:29:04, 4.31s/it]
69%|██████▉ | 2762/4000 [3:14:21<1:29:18, 4.33s/it]
69%|██████▉ | 2763/4000 [3:14:25<1:29:00, 4.32s/it]
69%|██████▉ | 2764/4000 [3:14:29<1:27:45, 4.26s/it]
69%|██████▉ | 2765/4000 [3:14:33<1:26:57, 4.22s/it]
69%|██████▉ | 2766/4000 [3:14:38<1:26:16, 4.19s/it]
69%|██████▉ | 2767/4000 [3:14:42<1:25:48, 4.18s/it]
69%|██████▉ | 2768/4000 [3:14:46<1:25:26, 4.16s/it]
69%|██████▉ | 2769/4000 [3:14:50<1:27:31, 4.27s/it]
69%|██████▉ | 2770/4000 [3:14:55<1:27:23, 4.26s/it]
{'loss': '0.406', 'grad_norm': '0.1118', 'learning_rate': '0.0001865', 'epoch': '0.6712'}
69%|██████▉ | 2770/4000 [3:14:55<1:27:23, 4.26s/it]
69%|██████▉ | 2771/4000 [3:14:59<1:27:25, 4.27s/it]
69%|██████▉ | 2772/4000 [3:15:03<1:26:29, 4.23s/it]
69%|██████▉ | 2773/4000 [3:15:07<1:26:06, 4.21s/it]
69%|██████▉ | 2774/4000 [3:15:12<1:27:04, 4.26s/it]
69%|██████▉ | 2775/4000 [3:15:16<1:26:44, 4.25s/it]
69%|██████▉ | 2776/4000 [3:15:20<1:28:14, 4.33s/it]
69%|██████▉ | 2777/4000 [3:15:24<1:27:12, 4.28s/it]
69%|██████▉ | 2778/4000 [3:15:29<1:27:15, 4.28s/it]
69%|██████▉ | 2779/4000 [3:15:33<1:26:32, 4.25s/it]
70%|██████▉ | 2780/4000 [3:15:37<1:25:41, 4.21s/it]
{'loss': '0.4112', 'grad_norm': '0.1211', 'learning_rate': '0.000185', 'epoch': '0.6736'}
70%|██████▉ | 2780/4000 [3:15:37<1:25:41, 4.21s/it]
70%|██████▉ | 2781/4000 [3:15:41<1:25:09, 4.19s/it]
70%|██████▉ | 2782/4000 [3:15:45<1:24:44, 4.17s/it]
70%|██████▉ | 2783/4000 [3:15:50<1:26:26, 4.26s/it]
70%|██████▉ | 2784/4000 [3:15:54<1:25:46, 4.23s/it]
70%|██████▉ | 2785/4000 [3:15:58<1:25:08, 4.20s/it]
70%|██████▉ | 2786/4000 [3:16:02<1:24:42, 4.19s/it]
70%|██████▉ | 2787/4000 [3:16:07<1:25:17, 4.22s/it]
70%|██████▉ | 2788/4000 [3:16:11<1:25:50, 4.25s/it]
70%|██████▉ | 2789/4000 [3:16:15<1:26:23, 4.28s/it]
70%|██████▉ | 2790/4000 [3:16:19<1:26:21, 4.28s/it]
{'loss': '0.363', 'grad_norm': '0.115', 'learning_rate': '0.0001835', 'epoch': '0.676'}
70%|██████▉ | 2790/4000 [3:16:19<1:26:21, 4.28s/it]
70%|██████▉ | 2791/4000 [3:16:24<1:26:29, 4.29s/it]
70%|██████▉ | 2792/4000 [3:16:28<1:26:43, 4.31s/it]
70%|██████▉ | 2793/4000 [3:16:32<1:25:41, 4.26s/it]
70%|██████▉ | 2794/4000 [3:16:37<1:25:23, 4.25s/it]
70%|██████▉ | 2795/4000 [3:16:41<1:25:42, 4.27s/it]
70%|██████▉ | 2796/4000 [3:16:45<1:27:17, 4.35s/it]
70%|██████▉ | 2797/4000 [3:16:50<1:25:54, 4.28s/it]
70%|██████▉ | 2798/4000 [3:16:54<1:24:52, 4.24s/it]
70%|██████▉ | 2799/4000 [3:16:58<1:24:12, 4.21s/it]
70%|███████ | 2800/4000 [3:17:02<1:23:43, 4.19s/it]
{'loss': '0.3853', 'grad_norm': '0.1256', 'learning_rate': '0.000182', 'epoch': '0.6784'}
70%|███████ | 2800/4000 [3:17:02<1:23:43, 4.19s/it]
70%|███████ | 2801/4000 [3:17:06<1:23:17, 4.17s/it]
70%|███████ | 2802/4000 [3:17:10<1:23:02, 4.16s/it]
70%|███████ | 2803/4000 [3:17:15<1:25:04, 4.26s/it]
70%|███████ | 2804/4000 [3:17:19<1:24:36, 4.24s/it]
70%|███████ | 2805/4000 [3:17:23<1:25:24, 4.29s/it]
70%|███████ | 2806/4000 [3:17:27<1:24:29, 4.25s/it]
70%|███████ | 2807/4000 [3:17:32<1:23:47, 4.21s/it]
70%|███████ | 2808/4000 [3:17:36<1:23:50, 4.22s/it]
70%|███████ | 2809/4000 [3:17:40<1:24:37, 4.26s/it]
70%|███████ | 2810/4000 [3:17:45<1:26:05, 4.34s/it]
{'loss': '0.4152', 'grad_norm': '0.1047', 'learning_rate': '0.0001805', 'epoch': '0.6809'}
70%|███████ | 2810/4000 [3:17:45<1:26:05, 4.34s/it]
70%|███████ | 2811/4000 [3:17:49<1:25:22, 4.31s/it]
70%|███████ | 2812/4000 [3:17:53<1:24:51, 4.29s/it]
70%|███████ | 2813/4000 [3:17:57<1:23:51, 4.24s/it]
70%|███████ | 2814/4000 [3:18:01<1:23:12, 4.21s/it]
70%|███████ | 2815/4000 [3:18:06<1:22:40, 4.19s/it]
70%|███████ | 2816/4000 [3:18:10<1:24:18, 4.27s/it]
70%|███████ | 2817/4000 [3:18:14<1:23:25, 4.23s/it]
70%|███████ | 2818/4000 [3:18:18<1:22:42, 4.20s/it]
70%|███████ | 2819/4000 [3:18:22<1:22:14, 4.18s/it]
70%|███████ | 2820/4000 [3:18:27<1:21:51, 4.16s/it]
{'loss': '0.3839', 'grad_norm': '0.1445', 'learning_rate': '0.0001789', 'epoch': '0.6833'}
70%|███████ | 2820/4000 [3:18:27<1:21:51, 4.16s/it]
71%|███████ | 2821/4000 [3:18:31<1:22:20, 4.19s/it]
71%|███████ | 2822/4000 [3:18:35<1:23:39, 4.26s/it]
71%|███████ | 2823/4000 [3:18:40<1:25:10, 4.34s/it]
71%|███████ | 2824/4000 [3:18:44<1:23:55, 4.28s/it]
71%|███████ | 2825/4000 [3:18:48<1:23:18, 4.25s/it]
71%|███████ | 2826/4000 [3:18:52<1:23:22, 4.26s/it]
71%|███████ | 2827/4000 [3:18:56<1:22:32, 4.22s/it]
71%|███████ | 2828/4000 [3:19:01<1:22:41, 4.23s/it]
71%|███████ | 2829/4000 [3:19:05<1:22:33, 4.23s/it]
71%|███████ | 2830/4000 [3:19:09<1:24:05, 4.31s/it]
{'loss': '0.4183', 'grad_norm': '0.131', 'learning_rate': '0.0001774', 'epoch': '0.6857'}
71%|███████ | 2830/4000 [3:19:09<1:24:05, 4.31s/it]
71%|███████ | 2831/4000 [3:19:14<1:22:57, 4.26s/it]
71%|███████ | 2832/4000 [3:19:18<1:22:08, 4.22s/it]
71%|███████ | 2833/4000 [3:19:22<1:21:32, 4.19s/it]
71%|███████ | 2834/4000 [3:19:26<1:21:05, 4.17s/it]
71%|███████ | 2835/4000 [3:19:30<1:20:46, 4.16s/it]
71%|███████ | 2836/4000 [3:19:34<1:21:34, 4.21s/it]
71%|███████ | 2837/4000 [3:19:39<1:22:09, 4.24s/it]
71%|███████ | 2838/4000 [3:19:43<1:22:05, 4.24s/it]
71%|███████ | 2839/4000 [3:19:47<1:22:33, 4.27s/it]
71%|███████ | 2840/4000 [3:19:51<1:21:46, 4.23s/it]
{'loss': '0.3865', 'grad_norm': '0.1168', 'learning_rate': '0.0001759', 'epoch': '0.6881'}
71%|███████ | 2840/4000 [3:19:51<1:21:46, 4.23s/it]
71%|███████ | 2841/4000 [3:19:56<1:21:10, 4.20s/it]
71%|███████ | 2842/4000 [3:20:00<1:20:37, 4.18s/it]
71%|███████ | 2843/4000 [3:20:05<1:24:25, 4.38s/it]
71%|███████ | 2844/4000 [3:20:09<1:23:57, 4.36s/it]
71%|███████ | 2845/4000 [3:20:13<1:23:17, 4.33s/it]
71%|███████ | 2846/4000 [3:20:17<1:22:22, 4.28s/it]
71%|███████ | 2847/4000 [3:20:21<1:21:25, 4.24s/it]
71%|███████ | 2848/4000 [3:20:26<1:20:48, 4.21s/it]
71%|███████ | 2849/4000 [3:20:30<1:20:20, 4.19s/it]
71%|███████▏ | 2850/4000 [3:20:34<1:21:45, 4.27s/it]
{'loss': '0.3943', 'grad_norm': '0.1227', 'learning_rate': '0.0001744', 'epoch': '0.6906'}
71%|███████▏ | 2850/4000 [3:20:34<1:21:45, 4.27s/it]
71%|███████▏ | 2851/4000 [3:20:38<1:21:03, 4.23s/it]
71%|███████▏ | 2852/4000 [3:20:42<1:20:25, 4.20s/it]
71%|███████▏ | 2853/4000 [3:20:47<1:20:00, 4.18s/it]
71%|███████▏ | 2854/4000 [3:20:51<1:19:40, 4.17s/it]
71%|███████▏ | 2855/4000 [3:20:55<1:19:40, 4.18s/it]
71%|███████▏ | 2856/4000 [3:20:59<1:19:56, 4.19s/it]
71%|███████▏ | 2857/4000 [3:21:04<1:22:14, 4.32s/it]
71%|███████▏ | 2858/4000 [3:21:08<1:21:07, 4.26s/it]
71%|███████▏ | 2859/4000 [3:21:12<1:20:19, 4.22s/it]
72%|███████▏ | 2860/4000 [3:21:16<1:20:44, 4.25s/it]
{'loss': '0.3776', 'grad_norm': '0.1255', 'learning_rate': '0.0001729', 'epoch': '0.693'}
72%|███████▏ | 2860/4000 [3:21:16<1:20:44, 4.25s/it]
72%|███████▏ | 2861/4000 [3:21:21<1:22:30, 4.35s/it]
72%|███████▏ | 2862/4000 [3:21:25<1:22:37, 4.36s/it]
72%|███████▏ | 2863/4000 [3:21:30<1:22:30, 4.35s/it]
72%|███████▏ | 2864/4000 [3:21:34<1:22:10, 4.34s/it]
72%|███████▏ | 2865/4000 [3:21:38<1:20:57, 4.28s/it]
72%|███████▏ | 2866/4000 [3:21:42<1:20:00, 4.23s/it]
72%|███████▏ | 2867/4000 [3:21:46<1:19:27, 4.21s/it]
72%|███████▏ | 2868/4000 [3:21:51<1:18:57, 4.19s/it]
72%|███████▏ | 2869/4000 [3:21:55<1:18:37, 4.17s/it]
72%|███████▏ | 2870/4000 [3:21:59<1:20:30, 4.27s/it]
{'loss': '0.418', 'grad_norm': '0.1241', 'learning_rate': '0.0001714', 'epoch': '0.6954'}
72%|███████▏ | 2870/4000 [3:21:59<1:20:30, 4.27s/it]
72%|███████▏ | 2871/4000 [3:22:03<1:19:39, 4.23s/it]
72%|███████▏ | 2872/4000 [3:22:07<1:18:57, 4.20s/it]
72%|███████▏ | 2873/4000 [3:22:12<1:19:00, 4.21s/it]
72%|███████▏ | 2874/4000 [3:22:16<1:19:12, 4.22s/it]
72%|███████▏ | 2875/4000 [3:22:20<1:18:34, 4.19s/it]
72%|███████▏ | 2876/4000 [3:22:24<1:18:15, 4.18s/it]
72%|███████▏ | 2877/4000 [3:22:29<1:20:44, 4.31s/it]
72%|███████▏ | 2878/4000 [3:22:33<1:21:07, 4.34s/it]
72%|███████▏ | 2879/4000 [3:22:38<1:21:34, 4.37s/it]
72%|███████▏ | 2880/4000 [3:22:42<1:20:30, 4.31s/it]
{'loss': '0.3798', 'grad_norm': '0.1186', 'learning_rate': '0.0001698', 'epoch': '0.6978'}
72%|███████▏ | 2880/4000 [3:22:42<1:20:30, 4.31s/it]
72%|███████▏ | 2881/4000 [3:22:46<1:19:23, 4.26s/it]
72%|███████▏ | 2882/4000 [3:22:50<1:18:38, 4.22s/it]
72%|███████▏ | 2883/4000 [3:22:54<1:18:05, 4.19s/it]
72%|███████▏ | 2884/4000 [3:22:59<1:19:31, 4.28s/it]
72%|███████▏ | 2885/4000 [3:23:03<1:18:42, 4.24s/it]
72%|███████▏ | 2886/4000 [3:23:07<1:18:03, 4.20s/it]
72%|███████▏ | 2887/4000 [3:23:11<1:17:33, 4.18s/it]
72%|███████▏ | 2888/4000 [3:23:15<1:17:14, 4.17s/it]
72%|███████▏ | 2889/4000 [3:23:19<1:17:01, 4.16s/it]
72%|███████▏ | 2890/4000 [3:23:24<1:18:42, 4.25s/it]
{'loss': '0.4351', 'grad_norm': '0.1104', 'learning_rate': '0.0001683', 'epoch': '0.7002'}
72%|███████▏ | 2890/4000 [3:23:24<1:18:42, 4.25s/it]
72%|███████▏ | 2891/4000 [3:23:28<1:20:01, 4.33s/it]
72%|███████▏ | 2892/4000 [3:23:32<1:18:57, 4.28s/it]
72%|███████▏ | 2893/4000 [3:23:37<1:18:41, 4.27s/it]
72%|███████▏ | 2894/4000 [3:23:41<1:18:48, 4.28s/it]
72%|███████▏ | 2895/4000 [3:23:45<1:18:52, 4.28s/it]
72%|███████▏ | 2896/4000 [3:23:50<1:19:10, 4.30s/it]
72%|███████▏ | 2897/4000 [3:23:54<1:21:03, 4.41s/it]
72%|███████▏ | 2898/4000 [3:23:58<1:19:26, 4.33s/it]
72%|███████▏ | 2899/4000 [3:24:03<1:18:18, 4.27s/it]
72%|███████▎ | 2900/4000 [3:24:07<1:17:27, 4.22s/it]
{'loss': '0.4209', 'grad_norm': '0.1284', 'learning_rate': '0.0001668', 'epoch': '0.7027'}
72%|███████▎ | 2900/4000 [3:24:07<1:17:27, 4.22s/it]
73%|███████▎ | 2901/4000 [3:24:11<1:16:52, 4.20s/it]
73%|███████▎ | 2902/4000 [3:24:15<1:16:23, 4.17s/it]
73%|███████▎ | 2903/4000 [3:24:19<1:16:08, 4.16s/it]
73%|███████▎ | 2904/4000 [3:24:24<1:17:50, 4.26s/it]
73%|███████▎ | 2905/4000 [3:24:28<1:17:01, 4.22s/it]
73%|███████▎ | 2906/4000 [3:24:32<1:16:26, 4.19s/it]
73%|███████▎ | 2907/4000 [3:24:36<1:16:02, 4.17s/it]
73%|███████▎ | 2908/4000 [3:24:40<1:16:42, 4.21s/it]
73%|███████▎ | 2909/4000 [3:24:44<1:16:28, 4.21s/it]
73%|███████▎ | 2910/4000 [3:24:49<1:16:57, 4.24s/it]
{'loss': '0.4031', 'grad_norm': '0.1219', 'learning_rate': '0.0001653', 'epoch': '0.7051'}
73%|███████▎ | 2910/4000 [3:24:49<1:16:57, 4.24s/it]
73%|███████▎ | 2911/4000 [3:24:54<1:19:31, 4.38s/it]
73%|███████▎ | 2912/4000 [3:24:58<1:18:56, 4.35s/it]
73%|███████▎ | 2913/4000 [3:25:02<1:18:49, 4.35s/it]
73%|███████▎ | 2914/4000 [3:25:06<1:18:37, 4.34s/it]
73%|███████▎ | 2915/4000 [3:25:11<1:17:21, 4.28s/it]
73%|███████▎ | 2916/4000 [3:25:15<1:16:30, 4.23s/it]
73%|███████▎ | 2917/4000 [3:25:19<1:16:49, 4.26s/it]
73%|███████▎ | 2918/4000 [3:25:23<1:16:56, 4.27s/it]
73%|███████▎ | 2919/4000 [3:25:27<1:16:04, 4.22s/it]
73%|███████▎ | 2920/4000 [3:25:32<1:15:31, 4.20s/it]
{'loss': '0.402', 'grad_norm': '0.1316', 'learning_rate': '0.0001638', 'epoch': '0.7075'}
73%|███████▎ | 2920/4000 [3:25:32<1:15:31, 4.20s/it]
73%|███████▎ | 2921/4000 [3:25:36<1:15:03, 4.17s/it]
73%|███████▎ | 2922/4000 [3:25:40<1:14:46, 4.16s/it]
73%|███████▎ | 2923/4000 [3:25:44<1:14:35, 4.16s/it]
73%|███████▎ | 2924/4000 [3:25:48<1:16:24, 4.26s/it]
73%|███████▎ | 2925/4000 [3:25:53<1:15:58, 4.24s/it]
73%|███████▎ | 2926/4000 [3:25:57<1:16:00, 4.25s/it]
73%|███████▎ | 2927/4000 [3:26:01<1:15:39, 4.23s/it]
73%|███████▎ | 2928/4000 [3:26:05<1:15:11, 4.21s/it]
73%|███████▎ | 2929/4000 [3:26:10<1:15:53, 4.25s/it]
73%|███████▎ | 2930/4000 [3:26:14<1:16:04, 4.27s/it]
{'loss': '0.3727', 'grad_norm': '0.1204', 'learning_rate': '0.0001623', 'epoch': '0.7099'}
73%|███████▎ | 2930/4000 [3:26:14<1:16:04, 4.27s/it]
73%|███████▎ | 2931/4000 [3:26:19<1:18:05, 4.38s/it]
73%|███████▎ | 2932/4000 [3:26:23<1:16:41, 4.31s/it]
73%|███████▎ | 2933/4000 [3:26:27<1:15:34, 4.25s/it]
73%|███████▎ | 2934/4000 [3:26:31<1:14:57, 4.22s/it]
73%|███████▎ | 2935/4000 [3:26:35<1:14:23, 4.19s/it]
73%|███████▎ | 2936/4000 [3:26:39<1:13:58, 4.17s/it]
73%|███████▎ | 2937/4000 [3:26:43<1:13:35, 4.15s/it]
73%|███████▎ | 2938/4000 [3:26:48<1:15:11, 4.25s/it]
73%|███████▎ | 2939/4000 [3:26:52<1:14:27, 4.21s/it]
74%|███████▎ | 2940/4000 [3:26:56<1:13:53, 4.18s/it]
{'loss': '0.4068', 'grad_norm': '0.1519', 'learning_rate': '0.0001608', 'epoch': '0.7124'}
74%|███████▎ | 2940/4000 [3:26:56<1:13:53, 4.18s/it]
74%|███████▎ | 2941/4000 [3:27:00<1:13:34, 4.17s/it]
74%|███████▎ | 2942/4000 [3:27:04<1:13:32, 4.17s/it]
74%|███████▎ | 2943/4000 [3:27:09<1:13:56, 4.20s/it]
74%|███████▎ | 2944/4000 [3:27:13<1:16:35, 4.35s/it]
74%|███████▎ | 2945/4000 [3:27:18<1:15:38, 4.30s/it]
74%|███████▎ | 2946/4000 [3:27:22<1:15:26, 4.29s/it]
74%|███████▎ | 2947/4000 [3:27:26<1:15:39, 4.31s/it]
74%|███████▎ | 2948/4000 [3:27:30<1:15:45, 4.32s/it]
74%|███████▎ | 2949/4000 [3:27:35<1:14:55, 4.28s/it]
74%|███████▍ | 2950/4000 [3:27:39<1:14:04, 4.23s/it]
{'loss': '0.369', 'grad_norm': '0.1085', 'learning_rate': '0.0001592', 'epoch': '0.7148'}
74%|███████▍ | 2950/4000 [3:27:39<1:14:04, 4.23s/it]
74%|███████▍ | 2951/4000 [3:27:43<1:15:03, 4.29s/it]
74%|███████▍ | 2952/4000 [3:27:47<1:14:11, 4.25s/it]
74%|███████▍ | 2953/4000 [3:27:52<1:13:38, 4.22s/it]
74%|███████▍ | 2954/4000 [3:27:56<1:13:09, 4.20s/it]
74%|███████▍ | 2955/4000 [3:28:00<1:12:48, 4.18s/it]
74%|███████▍ | 2956/4000 [3:28:04<1:12:32, 4.17s/it]
74%|███████▍ | 2957/4000 [3:28:08<1:12:17, 4.16s/it]
74%|███████▍ | 2958/4000 [3:28:13<1:13:58, 4.26s/it]
74%|███████▍ | 2959/4000 [3:28:17<1:13:19, 4.23s/it]
74%|███████▍ | 2960/4000 [3:28:21<1:13:16, 4.23s/it]
{'loss': '0.3512', 'grad_norm': '0.1349', 'learning_rate': '0.0001577', 'epoch': '0.7172'}
74%|███████▍ | 2960/4000 [3:28:21<1:13:16, 4.23s/it]
74%|███████▍ | 2961/4000 [3:28:25<1:13:07, 4.22s/it]
74%|███████▍ | 2962/4000 [3:28:29<1:13:32, 4.25s/it]
74%|███████▍ | 2963/4000 [3:28:34<1:13:18, 4.24s/it]
74%|███████▍ | 2964/4000 [3:28:38<1:14:00, 4.29s/it]
74%|███████▍ | 2965/4000 [3:28:43<1:15:32, 4.38s/it]
74%|███████▍ | 2966/4000 [3:28:47<1:14:14, 4.31s/it]
74%|███████▍ | 2967/4000 [3:28:51<1:13:11, 4.25s/it]
74%|███████▍ | 2968/4000 [3:28:55<1:12:30, 4.22s/it]
74%|███████▍ | 2969/4000 [3:28:59<1:12:04, 4.19s/it]
74%|███████▍ | 2970/4000 [3:29:03<1:11:37, 4.17s/it]
{'loss': '0.3859', 'grad_norm': '0.1437', 'learning_rate': '0.0001562', 'epoch': '0.7196'}
74%|███████▍ | 2970/4000 [3:29:03<1:11:37, 4.17s/it]
74%|███████▍ | 2971/4000 [3:29:08<1:13:21, 4.28s/it]
74%|███████▍ | 2972/4000 [3:29:12<1:12:30, 4.23s/it]
74%|███████▍ | 2973/4000 [3:29:16<1:11:51, 4.20s/it]
74%|███████▍ | 2974/4000 [3:29:20<1:11:29, 4.18s/it]
74%|███████▍ | 2975/4000 [3:29:24<1:11:12, 4.17s/it]
74%|███████▍ | 2976/4000 [3:29:29<1:10:53, 4.15s/it]
74%|███████▍ | 2977/4000 [3:29:33<1:11:15, 4.18s/it]
74%|███████▍ | 2978/4000 [3:29:37<1:13:47, 4.33s/it]
74%|███████▍ | 2979/4000 [3:29:42<1:13:40, 4.33s/it]
74%|███████▍ | 2980/4000 [3:29:46<1:12:47, 4.28s/it]
{'loss': '0.4049', 'grad_norm': '0.1137', 'learning_rate': '0.0001547', 'epoch': '0.7221'}
74%|███████▍ | 2980/4000 [3:29:46<1:12:47, 4.28s/it]
75%|███████▍ | 2981/4000 [3:29:50<1:12:28, 4.27s/it]
75%|███████▍ | 2982/4000 [3:29:54<1:12:05, 4.25s/it]
75%|███████▍ | 2983/4000 [3:29:59<1:11:44, 4.23s/it]
75%|███████▍ | 2984/4000 [3:30:03<1:11:14, 4.21s/it]
75%|███████▍ | 2985/4000 [3:30:07<1:12:22, 4.28s/it]
75%|███████▍ | 2986/4000 [3:30:11<1:11:29, 4.23s/it]
75%|███████▍ | 2987/4000 [3:30:15<1:10:55, 4.20s/it]
75%|███████▍ | 2988/4000 [3:30:20<1:10:27, 4.18s/it]
75%|███████▍ | 2989/4000 [3:30:24<1:10:06, 4.16s/it]
75%|███████▍ | 2990/4000 [3:30:28<1:09:52, 4.15s/it]
{'loss': '0.3667', 'grad_norm': '0.1142', 'learning_rate': '0.0001532', 'epoch': '0.7245'}
75%|███████▍ | 2990/4000 [3:30:28<1:09:52, 4.15s/it]
75%|███████▍ | 2991/4000 [3:30:32<1:09:40, 4.14s/it]
75%|███████▍ | 2992/4000 [3:30:36<1:11:29, 4.26s/it]
75%|███████▍ | 2993/4000 [3:30:41<1:10:45, 4.22s/it]
75%|███████▍ | 2994/4000 [3:30:45<1:10:39, 4.21s/it]
75%|███████▍ | 2995/4000 [3:30:49<1:10:36, 4.22s/it]
75%|███████▍ | 2996/4000 [3:30:53<1:10:37, 4.22s/it]
75%|███████▍ | 2997/4000 [3:30:58<1:11:02, 4.25s/it]
75%|███████▍ | 2998/4000 [3:31:02<1:12:42, 4.35s/it]
75%|███████▍ | 2999/4000 [3:31:06<1:12:14, 4.33s/it]
75%|███████▌ | 3000/4000 [3:31:11<1:11:37, 4.30s/it]
{'loss': '0.3712', 'grad_norm': '0.1242', 'learning_rate': '0.0001517', 'epoch': '0.7269'}
75%|███████▌ | 3000/4000 [3:31:11<1:11:37, 4.30s/it]
75%|███████▌ | 3001/4000 [3:31:15<1:10:43, 4.25s/it]
75%|███████▌ | 3002/4000 [3:31:19<1:10:03, 4.21s/it]
75%|███████▌ | 3003/4000 [3:31:23<1:09:34, 4.19s/it]
75%|███████▌ | 3004/4000 [3:31:27<1:09:19, 4.18s/it]
75%|███████▌ | 3005/4000 [3:31:32<1:10:54, 4.28s/it]
75%|███████▌ | 3006/4000 [3:31:36<1:10:07, 4.23s/it]
75%|███████▌ | 3007/4000 [3:31:40<1:09:31, 4.20s/it]
75%|███████▌ | 3008/4000 [3:31:44<1:09:08, 4.18s/it]
75%|███████▌ | 3009/4000 [3:31:48<1:08:50, 4.17s/it]
75%|███████▌ | 3010/4000 [3:31:52<1:08:33, 4.16s/it]
{'loss': '0.3987', 'grad_norm': '0.1528', 'learning_rate': '0.0001502', 'epoch': '0.7293'}
75%|███████▌ | 3010/4000 [3:31:52<1:08:33, 4.16s/it]
75%|███████▌ | 3011/4000 [3:31:57<1:08:40, 4.17s/it]
75%|███████▌ | 3012/4000 [3:32:01<1:10:58, 4.31s/it]
75%|███████▌ | 3013/4000 [3:32:05<1:10:24, 4.28s/it]
75%|███████▌ | 3014/4000 [3:32:10<1:11:11, 4.33s/it]
75%|███████▌ | 3015/4000 [3:32:14<1:10:38, 4.30s/it]
75%|███████▌ | 3016/4000 [3:32:18<1:10:55, 4.33s/it]
75%|███████▌ | 3017/4000 [3:32:23<1:09:51, 4.26s/it]
75%|███████▌ | 3018/4000 [3:32:27<1:09:47, 4.26s/it]
75%|███████▌ | 3019/4000 [3:32:31<1:09:44, 4.27s/it]
76%|███████▌ | 3020/4000 [3:32:35<1:08:52, 4.22s/it]
{'loss': '0.3992', 'grad_norm': '0.1216', 'learning_rate': '0.0001486', 'epoch': '0.7317'}
76%|███████▌ | 3020/4000 [3:32:35<1:08:52, 4.22s/it]
76%|███████▌ | 3021/4000 [3:32:39<1:08:16, 4.18s/it]
76%|███████▌ | 3022/4000 [3:32:43<1:07:51, 4.16s/it]
76%|███████▌ | 3023/4000 [3:32:48<1:07:29, 4.14s/it]
76%|███████▌ | 3024/4000 [3:32:52<1:07:19, 4.14s/it]
76%|███████▌ | 3025/4000 [3:32:56<1:09:42, 4.29s/it]
76%|███████▌ | 3026/4000 [3:33:00<1:08:53, 4.24s/it]
76%|███████▌ | 3027/4000 [3:33:05<1:08:12, 4.21s/it]
76%|███████▌ | 3028/4000 [3:33:09<1:07:48, 4.19s/it]
76%|███████▌ | 3029/4000 [3:33:13<1:07:24, 4.17s/it]
76%|███████▌ | 3030/4000 [3:33:17<1:07:10, 4.16s/it]
{'loss': '0.427', 'grad_norm': '0.1293', 'learning_rate': '0.0001471', 'epoch': '0.7342'}
76%|███████▌ | 3030/4000 [3:33:17<1:07:10, 4.16s/it]
76%|███████▌ | 3031/4000 [3:33:21<1:06:57, 4.15s/it]
76%|███████▌ | 3032/4000 [3:33:26<1:08:44, 4.26s/it]
76%|███████▌ | 3033/4000 [3:33:30<1:08:03, 4.22s/it]
76%|███████▌ | 3034/4000 [3:33:34<1:07:31, 4.19s/it]
76%|███████▌ | 3035/4000 [3:33:38<1:07:09, 4.18s/it]
76%|███████▌ | 3036/4000 [3:33:42<1:06:50, 4.16s/it]
76%|███████▌ | 3037/4000 [3:33:46<1:06:30, 4.14s/it]
76%|███████▌ | 3038/4000 [3:33:50<1:06:18, 4.14s/it]
76%|███████▌ | 3039/4000 [3:33:55<1:07:45, 4.23s/it]
76%|███████▌ | 3040/4000 [3:33:59<1:07:07, 4.19s/it]
{'loss': '0.3702', 'grad_norm': '0.09895', 'learning_rate': '0.0001456', 'epoch': '0.7366'}
76%|███████▌ | 3040/4000 [3:33:59<1:07:07, 4.19s/it]
76%|███████▌ | 3041/4000 [3:34:03<1:06:39, 4.17s/it]
76%|███████▌ | 3042/4000 [3:34:07<1:06:18, 4.15s/it]
76%|███████▌ | 3043/4000 [3:34:11<1:06:04, 4.14s/it]
76%|███████▌ | 3044/4000 [3:34:15<1:05:51, 4.13s/it]
76%|███████▌ | 3045/4000 [3:34:20<1:06:32, 4.18s/it]
76%|███████▌ | 3046/4000 [3:34:24<1:07:14, 4.23s/it]
76%|███████▌ | 3047/4000 [3:34:28<1:06:52, 4.21s/it]
76%|███████▌ | 3048/4000 [3:34:32<1:06:18, 4.18s/it]
76%|███████▌ | 3049/4000 [3:34:36<1:05:55, 4.16s/it]
76%|███████▋ | 3050/4000 [3:34:40<1:05:41, 4.15s/it]
{'loss': '0.3753', 'grad_norm': '0.1163', 'learning_rate': '0.0001441', 'epoch': '0.739'}
76%|███████▋ | 3050/4000 [3:34:41<1:05:41, 4.15s/it]
76%|███████▋ | 3051/4000 [3:34:45<1:05:25, 4.14s/it]
76%|███████▋ | 3052/4000 [3:34:49<1:06:48, 4.23s/it]
76%|███████▋ | 3053/4000 [3:34:53<1:06:13, 4.20s/it]
76%|███████▋ | 3054/4000 [3:34:57<1:05:46, 4.17s/it]
76%|███████▋ | 3055/4000 [3:35:01<1:05:24, 4.15s/it]
76%|███████▋ | 3056/4000 [3:35:06<1:05:10, 4.14s/it]
76%|███████▋ | 3057/4000 [3:35:10<1:05:01, 4.14s/it]
76%|███████▋ | 3058/4000 [3:35:14<1:05:01, 4.14s/it]
76%|███████▋ | 3059/4000 [3:35:18<1:06:47, 4.26s/it]
76%|███████▋ | 3060/4000 [3:35:23<1:06:26, 4.24s/it]
{'loss': '0.3737', 'grad_norm': '0.1183', 'learning_rate': '0.0001426', 'epoch': '0.7414'}
76%|███████▋ | 3060/4000 [3:35:23<1:06:26, 4.24s/it]
77%|███████▋ | 3061/4000 [3:35:27<1:05:53, 4.21s/it]
77%|███████▋ | 3062/4000 [3:35:31<1:05:30, 4.19s/it]
77%|███████▋ | 3063/4000 [3:35:35<1:05:19, 4.18s/it]
77%|███████▋ | 3064/4000 [3:35:39<1:05:07, 4.17s/it]
77%|███████▋ | 3065/4000 [3:35:43<1:04:54, 4.17s/it]
77%|███████▋ | 3066/4000 [3:35:48<1:06:26, 4.27s/it]
77%|███████▋ | 3067/4000 [3:35:52<1:05:46, 4.23s/it]
77%|███████▋ | 3068/4000 [3:35:56<1:05:18, 4.20s/it]
77%|███████▋ | 3069/4000 [3:36:00<1:04:56, 4.18s/it]
77%|███████▋ | 3070/4000 [3:36:04<1:04:39, 4.17s/it]
{'loss': '0.4057', 'grad_norm': '0.1133', 'learning_rate': '0.0001411', 'epoch': '0.7439'}
77%|███████▋ | 3070/4000 [3:36:04<1:04:39, 4.17s/it]
77%|███████▋ | 3071/4000 [3:36:08<1:04:27, 4.16s/it]
77%|███████▋ | 3072/4000 [3:36:13<1:05:59, 4.27s/it]
77%|███████▋ | 3073/4000 [3:36:17<1:05:20, 4.23s/it]
77%|███████▋ | 3074/4000 [3:36:21<1:04:50, 4.20s/it]
77%|███████▋ | 3075/4000 [3:36:25<1:04:30, 4.18s/it]
77%|███████▋ | 3076/4000 [3:36:30<1:04:14, 4.17s/it]
77%|███████▋ | 3077/4000 [3:36:34<1:05:02, 4.23s/it]
77%|███████▋ | 3078/4000 [3:36:38<1:05:02, 4.23s/it]
77%|███████▋ | 3079/4000 [3:36:43<1:07:06, 4.37s/it]
77%|███████▋ | 3080/4000 [3:36:47<1:06:27, 4.33s/it]
{'loss': '0.4005', 'grad_norm': '0.117', 'learning_rate': '0.0001395', 'epoch': '0.7463'}
77%|███████▋ | 3080/4000 [3:36:47<1:06:27, 4.33s/it]
77%|███████▋ | 3081/4000 [3:36:51<1:05:59, 4.31s/it]
77%|███████▋ | 3082/4000 [3:36:56<1:05:43, 4.30s/it]
77%|███████▋ | 3083/4000 [3:37:00<1:05:19, 4.27s/it]
77%|███████▋ | 3084/4000 [3:37:04<1:04:39, 4.23s/it]
77%|███████▋ | 3085/4000 [3:37:08<1:04:04, 4.20s/it]
77%|███████▋ | 3086/4000 [3:37:13<1:05:21, 4.29s/it]
77%|███████▋ | 3087/4000 [3:37:17<1:04:34, 4.24s/it]
77%|███████▋ | 3088/4000 [3:37:21<1:04:02, 4.21s/it]
77%|███████▋ | 3089/4000 [3:37:25<1:03:34, 4.19s/it]
77%|███████▋ | 3090/4000 [3:37:29<1:03:16, 4.17s/it]
{'loss': '0.3755', 'grad_norm': '0.1228', 'learning_rate': '0.000138', 'epoch': '0.7487'}
77%|███████▋ | 3090/4000 [3:37:29<1:03:16, 4.17s/it]
77%|███████▋ | 3091/4000 [3:37:33<1:02:57, 4.16s/it]
77%|███████▋ | 3092/4000 [3:37:37<1:02:47, 4.15s/it]
77%|███████▋ | 3093/4000 [3:37:42<1:04:23, 4.26s/it]
77%|███████▋ | 3094/4000 [3:37:46<1:03:44, 4.22s/it]
77%|███████▋ | 3095/4000 [3:37:50<1:03:14, 4.19s/it]
77%|███████▋ | 3096/4000 [3:37:54<1:03:19, 4.20s/it]
77%|███████▋ | 3097/4000 [3:37:59<1:03:17, 4.21s/it]
77%|███████▋ | 3098/4000 [3:38:03<1:02:52, 4.18s/it]
77%|███████▋ | 3099/4000 [3:38:07<1:04:22, 4.29s/it]
78%|███████▊ | 3100/4000 [3:38:12<1:04:34, 4.31s/it]
{'loss': '0.3929', 'grad_norm': '0.1191', 'learning_rate': '0.0001365', 'epoch': '0.7511'}
78%|███████▊ | 3100/4000 [3:38:12<1:04:34, 4.31s/it]
78%|███████▊ | 3101/4000 [3:38:16<1:04:28, 4.30s/it]
78%|███████▊ | 3102/4000 [3:38:20<1:04:13, 4.29s/it]
78%|███████▊ | 3103/4000 [3:38:24<1:04:09, 4.29s/it]
78%|███████▊ | 3104/4000 [3:38:29<1:04:06, 4.29s/it]
78%|███████▊ | 3105/4000 [3:38:33<1:03:17, 4.24s/it]
78%|███████▊ | 3106/4000 [3:38:37<1:04:30, 4.33s/it]
78%|███████▊ | 3107/4000 [3:38:42<1:03:31, 4.27s/it]
78%|███████▊ | 3108/4000 [3:38:46<1:02:48, 4.23s/it]
78%|███████▊ | 3109/4000 [3:38:50<1:02:21, 4.20s/it]
78%|███████▊ | 3110/4000 [3:38:54<1:01:56, 4.18s/it]
{'loss': '0.3758', 'grad_norm': '0.1098', 'learning_rate': '0.000135', 'epoch': '0.7536'}
78%|███████▊ | 3110/4000 [3:38:54<1:01:56, 4.18s/it]
78%|███████▊ | 3111/4000 [3:38:58<1:01:42, 4.16s/it]
78%|███████▊ | 3112/4000 [3:39:02<1:01:52, 4.18s/it]
78%|███████▊ | 3113/4000 [3:39:07<1:03:23, 4.29s/it]
78%|███████▊ | 3114/4000 [3:39:11<1:02:34, 4.24s/it]
78%|███████▊ | 3115/4000 [3:39:15<1:01:59, 4.20s/it]
78%|███████▊ | 3116/4000 [3:39:19<1:01:36, 4.18s/it]
78%|███████▊ | 3117/4000 [3:39:23<1:01:16, 4.16s/it]
78%|███████▊ | 3118/4000 [3:39:27<1:01:04, 4.15s/it]
78%|███████▊ | 3119/4000 [3:39:32<1:01:40, 4.20s/it]
78%|███████▊ | 3120/4000 [3:39:36<1:02:16, 4.25s/it]
{'loss': '0.3856', 'grad_norm': '0.1115', 'learning_rate': '0.0001335', 'epoch': '0.756'}
78%|███████▊ | 3120/4000 [3:39:36<1:02:16, 4.25s/it]
78%|███████▊ | 3121/4000 [3:39:40<1:02:36, 4.27s/it]
78%|███████▊ | 3122/4000 [3:39:45<1:02:28, 4.27s/it]
78%|███████▊ | 3123/4000 [3:39:49<1:02:19, 4.26s/it]
78%|███████▊ | 3124/4000 [3:39:53<1:02:04, 4.25s/it]
78%|███████▊ | 3125/4000 [3:39:57<1:01:30, 4.22s/it]
78%|███████▊ | 3126/4000 [3:40:02<1:02:51, 4.31s/it]
78%|███████▊ | 3127/4000 [3:40:06<1:01:56, 4.26s/it]
78%|███████▊ | 3128/4000 [3:40:10<1:01:55, 4.26s/it]
78%|███████▊ | 3129/4000 [3:40:14<1:01:15, 4.22s/it]
78%|███████▊ | 3130/4000 [3:40:19<1:00:50, 4.20s/it]
{'loss': '0.4092', 'grad_norm': '0.1183', 'learning_rate': '0.000132', 'epoch': '0.7584'}
78%|███████▊ | 3130/4000 [3:40:19<1:00:50, 4.20s/it]
78%|███████▊ | 3131/4000 [3:40:23<1:00:29, 4.18s/it]
78%|███████▊ | 3132/4000 [3:40:27<1:00:12, 4.16s/it]
78%|███████▊ | 3133/4000 [3:40:31<1:01:38, 4.27s/it]
78%|███████▊ | 3134/4000 [3:40:35<1:00:59, 4.23s/it]
78%|███████▊ | 3135/4000 [3:40:40<1:00:28, 4.19s/it]
78%|███████▊ | 3136/4000 [3:40:44<1:00:31, 4.20s/it]
78%|███████▊ | 3137/4000 [3:40:48<1:00:05, 4.18s/it]
78%|███████▊ | 3138/4000 [3:40:52<59:51, 4.17s/it]
78%|███████▊ | 3139/4000 [3:40:56<59:35, 4.15s/it]
78%|███████▊ | 3140/4000 [3:41:01<1:02:02, 4.33s/it]
{'loss': '0.397', 'grad_norm': '0.1387', 'learning_rate': '0.0001305', 'epoch': '0.7608'}
78%|███████▊ | 3140/4000 [3:41:01<1:02:02, 4.33s/it]
79%|███████▊ | 3141/4000 [3:41:05<1:01:46, 4.31s/it]
79%|███████▊ | 3142/4000 [3:41:09<1:01:42, 4.32s/it]
79%|███████▊ | 3143/4000 [3:41:14<1:01:07, 4.28s/it]
79%|███████▊ | 3144/4000 [3:41:18<1:01:52, 4.34s/it]
79%|███████▊ | 3145/4000 [3:41:22<1:01:08, 4.29s/it]
79%|███████▊ | 3146/4000 [3:41:27<1:01:20, 4.31s/it]
79%|███████▊ | 3147/4000 [3:41:31<1:01:23, 4.32s/it]
79%|███████▊ | 3148/4000 [3:41:35<1:00:34, 4.27s/it]
79%|███████▊ | 3149/4000 [3:41:39<59:58, 4.23s/it]
79%|███████▉ | 3150/4000 [3:41:43<59:30, 4.20s/it]
{'loss': '0.3628', 'grad_norm': '0.1247', 'learning_rate': '0.0001289', 'epoch': '0.7632'}
79%|███████▉ | 3150/4000 [3:41:43<59:30, 4.20s/it]
79%|███████▉ | 3151/4000 [3:41:48<59:38, 4.21s/it]
79%|███████▉ | 3152/4000 [3:41:52<59:18, 4.20s/it]
79%|███████▉ | 3153/4000 [3:41:56<1:00:27, 4.28s/it]
79%|███████▉ | 3154/4000 [3:42:00<59:47, 4.24s/it]
79%|███████▉ | 3155/4000 [3:42:05<59:14, 4.21s/it]
79%|███████▉ | 3156/4000 [3:42:09<58:54, 4.19s/it]
79%|███████▉ | 3157/4000 [3:42:13<58:37, 4.17s/it]
79%|███████▉ | 3158/4000 [3:42:17<59:19, 4.23s/it]
79%|███████▉ | 3159/4000 [3:42:21<59:21, 4.24s/it]
79%|███████▉ | 3160/4000 [3:42:26<1:01:38, 4.40s/it]
{'loss': '0.3915', 'grad_norm': '0.1734', 'learning_rate': '0.0001274', 'epoch': '0.7657'}
79%|███████▉ | 3160/4000 [3:42:26<1:01:38, 4.40s/it]
79%|███████▉ | 3161/4000 [3:42:31<1:00:51, 4.35s/it]
79%|███████▉ | 3162/4000 [3:42:35<59:54, 4.29s/it]
79%|███████▉ | 3163/4000 [3:42:39<59:09, 4.24s/it]
79%|███████▉ | 3164/4000 [3:42:43<58:37, 4.21s/it]
79%|███████▉ | 3165/4000 [3:42:47<58:13, 4.18s/it]
79%|███████▉ | 3166/4000 [3:42:51<57:55, 4.17s/it]
79%|███████▉ | 3167/4000 [3:42:56<59:37, 4.29s/it]
79%|███████▉ | 3168/4000 [3:43:00<58:58, 4.25s/it]
79%|███████▉ | 3169/4000 [3:43:04<58:24, 4.22s/it]
79%|███████▉ | 3170/4000 [3:43:08<57:58, 4.19s/it]
{'loss': '0.3829', 'grad_norm': '0.1434', 'learning_rate': '0.0001259', 'epoch': '0.7681'}
79%|███████▉ | 3170/4000 [3:43:08<57:58, 4.19s/it]
79%|███████▉ | 3171/4000 [3:43:12<57:38, 4.17s/it]
79%|███████▉ | 3172/4000 [3:43:16<57:24, 4.16s/it]
79%|███████▉ | 3173/4000 [3:43:21<57:55, 4.20s/it]
79%|███████▉ | 3174/4000 [3:43:25<58:21, 4.24s/it]
79%|███████▉ | 3175/4000 [3:43:29<58:16, 4.24s/it]
79%|███████▉ | 3176/4000 [3:43:33<57:45, 4.21s/it]
79%|███████▉ | 3177/4000 [3:43:38<58:03, 4.23s/it]
79%|███████▉ | 3178/4000 [3:43:42<58:40, 4.28s/it]
79%|███████▉ | 3179/4000 [3:43:47<59:05, 4.32s/it]
80%|███████▉ | 3180/4000 [3:43:51<59:57, 4.39s/it]
{'loss': '0.3915', 'grad_norm': '0.1346', 'learning_rate': '0.0001244', 'epoch': '0.7705'}
80%|███████▉ | 3180/4000 [3:43:51<59:57, 4.39s/it]
80%|███████▉ | 3181/4000 [3:43:55<58:53, 4.31s/it]
80%|███████▉ | 3182/4000 [3:44:00<58:44, 4.31s/it]
80%|███████▉ | 3183/4000 [3:44:04<58:30, 4.30s/it]
80%|███████▉ | 3184/4000 [3:44:08<57:49, 4.25s/it]
80%|███████▉ | 3185/4000 [3:44:12<57:17, 4.22s/it]
80%|███████▉ | 3186/4000 [3:44:16<56:55, 4.20s/it]
80%|███████▉ | 3187/4000 [3:44:21<57:54, 4.27s/it]
80%|███████▉ | 3188/4000 [3:44:25<57:18, 4.23s/it]
80%|███████▉ | 3189/4000 [3:44:29<56:50, 4.21s/it]
80%|███████▉ | 3190/4000 [3:44:33<56:31, 4.19s/it]
{'loss': '0.3685', 'grad_norm': '0.1206', 'learning_rate': '0.0001229', 'epoch': '0.7729'}
80%|███████▉ | 3190/4000 [3:44:33<56:31, 4.19s/it]
80%|███████▉ | 3191/4000 [3:44:37<56:21, 4.18s/it]
80%|███████▉ | 3192/4000 [3:44:41<56:26, 4.19s/it]
80%|███████▉ | 3193/4000 [3:44:46<56:06, 4.17s/it]
80%|███████▉ | 3194/4000 [3:44:50<57:30, 4.28s/it]
80%|███████▉ | 3195/4000 [3:44:54<57:13, 4.27s/it]
80%|███████▉ | 3196/4000 [3:44:59<57:29, 4.29s/it]
80%|███████▉ | 3197/4000 [3:45:03<57:10, 4.27s/it]
80%|███████▉ | 3198/4000 [3:45:07<57:08, 4.27s/it]
80%|███████▉ | 3199/4000 [3:45:12<57:12, 4.29s/it]
80%|████████ | 3200/4000 [3:45:16<58:11, 4.36s/it]
{'loss': '0.3733', 'grad_norm': '0.1379', 'learning_rate': '0.0001214', 'epoch': '0.7754'}
80%|████████ | 3200/4000 [3:45:16<58:11, 4.36s/it]
80%|████████ | 3201/4000 [3:45:20<57:10, 4.29s/it]
80%|████████ | 3202/4000 [3:45:24<56:30, 4.25s/it]
80%|████████ | 3203/4000 [3:45:29<56:00, 4.22s/it]
80%|████████ | 3204/4000 [3:45:33<55:38, 4.19s/it]
80%|████████ | 3205/4000 [3:45:37<55:21, 4.18s/it]
80%|████████ | 3206/4000 [3:45:41<55:07, 4.17s/it]
80%|████████ | 3207/4000 [3:45:45<56:24, 4.27s/it]
80%|████████ | 3208/4000 [3:45:50<56:14, 4.26s/it]
80%|████████ | 3209/4000 [3:45:54<55:54, 4.24s/it]
80%|████████ | 3210/4000 [3:45:58<55:30, 4.22s/it]
{'loss': '0.3659', 'grad_norm': '0.1027', 'learning_rate': '0.0001198', 'epoch': '0.7778'}
80%|████████ | 3210/4000 [3:45:58<55:30, 4.22s/it]
80%|████████ | 3211/4000 [3:46:02<55:08, 4.19s/it]
80%|████████ | 3212/4000 [3:46:06<54:53, 4.18s/it]
80%|████████ | 3213/4000 [3:46:11<54:49, 4.18s/it]
80%|████████ | 3214/4000 [3:46:15<57:17, 4.37s/it]
80%|████████ | 3215/4000 [3:46:20<57:21, 4.38s/it]
80%|████████ | 3216/4000 [3:46:24<56:50, 4.35s/it]
80%|████████ | 3217/4000 [3:46:28<56:03, 4.30s/it]
80%|████████ | 3218/4000 [3:46:32<55:21, 4.25s/it]
80%|████████ | 3219/4000 [3:46:36<54:53, 4.22s/it]
80%|████████ | 3220/4000 [3:46:41<54:29, 4.19s/it]
{'loss': '0.394', 'grad_norm': '0.1129', 'learning_rate': '0.0001183', 'epoch': '0.7802'}
80%|████████ | 3220/4000 [3:46:41<54:29, 4.19s/it]
81%|████████ | 3221/4000 [3:46:45<55:33, 4.28s/it]
81%|████████ | 3222/4000 [3:46:49<54:56, 4.24s/it]
81%|████████ | 3223/4000 [3:46:53<54:26, 4.20s/it]
81%|████████ | 3224/4000 [3:46:58<54:48, 4.24s/it]
81%|████████ | 3225/4000 [3:47:02<54:24, 4.21s/it]
81%|████████ | 3226/4000 [3:47:06<54:03, 4.19s/it]
81%|████████ | 3227/4000 [3:47:10<54:39, 4.24s/it]
81%|████████ | 3228/4000 [3:47:15<54:48, 4.26s/it]
81%|████████ | 3229/4000 [3:47:19<54:15, 4.22s/it]
81%|████████ | 3230/4000 [3:47:23<54:12, 4.22s/it]
{'loss': '0.4145', 'grad_norm': '0.1248', 'learning_rate': '0.0001168', 'epoch': '0.7826'}
81%|████████ | 3230/4000 [3:47:23<54:12, 4.22s/it]
81%|████████ | 3231/4000 [3:47:27<53:51, 4.20s/it]
81%|████████ | 3232/4000 [3:47:31<53:46, 4.20s/it]
81%|████████ | 3233/4000 [3:47:36<53:58, 4.22s/it]
81%|████████ | 3234/4000 [3:47:40<55:46, 4.37s/it]
81%|████████ | 3235/4000 [3:47:44<54:59, 4.31s/it]
81%|████████ | 3236/4000 [3:47:49<54:13, 4.26s/it]
81%|████████ | 3237/4000 [3:47:53<53:40, 4.22s/it]
81%|████████ | 3238/4000 [3:47:57<53:14, 4.19s/it]
81%|████████ | 3239/4000 [3:48:01<52:59, 4.18s/it]
81%|████████ | 3240/4000 [3:48:05<53:56, 4.26s/it]
{'loss': '0.3627', 'grad_norm': '0.1018', 'learning_rate': '0.0001153', 'epoch': '0.7851'}
81%|████████ | 3240/4000 [3:48:05<53:56, 4.26s/it]
81%|████████ | 3241/4000 [3:48:10<54:46, 4.33s/it]
81%|████████ | 3242/4000 [3:48:14<53:55, 4.27s/it]
81%|████████ | 3243/4000 [3:48:18<53:18, 4.23s/it]
81%|████████ | 3244/4000 [3:48:22<52:51, 4.20s/it]
81%|████████ | 3245/4000 [3:48:26<52:31, 4.17s/it]
81%|████████ | 3246/4000 [3:48:31<52:15, 4.16s/it]
81%|████████ | 3247/4000 [3:48:35<53:17, 4.25s/it]
81%|████████ | 3248/4000 [3:48:39<53:31, 4.27s/it]
81%|████████ | 3249/4000 [3:48:43<52:54, 4.23s/it]
81%|████████▏ | 3250/4000 [3:48:48<52:27, 4.20s/it]
{'loss': '0.3812', 'grad_norm': '0.1384', 'learning_rate': '0.0001138', 'epoch': '0.7875'}
81%|████████▏ | 3250/4000 [3:48:48<52:27, 4.20s/it]
81%|████████▏ | 3251/4000 [3:48:52<52:13, 4.18s/it]
81%|████████▏ | 3252/4000 [3:48:56<52:31, 4.21s/it]
81%|████████▏ | 3253/4000 [3:49:00<52:37, 4.23s/it]
81%|████████▏ | 3254/4000 [3:49:05<54:16, 4.37s/it]
81%|████████▏ | 3255/4000 [3:49:09<54:10, 4.36s/it]
81%|████████▏ | 3256/4000 [3:49:14<53:19, 4.30s/it]
81%|████████▏ | 3257/4000 [3:49:18<52:40, 4.25s/it]
81%|████████▏ | 3258/4000 [3:49:22<52:07, 4.21s/it]
81%|████████▏ | 3259/4000 [3:49:26<51:45, 4.19s/it]
82%|████████▏ | 3260/4000 [3:49:30<51:30, 4.18s/it]
{'loss': '0.3587', 'grad_norm': '0.1204', 'learning_rate': '0.0001123', 'epoch': '0.7899'}
82%|████████▏ | 3260/4000 [3:49:30<51:30, 4.18s/it]
82%|████████▏ | 3261/4000 [3:49:35<52:39, 4.27s/it]
82%|████████▏ | 3262/4000 [3:49:39<52:01, 4.23s/it]
82%|████████▏ | 3263/4000 [3:49:43<52:31, 4.28s/it]
82%|████████▏ | 3264/4000 [3:49:47<51:59, 4.24s/it]
82%|████████▏ | 3265/4000 [3:49:51<51:33, 4.21s/it]
82%|████████▏ | 3266/4000 [3:49:56<51:14, 4.19s/it]
82%|████████▏ | 3267/4000 [3:50:00<50:55, 4.17s/it]
82%|████████▏ | 3268/4000 [3:50:04<52:04, 4.27s/it]
82%|████████▏ | 3269/4000 [3:50:08<51:33, 4.23s/it]
82%|████████▏ | 3270/4000 [3:50:12<51:11, 4.21s/it]
{'loss': '0.3955', 'grad_norm': '0.1113', 'learning_rate': '0.0001108', 'epoch': '0.7923'}
82%|████████▏ | 3270/4000 [3:50:12<51:11, 4.21s/it]
82%|████████▏ | 3271/4000 [3:50:17<51:37, 4.25s/it]
82%|████████▏ | 3272/4000 [3:50:21<51:37, 4.26s/it]
82%|████████▏ | 3273/4000 [3:50:25<51:40, 4.26s/it]
82%|████████▏ | 3274/4000 [3:50:30<52:38, 4.35s/it]
82%|████████▏ | 3275/4000 [3:50:34<52:49, 4.37s/it]
82%|████████▏ | 3276/4000 [3:50:38<51:53, 4.30s/it]
82%|████████▏ | 3277/4000 [3:50:43<51:11, 4.25s/it]
82%|████████▏ | 3278/4000 [3:50:47<50:41, 4.21s/it]
82%|████████▏ | 3279/4000 [3:50:51<50:48, 4.23s/it]
82%|████████▏ | 3280/4000 [3:50:55<50:26, 4.20s/it]
{'loss': '0.3985', 'grad_norm': '0.123', 'learning_rate': '0.0001092', 'epoch': '0.7947'}
82%|████████▏ | 3280/4000 [3:50:55<50:26, 4.20s/it]
82%|████████▏ | 3281/4000 [3:51:00<51:29, 4.30s/it]
82%|████████▏ | 3282/4000 [3:51:04<50:50, 4.25s/it]
82%|████████▏ | 3283/4000 [3:51:08<50:19, 4.21s/it]
82%|████████▏ | 3284/4000 [3:51:12<49:57, 4.19s/it]
82%|████████▏ | 3285/4000 [3:51:16<49:39, 4.17s/it]
82%|████████▏ | 3286/4000 [3:51:20<49:29, 4.16s/it]
82%|████████▏ | 3287/4000 [3:51:24<49:20, 4.15s/it]
82%|████████▏ | 3288/4000 [3:51:29<50:20, 4.24s/it]
82%|████████▏ | 3289/4000 [3:51:33<49:51, 4.21s/it]
82%|████████▏ | 3290/4000 [3:51:37<49:31, 4.19s/it]
{'loss': '0.3756', 'grad_norm': '0.1178', 'learning_rate': '0.0001077', 'epoch': '0.7972'}
82%|████████▏ | 3290/4000 [3:51:37<49:31, 4.19s/it]
82%|████████▏ | 3291/4000 [3:51:41<50:02, 4.23s/it]
82%|████████▏ | 3292/4000 [3:51:46<50:01, 4.24s/it]
82%|████████▏ | 3293/4000 [3:51:50<50:10, 4.26s/it]
82%|████████▏ | 3294/4000 [3:51:54<50:03, 4.25s/it]
82%|████████▏ | 3295/4000 [3:51:59<51:35, 4.39s/it]
82%|████████▏ | 3296/4000 [3:52:03<51:35, 4.40s/it]
82%|████████▏ | 3297/4000 [3:52:08<50:37, 4.32s/it]
82%|████████▏ | 3298/4000 [3:52:12<49:55, 4.27s/it]
82%|████████▏ | 3299/4000 [3:52:16<49:25, 4.23s/it]
82%|████████▎ | 3300/4000 [3:52:20<49:01, 4.20s/it]
{'loss': '0.3911', 'grad_norm': '0.1171', 'learning_rate': '0.0001062', 'epoch': '0.7996'}
82%|████████▎ | 3300/4000 [3:52:20<49:01, 4.20s/it]
83%|████████▎ | 3301/4000 [3:52:24<49:31, 4.25s/it]
83%|████████▎ | 3302/4000 [3:52:29<49:41, 4.27s/it]
83%|████████▎ | 3303/4000 [3:52:33<49:09, 4.23s/it]
83%|████████▎ | 3304/4000 [3:52:37<48:47, 4.21s/it]
83%|████████▎ | 3305/4000 [3:52:41<48:28, 4.18s/it]
83%|████████▎ | 3306/4000 [3:52:45<48:15, 4.17s/it]
83%|████████▎ | 3307/4000 [3:52:49<48:04, 4.16s/it]
83%|████████▎ | 3308/4000 [3:52:54<49:11, 4.27s/it]
83%|████████▎ | 3309/4000 [3:52:58<48:45, 4.23s/it]
83%|████████▎ | 3310/4000 [3:53:02<48:39, 4.23s/it]
{'loss': '0.4119', 'grad_norm': '0.1113', 'learning_rate': '0.0001047', 'epoch': '0.802'}
83%|████████▎ | 3310/4000 [3:53:02<48:39, 4.23s/it]
83%|████████▎ | 3311/4000 [3:53:07<48:57, 4.26s/it]
83%|████████▎ | 3312/4000 [3:53:11<48:41, 4.25s/it]
83%|████████▎ | 3313/4000 [3:53:15<48:41, 4.25s/it]
83%|████████▎ | 3314/4000 [3:53:19<48:37, 4.25s/it]
83%|████████▎ | 3315/4000 [3:53:24<50:10, 4.40s/it]
83%|████████▎ | 3316/4000 [3:53:28<49:27, 4.34s/it]
83%|████████▎ | 3317/4000 [3:53:32<48:40, 4.28s/it]
83%|████████▎ | 3318/4000 [3:53:36<48:04, 4.23s/it]
83%|████████▎ | 3319/4000 [3:53:41<47:38, 4.20s/it]
83%|████████▎ | 3320/4000 [3:53:45<47:19, 4.18s/it]
{'loss': '0.3591', 'grad_norm': '0.1191', 'learning_rate': '0.0001032', 'epoch': '0.8044'}
83%|████████▎ | 3320/4000 [3:53:45<47:19, 4.18s/it]
83%|████████▎ | 3321/4000 [3:53:49<47:05, 4.16s/it]
83%|████████▎ | 3322/4000 [3:53:53<48:06, 4.26s/it]
83%|████████▎ | 3323/4000 [3:53:57<47:38, 4.22s/it]
83%|████████▎ | 3324/4000 [3:54:02<47:13, 4.19s/it]
83%|████████▎ | 3325/4000 [3:54:06<46:58, 4.17s/it]
83%|████████▎ | 3326/4000 [3:54:10<46:47, 4.16s/it]
83%|████████▎ | 3327/4000 [3:54:14<46:33, 4.15s/it]
83%|████████▎ | 3328/4000 [3:54:18<47:38, 4.25s/it]
83%|████████▎ | 3329/4000 [3:54:23<47:56, 4.29s/it]
83%|████████▎ | 3330/4000 [3:54:27<47:31, 4.26s/it]
{'loss': '0.3953', 'grad_norm': '0.1186', 'learning_rate': '0.0001017', 'epoch': '0.8069'}
83%|████████▎ | 3330/4000 [3:54:27<47:31, 4.26s/it]
83%|████████▎ | 3331/4000 [3:54:31<47:04, 4.22s/it]
83%|████████▎ | 3332/4000 [3:54:36<47:26, 4.26s/it]
83%|████████▎ | 3333/4000 [3:54:40<47:27, 4.27s/it]
83%|████████▎ | 3334/4000 [3:54:44<47:18, 4.26s/it]
83%|████████▎ | 3335/4000 [3:54:49<48:50, 4.41s/it]
83%|████████▎ | 3336/4000 [3:54:53<48:06, 4.35s/it]
83%|████████▎ | 3337/4000 [3:54:57<47:21, 4.29s/it]
83%|████████▎ | 3338/4000 [3:55:01<46:46, 4.24s/it]
83%|████████▎ | 3339/4000 [3:55:05<46:20, 4.21s/it]
84%|████████▎ | 3340/4000 [3:55:10<46:03, 4.19s/it]
{'loss': '0.3978', 'grad_norm': '0.1635', 'learning_rate': '0.0001002', 'epoch': '0.8093'}
84%|████████▎ | 3340/4000 [3:55:10<46:03, 4.19s/it]
84%|████████▎ | 3341/4000 [3:55:14<45:49, 4.17s/it]
84%|████████▎ | 3342/4000 [3:55:18<46:55, 4.28s/it]
84%|████████▎ | 3343/4000 [3:55:22<46:27, 4.24s/it]
84%|████████▎ | 3344/4000 [3:55:27<46:04, 4.21s/it]
84%|████████▎ | 3345/4000 [3:55:31<45:44, 4.19s/it]
84%|████████▎ | 3346/4000 [3:55:35<45:52, 4.21s/it]
84%|████████▎ | 3347/4000 [3:55:39<45:58, 4.22s/it]
84%|████████▎ | 3348/4000 [3:55:43<45:38, 4.20s/it]
84%|████████▎ | 3349/4000 [3:55:48<46:34, 4.29s/it]
84%|████████▍ | 3350/4000 [3:55:52<46:26, 4.29s/it]
{'loss': '0.3527', 'grad_norm': '0.1133', 'learning_rate': '9.864e-05', 'epoch': '0.8117'}
84%|████████▍ | 3350/4000 [3:55:52<46:26, 4.29s/it]
84%|████████▍ | 3351/4000 [3:55:56<45:54, 4.24s/it]
84%|████████▍ | 3352/4000 [3:56:00<45:45, 4.24s/it]
84%|████████▍ | 3353/4000 [3:56:05<45:41, 4.24s/it]
84%|████████▍ | 3354/4000 [3:56:09<46:19, 4.30s/it]
84%|████████▍ | 3355/4000 [3:56:14<47:00, 4.37s/it]
84%|████████▍ | 3356/4000 [3:56:18<46:09, 4.30s/it]
84%|████████▍ | 3357/4000 [3:56:22<45:51, 4.28s/it]
84%|████████▍ | 3358/4000 [3:56:26<45:15, 4.23s/it]
84%|████████▍ | 3359/4000 [3:56:30<44:54, 4.20s/it]
84%|████████▍ | 3360/4000 [3:56:34<44:37, 4.18s/it]
{'loss': '0.4003', 'grad_norm': '0.1286', 'learning_rate': '9.712e-05', 'epoch': '0.8141'}
84%|████████▍ | 3360/4000 [3:56:34<44:37, 4.18s/it]
84%|████████▍ | 3361/4000 [3:56:39<44:22, 4.17s/it]
84%|████████▍ | 3362/4000 [3:56:43<45:26, 4.27s/it]
84%|████████▍ | 3363/4000 [3:56:47<45:37, 4.30s/it]
84%|████████▍ | 3364/4000 [3:56:52<45:10, 4.26s/it]
84%|████████▍ | 3365/4000 [3:56:56<45:03, 4.26s/it]
84%|████████▍ | 3366/4000 [3:57:00<44:36, 4.22s/it]
84%|████████▍ | 3367/4000 [3:57:04<44:29, 4.22s/it]
84%|████████▍ | 3368/4000 [3:57:08<44:17, 4.20s/it]
84%|████████▍ | 3369/4000 [3:57:13<45:54, 4.36s/it]
84%|████████▍ | 3370/4000 [3:57:17<45:09, 4.30s/it]
{'loss': '0.3655', 'grad_norm': '0.1472', 'learning_rate': '9.561e-05', 'epoch': '0.8165'}
84%|████████▍ | 3370/4000 [3:57:17<45:09, 4.30s/it]
84%|████████▍ | 3371/4000 [3:57:21<44:34, 4.25s/it]
84%|████████▍ | 3372/4000 [3:57:26<45:06, 4.31s/it]
84%|████████▍ | 3373/4000 [3:57:30<44:32, 4.26s/it]
84%|████████▍ | 3374/4000 [3:57:34<44:03, 4.22s/it]
84%|████████▍ | 3375/4000 [3:57:38<44:20, 4.26s/it]
84%|████████▍ | 3376/4000 [3:57:43<44:24, 4.27s/it]
84%|████████▍ | 3377/4000 [3:57:47<43:53, 4.23s/it]
84%|████████▍ | 3378/4000 [3:57:51<43:29, 4.20s/it]
84%|████████▍ | 3379/4000 [3:57:55<43:11, 4.17s/it]
84%|████████▍ | 3380/4000 [3:57:59<43:12, 4.18s/it]
{'loss': '0.3818', 'grad_norm': '0.1262', 'learning_rate': '9.409e-05', 'epoch': '0.819'}
84%|████████▍ | 3380/4000 [3:57:59<43:12, 4.18s/it]
85%|████████▍ | 3381/4000 [3:58:04<43:27, 4.21s/it]
85%|████████▍ | 3382/4000 [3:58:08<44:49, 4.35s/it]
85%|████████▍ | 3383/4000 [3:58:12<44:04, 4.29s/it]
85%|████████▍ | 3384/4000 [3:58:17<43:54, 4.28s/it]
85%|████████▍ | 3385/4000 [3:58:21<43:45, 4.27s/it]
85%|████████▍ | 3386/4000 [3:58:25<43:17, 4.23s/it]
85%|████████▍ | 3387/4000 [3:58:29<42:57, 4.20s/it]
85%|████████▍ | 3388/4000 [3:58:33<42:38, 4.18s/it]
85%|████████▍ | 3389/4000 [3:58:38<43:46, 4.30s/it]
85%|████████▍ | 3390/4000 [3:58:42<44:04, 4.33s/it]
{'loss': '0.3634', 'grad_norm': '0.1173', 'learning_rate': '9.258e-05', 'epoch': '0.8214'}
85%|████████▍ | 3390/4000 [3:58:42<44:04, 4.33s/it]
85%|████████▍ | 3391/4000 [3:58:47<43:28, 4.28s/it]
85%|████████▍ | 3392/4000 [3:58:51<42:57, 4.24s/it]
85%|████████▍ | 3393/4000 [3:58:55<42:32, 4.20s/it]
85%|████████▍ | 3394/4000 [3:58:59<42:14, 4.18s/it]
85%|████████▍ | 3395/4000 [3:59:03<42:00, 4.17s/it]
85%|████████▍ | 3396/4000 [3:59:08<43:02, 4.28s/it]
85%|████████▍ | 3397/4000 [3:59:12<42:32, 4.23s/it]
85%|████████▍ | 3398/4000 [3:59:16<42:28, 4.23s/it]
85%|████████▍ | 3399/4000 [3:59:20<42:29, 4.24s/it]
85%|████████▌ | 3400/4000 [3:59:24<42:13, 4.22s/it]
{'loss': '0.3955', 'grad_norm': '0.114', 'learning_rate': '9.106e-05', 'epoch': '0.8238'}
85%|████████▌ | 3400/4000 [3:59:24<42:13, 4.22s/it]
85%|████████▌ | 3401/4000 [3:59:29<42:31, 4.26s/it]
85%|████████▌ | 3402/4000 [3:59:33<43:14, 4.34s/it]
85%|████████▌ | 3403/4000 [3:59:38<43:12, 4.34s/it]
85%|████████▌ | 3404/4000 [3:59:42<42:28, 4.28s/it]
85%|████████▌ | 3405/4000 [3:59:46<41:58, 4.23s/it]
85%|████████▌ | 3406/4000 [3:59:50<41:35, 4.20s/it]
85%|████████▌ | 3407/4000 [3:59:54<42:15, 4.28s/it]
85%|████████▌ | 3408/4000 [3:59:59<42:01, 4.26s/it]
85%|████████▌ | 3409/4000 [4:00:03<42:42, 4.34s/it]
85%|████████▌ | 3410/4000 [4:00:07<42:04, 4.28s/it]
{'loss': '0.3678', 'grad_norm': '0.1184', 'learning_rate': '8.955e-05', 'epoch': '0.8262'}
85%|████████▌ | 3410/4000 [4:00:07<42:04, 4.28s/it]
85%|████████▌ | 3411/4000 [4:00:11<41:33, 4.23s/it]
85%|████████▌ | 3412/4000 [4:00:16<41:10, 4.20s/it]
85%|████████▌ | 3413/4000 [4:00:20<40:55, 4.18s/it]
85%|████████▌ | 3414/4000 [4:00:24<40:39, 4.16s/it]
85%|████████▌ | 3415/4000 [4:00:28<41:11, 4.23s/it]
85%|████████▌ | 3416/4000 [4:00:33<42:52, 4.41s/it]
85%|████████▌ | 3417/4000 [4:00:37<42:06, 4.33s/it]
85%|████████▌ | 3418/4000 [4:00:42<42:05, 4.34s/it]
85%|████████▌ | 3419/4000 [4:00:46<41:36, 4.30s/it]
86%|████████▌ | 3420/4000 [4:00:50<41:00, 4.24s/it]
{'loss': '0.376', 'grad_norm': '0.1444', 'learning_rate': '8.803e-05', 'epoch': '0.8287'}
86%|████████▌ | 3420/4000 [4:00:50<41:00, 4.24s/it]
86%|████████▌ | 3421/4000 [4:00:54<40:40, 4.21s/it]
86%|████████▌ | 3422/4000 [4:00:58<40:23, 4.19s/it]
86%|████████▌ | 3423/4000 [4:01:03<41:07, 4.28s/it]
86%|████████▌ | 3424/4000 [4:01:07<41:25, 4.32s/it]
86%|████████▌ | 3425/4000 [4:01:11<41:20, 4.31s/it]
86%|████████▌ | 3426/4000 [4:01:15<40:46, 4.26s/it]
86%|████████▌ | 3427/4000 [4:01:20<40:19, 4.22s/it]
86%|████████▌ | 3428/4000 [4:01:24<39:58, 4.19s/it]
86%|████████▌ | 3429/4000 [4:01:28<40:17, 4.23s/it]
86%|████████▌ | 3430/4000 [4:01:32<40:25, 4.26s/it]
{'loss': '0.397', 'grad_norm': '0.1081', 'learning_rate': '8.652e-05', 'epoch': '0.8311'}
86%|████████▌ | 3430/4000 [4:01:32<40:25, 4.26s/it]
86%|████████▌ | 3431/4000 [4:01:36<39:59, 4.22s/it]
86%|████████▌ | 3432/4000 [4:01:41<40:25, 4.27s/it]
86%|████████▌ | 3433/4000 [4:01:45<40:32, 4.29s/it]
86%|████████▌ | 3434/4000 [4:01:50<40:40, 4.31s/it]
86%|████████▌ | 3435/4000 [4:01:54<40:22, 4.29s/it]
86%|████████▌ | 3436/4000 [4:01:58<41:18, 4.39s/it]
86%|████████▌ | 3437/4000 [4:02:03<40:28, 4.31s/it]
86%|████████▌ | 3438/4000 [4:02:07<39:51, 4.26s/it]
86%|████████▌ | 3439/4000 [4:02:11<39:25, 4.22s/it]
86%|████████▌ | 3440/4000 [4:02:15<39:05, 4.19s/it]
{'loss': '0.3547', 'grad_norm': '0.1245', 'learning_rate': '8.5e-05', 'epoch': '0.8335'}
86%|████████▌ | 3440/4000 [4:02:15<39:05, 4.19s/it]
86%|████████▌ | 3441/4000 [4:02:19<39:03, 4.19s/it]
86%|████████▌ | 3442/4000 [4:02:23<38:55, 4.19s/it]
86%|████████▌ | 3443/4000 [4:02:28<39:50, 4.29s/it]
86%|████████▌ | 3444/4000 [4:02:32<39:16, 4.24s/it]
86%|████████▌ | 3445/4000 [4:02:36<38:52, 4.20s/it]
86%|████████▌ | 3446/4000 [4:02:40<38:36, 4.18s/it]
86%|████████▌ | 3447/4000 [4:02:44<38:22, 4.16s/it]
86%|████████▌ | 3448/4000 [4:02:48<38:10, 4.15s/it]
86%|████████▌ | 3449/4000 [4:02:53<38:31, 4.20s/it]
86%|████████▋ | 3450/4000 [4:02:57<39:25, 4.30s/it]
{'loss': '0.4024', 'grad_norm': '0.117', 'learning_rate': '8.348e-05', 'epoch': '0.8359'}
86%|████████▋ | 3450/4000 [4:02:57<39:25, 4.30s/it]
86%|████████▋ | 3451/4000 [4:03:02<39:43, 4.34s/it]
86%|████████▋ | 3452/4000 [4:03:06<39:45, 4.35s/it]
86%|████████▋ | 3453/4000 [4:03:10<39:05, 4.29s/it]
86%|████████▋ | 3454/4000 [4:03:14<38:36, 4.24s/it]
86%|████████▋ | 3455/4000 [4:03:19<38:12, 4.21s/it]
86%|████████▋ | 3456/4000 [4:03:23<38:21, 4.23s/it]
86%|████████▋ | 3457/4000 [4:03:27<38:27, 4.25s/it]
86%|████████▋ | 3458/4000 [4:03:31<38:32, 4.27s/it]
86%|████████▋ | 3459/4000 [4:03:36<38:25, 4.26s/it]
86%|████████▋ | 3460/4000 [4:03:40<38:10, 4.24s/it]
{'loss': '0.3786', 'grad_norm': '0.1311', 'learning_rate': '8.197e-05', 'epoch': '0.8384'}
86%|████████▋ | 3460/4000 [4:03:40<38:10, 4.24s/it]
87%|████████▋ | 3461/4000 [4:03:44<37:49, 4.21s/it]
87%|████████▋ | 3462/4000 [4:03:48<37:33, 4.19s/it]
87%|████████▋ | 3463/4000 [4:03:53<38:20, 4.28s/it]
87%|████████▋ | 3464/4000 [4:03:57<37:50, 4.24s/it]
87%|████████▋ | 3465/4000 [4:04:01<37:29, 4.20s/it]
87%|████████▋ | 3466/4000 [4:04:05<37:25, 4.20s/it]
87%|████████▋ | 3467/4000 [4:04:09<37:26, 4.21s/it]
87%|████████▋ | 3468/4000 [4:04:14<37:29, 4.23s/it]
87%|████████▋ | 3469/4000 [4:04:18<37:45, 4.27s/it]
87%|████████▋ | 3470/4000 [4:04:23<38:33, 4.37s/it]
{'loss': '0.3573', 'grad_norm': '0.1071', 'learning_rate': '8.045e-05', 'epoch': '0.8408'}
87%|████████▋ | 3470/4000 [4:04:23<38:33, 4.37s/it]
87%|████████▋ | 3471/4000 [4:04:27<37:51, 4.29s/it]
87%|████████▋ | 3472/4000 [4:04:31<37:23, 4.25s/it]
87%|████████▋ | 3473/4000 [4:04:35<37:00, 4.21s/it]
87%|████████▋ | 3474/4000 [4:04:39<36:43, 4.19s/it]
87%|████████▋ | 3475/4000 [4:04:43<37:10, 4.25s/it]
87%|████████▋ | 3476/4000 [4:04:48<36:50, 4.22s/it]
87%|████████▋ | 3477/4000 [4:04:52<37:55, 4.35s/it]
87%|████████▋ | 3478/4000 [4:04:56<37:24, 4.30s/it]
87%|████████▋ | 3479/4000 [4:05:01<36:54, 4.25s/it]
87%|████████▋ | 3480/4000 [4:05:05<36:30, 4.21s/it]
{'loss': '0.3821', 'grad_norm': '0.1146', 'learning_rate': '7.894e-05', 'epoch': '0.8432'}
87%|████████▋ | 3480/4000 [4:05:05<36:30, 4.21s/it]
87%|████████▋ | 3481/4000 [4:05:09<36:14, 4.19s/it]
87%|████████▋ | 3482/4000 [4:05:13<36:00, 4.17s/it]
87%|████████▋ | 3483/4000 [4:05:18<37:12, 4.32s/it]
87%|████████▋ | 3484/4000 [4:05:22<36:48, 4.28s/it]
87%|████████▋ | 3485/4000 [4:05:26<36:48, 4.29s/it]
87%|████████▋ | 3486/4000 [4:05:30<36:44, 4.29s/it]
87%|████████▋ | 3487/4000 [4:05:35<36:31, 4.27s/it]
87%|████████▋ | 3488/4000 [4:05:39<36:07, 4.23s/it]
87%|████████▋ | 3489/4000 [4:05:43<35:49, 4.21s/it]
87%|████████▋ | 3490/4000 [4:05:47<36:29, 4.29s/it]
{'loss': '0.4101', 'grad_norm': '0.1131', 'learning_rate': '7.742e-05', 'epoch': '0.8456'}
87%|████████▋ | 3490/4000 [4:05:47<36:29, 4.29s/it]
87%|████████▋ | 3491/4000 [4:05:52<36:01, 4.25s/it]
87%|████████▋ | 3492/4000 [4:05:56<35:54, 4.24s/it]
87%|████████▋ | 3493/4000 [4:06:00<35:32, 4.21s/it]
87%|████████▋ | 3494/4000 [4:06:04<35:40, 4.23s/it]
87%|████████▋ | 3495/4000 [4:06:08<35:22, 4.20s/it]
87%|████████▋ | 3496/4000 [4:06:13<35:09, 4.19s/it]
87%|████████▋ | 3497/4000 [4:06:17<35:53, 4.28s/it]
87%|████████▋ | 3498/4000 [4:06:21<35:25, 4.23s/it]
87%|████████▋ | 3499/4000 [4:06:25<35:05, 4.20s/it]
88%|████████▊ | 3500/4000 [4:06:29<34:50, 4.18s/it]
{'loss': '0.375', 'grad_norm': '0.121', 'learning_rate': '7.591e-05', 'epoch': '0.848'}
88%|████████▊ | 3500/4000 [4:06:29<34:50, 4.18s/it]
88%|████████▊ | 3501/4000 [4:06:34<35:24, 4.26s/it]
88%|████████▊ | 3502/4000 [4:06:38<35:30, 4.28s/it]
88%|████████▊ | 3503/4000 [4:06:43<35:38, 4.30s/it]
88%|████████▊ | 3504/4000 [4:06:47<36:18, 4.39s/it]
88%|████████▊ | 3505/4000 [4:06:51<35:40, 4.32s/it]
88%|████████▊ | 3506/4000 [4:06:55<35:05, 4.26s/it]
88%|████████▊ | 3507/4000 [4:07:00<34:41, 4.22s/it]
88%|████████▊ | 3508/4000 [4:07:04<34:44, 4.24s/it]
88%|████████▊ | 3509/4000 [4:07:08<34:45, 4.25s/it]
88%|████████▊ | 3510/4000 [4:07:13<35:17, 4.32s/it]
{'loss': '0.3447', 'grad_norm': '0.1257', 'learning_rate': '7.439e-05', 'epoch': '0.8505'}
88%|████████▊ | 3510/4000 [4:07:13<35:17, 4.32s/it]
88%|████████▊ | 3511/4000 [4:07:17<34:53, 4.28s/it]
88%|████████▊ | 3512/4000 [4:07:21<34:27, 4.24s/it]
88%|████████▊ | 3513/4000 [4:07:25<34:06, 4.20s/it]
88%|████████▊ | 3514/4000 [4:07:29<33:49, 4.18s/it]
88%|████████▊ | 3515/4000 [4:07:33<33:36, 4.16s/it]
88%|████████▊ | 3516/4000 [4:07:37<33:29, 4.15s/it]
88%|████████▊ | 3517/4000 [4:07:42<34:15, 4.26s/it]
88%|████████▊ | 3518/4000 [4:07:46<34:06, 4.25s/it]
88%|████████▊ | 3519/4000 [4:07:50<34:03, 4.25s/it]
88%|████████▊ | 3520/4000 [4:07:55<33:51, 4.23s/it]
{'loss': '0.3932', 'grad_norm': '0.1423', 'learning_rate': '7.288e-05', 'epoch': '0.8529'}
88%|████████▊ | 3520/4000 [4:07:55<33:51, 4.23s/it]
88%|████████▊ | 3521/4000 [4:07:59<34:19, 4.30s/it]
88%|████████▊ | 3522/4000 [4:08:03<33:59, 4.27s/it]
88%|████████▊ | 3523/4000 [4:08:07<33:36, 4.23s/it]
88%|████████▊ | 3524/4000 [4:08:12<34:05, 4.30s/it]
88%|████████▊ | 3525/4000 [4:08:16<34:13, 4.32s/it]
88%|████████▊ | 3526/4000 [4:08:20<33:45, 4.27s/it]
88%|████████▊ | 3527/4000 [4:08:24<33:21, 4.23s/it]
88%|████████▊ | 3528/4000 [4:08:29<33:05, 4.21s/it]
88%|████████▊ | 3529/4000 [4:08:33<33:00, 4.20s/it]
88%|████████▊ | 3530/4000 [4:08:37<32:45, 4.18s/it]
{'loss': '0.4123', 'grad_norm': '0.1177', 'learning_rate': '7.136e-05', 'epoch': '0.8553'}
88%|████████▊ | 3530/4000 [4:08:37<32:45, 4.18s/it]
88%|████████▊ | 3531/4000 [4:08:41<33:28, 4.28s/it]
88%|████████▊ | 3532/4000 [4:08:46<33:00, 4.23s/it]
88%|████████▊ | 3533/4000 [4:08:50<32:43, 4.20s/it]
88%|████████▊ | 3534/4000 [4:08:54<32:30, 4.18s/it]
88%|████████▊ | 3535/4000 [4:08:58<32:33, 4.20s/it]
88%|████████▊ | 3536/4000 [4:09:02<32:22, 4.19s/it]
88%|████████▊ | 3537/4000 [4:09:07<33:28, 4.34s/it]
88%|████████▊ | 3538/4000 [4:09:11<33:29, 4.35s/it]
88%|████████▊ | 3539/4000 [4:09:16<33:14, 4.33s/it]
88%|████████▊ | 3540/4000 [4:09:20<32:44, 4.27s/it]
{'loss': '0.361', 'grad_norm': '0.1338', 'learning_rate': '6.985e-05', 'epoch': '0.8577'}
88%|████████▊ | 3540/4000 [4:09:20<32:44, 4.27s/it]
89%|████████▊ | 3541/4000 [4:09:24<32:22, 4.23s/it]
89%|████████▊ | 3542/4000 [4:09:28<32:22, 4.24s/it]
89%|████████▊ | 3543/4000 [4:09:32<32:04, 4.21s/it]
89%|████████▊ | 3544/4000 [4:09:37<32:39, 4.30s/it]
89%|████████▊ | 3545/4000 [4:09:41<32:13, 4.25s/it]
89%|████████▊ | 3546/4000 [4:09:45<32:12, 4.26s/it]
89%|████████▊ | 3547/4000 [4:09:49<31:51, 4.22s/it]
89%|████████▊ | 3548/4000 [4:09:53<31:34, 4.19s/it]
89%|████████▊ | 3549/4000 [4:09:58<31:21, 4.17s/it]
89%|████████▉ | 3550/4000 [4:10:02<31:09, 4.15s/it]
{'loss': '0.3551', 'grad_norm': '0.1287', 'learning_rate': '6.833e-05', 'epoch': '0.8602'}
89%|████████▉ | 3550/4000 [4:10:02<31:09, 4.15s/it]
89%|████████▉ | 3551/4000 [4:10:06<31:55, 4.27s/it]
89%|████████▉ | 3552/4000 [4:10:11<31:58, 4.28s/it]
89%|████████▉ | 3553/4000 [4:10:15<31:34, 4.24s/it]
89%|████████▉ | 3554/4000 [4:10:19<31:34, 4.25s/it]
89%|████████▉ | 3555/4000 [4:10:23<31:50, 4.29s/it]
89%|████████▉ | 3556/4000 [4:10:28<31:36, 4.27s/it]
89%|████████▉ | 3557/4000 [4:10:32<31:35, 4.28s/it]
89%|████████▉ | 3558/4000 [4:10:36<31:32, 4.28s/it]
89%|████████▉ | 3559/4000 [4:10:40<31:27, 4.28s/it]
89%|████████▉ | 3560/4000 [4:10:45<31:05, 4.24s/it]
{'loss': '0.3892', 'grad_norm': '0.1238', 'learning_rate': '6.682e-05', 'epoch': '0.8626'}
89%|████████▉ | 3560/4000 [4:10:45<31:05, 4.24s/it]
89%|████████▉ | 3561/4000 [4:10:49<30:45, 4.20s/it]
89%|████████▉ | 3562/4000 [4:10:53<30:30, 4.18s/it]
89%|████████▉ | 3563/4000 [4:10:57<30:25, 4.18s/it]
89%|████████▉ | 3564/4000 [4:11:01<31:03, 4.27s/it]
89%|████████▉ | 3565/4000 [4:11:06<30:41, 4.23s/it]
89%|████████▉ | 3566/4000 [4:11:10<30:25, 4.21s/it]
89%|████████▉ | 3567/4000 [4:11:14<30:11, 4.18s/it]
89%|████████▉ | 3568/4000 [4:11:18<30:00, 4.17s/it]
89%|████████▉ | 3569/4000 [4:11:22<30:07, 4.19s/it]
89%|████████▉ | 3570/4000 [4:11:26<29:56, 4.18s/it]
{'loss': '0.3515', 'grad_norm': '0.1203', 'learning_rate': '6.53e-05', 'epoch': '0.865'}
89%|████████▉ | 3570/4000 [4:11:26<29:56, 4.18s/it]
89%|████████▉ | 3571/4000 [4:11:31<30:57, 4.33s/it]
89%|████████▉ | 3572/4000 [4:11:35<30:59, 4.35s/it]
89%|████████▉ | 3573/4000 [4:11:40<30:40, 4.31s/it]
89%|████████▉ | 3574/4000 [4:11:44<30:17, 4.27s/it]
89%|████████▉ | 3575/4000 [4:11:48<29:54, 4.22s/it]
89%|████████▉ | 3576/4000 [4:11:52<29:50, 4.22s/it]
89%|████████▉ | 3577/4000 [4:11:56<29:37, 4.20s/it]
89%|████████▉ | 3578/4000 [4:12:01<30:10, 4.29s/it]
89%|████████▉ | 3579/4000 [4:12:05<29:44, 4.24s/it]
90%|████████▉ | 3580/4000 [4:12:09<29:32, 4.22s/it]
{'loss': '0.3733', 'grad_norm': '0.1103', 'learning_rate': '6.379e-05', 'epoch': '0.8674'}
90%|████████▉ | 3580/4000 [4:12:09<29:32, 4.22s/it]
90%|████████▉ | 3581/4000 [4:12:13<29:20, 4.20s/it]
90%|████████▉ | 3582/4000 [4:12:17<29:06, 4.18s/it]
90%|████████▉ | 3583/4000 [4:12:22<28:57, 4.17s/it]
90%|████████▉ | 3584/4000 [4:12:26<29:12, 4.21s/it]
90%|████████▉ | 3585/4000 [4:12:30<29:20, 4.24s/it]
90%|████████▉ | 3586/4000 [4:12:35<29:38, 4.30s/it]
90%|████████▉ | 3587/4000 [4:12:39<29:19, 4.26s/it]
90%|████████▉ | 3588/4000 [4:12:43<29:27, 4.29s/it]
90%|████████▉ | 3589/4000 [4:12:47<29:14, 4.27s/it]
90%|████████▉ | 3590/4000 [4:12:52<29:03, 4.25s/it]
{'loss': '0.3689', 'grad_norm': '0.1228', 'learning_rate': '6.227e-05', 'epoch': '0.8699'}
90%|████████▉ | 3590/4000 [4:12:52<29:03, 4.25s/it]
90%|████████▉ | 3591/4000 [4:12:56<29:50, 4.38s/it]
90%|████████▉ | 3592/4000 [4:13:01<29:30, 4.34s/it]
90%|████████▉ | 3593/4000 [4:13:05<29:19, 4.32s/it]
90%|████████▉ | 3594/4000 [4:13:09<28:56, 4.28s/it]
90%|████████▉ | 3595/4000 [4:13:13<28:35, 4.24s/it]
90%|████████▉ | 3596/4000 [4:13:17<28:22, 4.21s/it]
90%|████████▉ | 3597/4000 [4:13:21<28:15, 4.21s/it]
90%|████████▉ | 3598/4000 [4:13:26<28:48, 4.30s/it]
90%|████████▉ | 3599/4000 [4:13:30<28:23, 4.25s/it]
90%|█████████ | 3600/4000 [4:13:34<28:04, 4.21s/it]
{'loss': '0.4017', 'grad_norm': '0.1289', 'learning_rate': '6.076e-05', 'epoch': '0.8723'}
90%|█████████ | 3600/4000 [4:13:34<28:04, 4.21s/it]
90%|█████████ | 3601/4000 [4:13:38<27:49, 4.18s/it]
90%|█████████ | 3602/4000 [4:13:42<27:38, 4.17s/it]
90%|█████████ | 3603/4000 [4:13:47<27:36, 4.17s/it]
90%|█████████ | 3604/4000 [4:13:51<27:31, 4.17s/it]
90%|█████████ | 3605/4000 [4:13:56<28:26, 4.32s/it]
90%|█████████ | 3606/4000 [4:14:00<28:29, 4.34s/it]
90%|█████████ | 3607/4000 [4:14:04<28:05, 4.29s/it]
90%|█████████ | 3608/4000 [4:14:08<28:16, 4.33s/it]
90%|█████████ | 3609/4000 [4:14:13<27:50, 4.27s/it]
90%|█████████ | 3610/4000 [4:14:17<27:46, 4.27s/it]
{'loss': '0.367', 'grad_norm': '0.1241', 'learning_rate': '5.924e-05', 'epoch': '0.8747'}
90%|█████████ | 3610/4000 [4:14:17<27:46, 4.27s/it]
90%|█████████ | 3611/4000 [4:14:21<28:10, 4.35s/it]
90%|█████████ | 3612/4000 [4:14:26<27:41, 4.28s/it]
90%|█████████ | 3613/4000 [4:14:30<27:18, 4.23s/it]
90%|█████████ | 3614/4000 [4:14:34<27:11, 4.23s/it]
90%|█████████ | 3615/4000 [4:14:38<26:59, 4.21s/it]
90%|█████████ | 3616/4000 [4:14:42<26:46, 4.18s/it]
90%|█████████ | 3617/4000 [4:14:46<26:35, 4.17s/it]
90%|█████████ | 3618/4000 [4:14:51<27:10, 4.27s/it]
90%|█████████ | 3619/4000 [4:14:55<26:51, 4.23s/it]
90%|█████████ | 3620/4000 [4:14:59<26:56, 4.25s/it]
{'loss': '0.3656', 'grad_norm': '0.1169', 'learning_rate': '5.773e-05', 'epoch': '0.8771'}
90%|█████████ | 3620/4000 [4:14:59<26:56, 4.25s/it]
91%|█████████ | 3621/4000 [4:15:03<26:45, 4.24s/it]
91%|█████████ | 3622/4000 [4:15:08<26:56, 4.28s/it]
91%|█████████ | 3623/4000 [4:15:12<27:07, 4.32s/it]
91%|█████████ | 3624/4000 [4:15:16<26:43, 4.26s/it]
91%|█████████ | 3625/4000 [4:15:21<27:21, 4.38s/it]
91%|█████████ | 3626/4000 [4:15:25<26:50, 4.31s/it]
91%|█████████ | 3627/4000 [4:15:29<26:41, 4.29s/it]
91%|█████████ | 3628/4000 [4:15:34<26:22, 4.25s/it]
91%|█████████ | 3629/4000 [4:15:38<26:05, 4.22s/it]
91%|█████████ | 3630/4000 [4:15:42<25:52, 4.19s/it]
{'loss': '0.3462', 'grad_norm': '0.1212', 'learning_rate': '5.621e-05', 'epoch': '0.8795'}
91%|█████████ | 3630/4000 [4:15:42<25:52, 4.19s/it]
91%|█████████ | 3631/4000 [4:15:46<25:43, 4.18s/it]
91%|█████████ | 3632/4000 [4:15:51<26:23, 4.30s/it]
91%|█████████ | 3633/4000 [4:15:55<26:00, 4.25s/it]
91%|█████████ | 3634/4000 [4:15:59<25:43, 4.22s/it]
91%|█████████ | 3635/4000 [4:16:03<25:28, 4.19s/it]
91%|█████████ | 3636/4000 [4:16:07<25:17, 4.17s/it]
91%|█████████ | 3637/4000 [4:16:11<25:18, 4.18s/it]
91%|█████████ | 3638/4000 [4:16:16<26:02, 4.32s/it]
91%|█████████ | 3639/4000 [4:16:20<26:00, 4.32s/it]
91%|█████████ | 3640/4000 [4:16:25<26:03, 4.34s/it]
{'loss': '0.3344', 'grad_norm': '0.1189', 'learning_rate': '5.47e-05', 'epoch': '0.882'}
91%|█████████ | 3640/4000 [4:16:25<26:03, 4.34s/it]
91%|█████████ | 3641/4000 [4:16:29<25:37, 4.28s/it]
91%|█████████ | 3642/4000 [4:16:33<25:24, 4.26s/it]
91%|█████████ | 3643/4000 [4:16:37<25:07, 4.22s/it]
91%|█████████ | 3644/4000 [4:16:41<25:02, 4.22s/it]
91%|█████████ | 3645/4000 [4:16:46<25:26, 4.30s/it]
91%|█████████ | 3646/4000 [4:16:50<25:02, 4.24s/it]
91%|█████████ | 3647/4000 [4:16:54<24:44, 4.21s/it]
91%|█████████ | 3648/4000 [4:16:58<24:38, 4.20s/it]
91%|█████████ | 3649/4000 [4:17:02<24:32, 4.19s/it]
91%|█████████▏| 3650/4000 [4:17:07<24:19, 4.17s/it]
{'loss': '0.3955', 'grad_norm': '0.1442', 'learning_rate': '5.318e-05', 'epoch': '0.8844'}
91%|█████████▏| 3650/4000 [4:17:07<24:19, 4.17s/it]
91%|█████████▏| 3651/4000 [4:17:11<24:10, 4.16s/it]
91%|█████████▏| 3652/4000 [4:17:15<24:39, 4.25s/it]
91%|█████████▏| 3653/4000 [4:17:19<24:21, 4.21s/it]
91%|█████████▏| 3654/4000 [4:17:24<24:16, 4.21s/it]
91%|█████████▏| 3655/4000 [4:17:28<24:23, 4.24s/it]
91%|█████████▏| 3656/4000 [4:17:32<24:19, 4.24s/it]
91%|█████████▏| 3657/4000 [4:17:36<24:21, 4.26s/it]
91%|█████████▏| 3658/4000 [4:17:41<24:25, 4.28s/it]
91%|█████████▏| 3659/4000 [4:17:45<24:32, 4.32s/it]
92%|█████████▏| 3660/4000 [4:17:49<24:08, 4.26s/it]
{'loss': '0.3687', 'grad_norm': '0.1402', 'learning_rate': '5.167e-05', 'epoch': '0.8868'}
92%|█████████▏| 3660/4000 [4:17:49<24:08, 4.26s/it]
92%|█████████▏| 3661/4000 [4:17:53<24:02, 4.26s/it]
92%|█████████▏| 3662/4000 [4:17:58<23:47, 4.22s/it]
92%|█████████▏| 3663/4000 [4:18:02<23:32, 4.19s/it]
92%|█████████▏| 3664/4000 [4:18:06<23:22, 4.17s/it]
92%|█████████▏| 3665/4000 [4:18:10<23:47, 4.26s/it]
92%|█████████▏| 3666/4000 [4:18:15<23:34, 4.23s/it]
92%|█████████▏| 3667/4000 [4:18:19<23:18, 4.20s/it]
92%|█████████▏| 3668/4000 [4:18:23<23:06, 4.18s/it]
92%|█████████▏| 3669/4000 [4:18:27<22:56, 4.16s/it]
92%|█████████▏| 3670/4000 [4:18:31<22:48, 4.15s/it]
{'loss': '0.363', 'grad_norm': '0.1105', 'learning_rate': '5.015e-05', 'epoch': '0.8892'}
92%|█████████▏| 3670/4000 [4:18:31<22:48, 4.15s/it]
92%|█████████▏| 3671/4000 [4:18:35<22:41, 4.14s/it]
92%|█████████▏| 3672/4000 [4:18:40<23:35, 4.32s/it]
92%|█████████▏| 3673/4000 [4:18:44<23:26, 4.30s/it]
92%|█████████▏| 3674/4000 [4:18:48<23:17, 4.29s/it]
92%|█████████▏| 3675/4000 [4:18:53<23:07, 4.27s/it]
92%|█████████▏| 3676/4000 [4:18:57<22:56, 4.25s/it]
92%|█████████▏| 3677/4000 [4:19:01<22:42, 4.22s/it]
92%|█████████▏| 3678/4000 [4:19:05<22:38, 4.22s/it]
92%|█████████▏| 3679/4000 [4:19:10<23:11, 4.33s/it]
92%|█████████▏| 3680/4000 [4:19:14<22:47, 4.27s/it]
{'loss': '0.3841', 'grad_norm': '0.1173', 'learning_rate': '4.864e-05', 'epoch': '0.8917'}
92%|█████████▏| 3680/4000 [4:19:14<22:47, 4.27s/it]
92%|█████████▏| 3681/4000 [4:19:18<22:28, 4.23s/it]
92%|█████████▏| 3682/4000 [4:19:22<22:15, 4.20s/it]
92%|█████████▏| 3683/4000 [4:19:26<22:10, 4.20s/it]
92%|█████████▏| 3684/4000 [4:19:30<22:00, 4.18s/it]
92%|█████████▏| 3685/4000 [4:19:35<22:08, 4.22s/it]
92%|█████████▏| 3686/4000 [4:19:39<22:09, 4.24s/it]
92%|█████████▏| 3687/4000 [4:19:43<21:55, 4.20s/it]
92%|█████████▏| 3688/4000 [4:19:47<21:43, 4.18s/it]
92%|█████████▏| 3689/4000 [4:19:52<21:51, 4.22s/it]
92%|█████████▏| 3690/4000 [4:19:56<21:45, 4.21s/it]
{'loss': '0.3604', 'grad_norm': '0.1114', 'learning_rate': '4.712e-05', 'epoch': '0.8941'}
92%|█████████▏| 3690/4000 [4:19:56<21:45, 4.21s/it]
92%|█████████▏| 3691/4000 [4:20:00<21:34, 4.19s/it]
92%|█████████▏| 3692/4000 [4:20:04<22:01, 4.29s/it]
92%|█████████▏| 3693/4000 [4:20:09<21:45, 4.25s/it]
92%|█████████▏| 3694/4000 [4:20:13<21:35, 4.23s/it]
92%|█████████▏| 3695/4000 [4:20:17<21:21, 4.20s/it]
92%|█████████▏| 3696/4000 [4:20:21<21:20, 4.21s/it]
92%|█████████▏| 3697/4000 [4:20:25<21:09, 4.19s/it]
92%|█████████▏| 3698/4000 [4:20:29<20:58, 4.17s/it]
92%|█████████▏| 3699/4000 [4:20:34<21:22, 4.26s/it]
92%|█████████▎| 3700/4000 [4:20:38<21:16, 4.26s/it]
{'loss': '0.3999', 'grad_norm': '0.138', 'learning_rate': '4.561e-05', 'epoch': '0.8965'}
92%|█████████▎| 3700/4000 [4:20:38<21:16, 4.26s/it]
93%|█████████▎| 3701/4000 [4:20:42<21:01, 4.22s/it]
93%|█████████▎| 3702/4000 [4:20:46<20:48, 4.19s/it]
93%|█████████▎| 3703/4000 [4:20:51<20:38, 4.17s/it]
93%|█████████▎| 3704/4000 [4:20:55<20:29, 4.15s/it]
93%|█████████▎| 3705/4000 [4:20:59<20:23, 4.15s/it]
93%|█████████▎| 3706/4000 [4:21:04<21:15, 4.34s/it]
93%|█████████▎| 3707/4000 [4:21:08<21:02, 4.31s/it]
93%|█████████▎| 3708/4000 [4:21:12<20:47, 4.27s/it]
93%|█████████▎| 3709/4000 [4:21:16<20:45, 4.28s/it]
93%|█████████▎| 3710/4000 [4:21:20<20:29, 4.24s/it]
{'loss': '0.361', 'grad_norm': '0.1257', 'learning_rate': '4.409e-05', 'epoch': '0.8989'}
93%|█████████▎| 3710/4000 [4:21:20<20:29, 4.24s/it]
93%|█████████▎| 3711/4000 [4:21:25<20:26, 4.24s/it]
93%|█████████▎| 3712/4000 [4:21:29<20:28, 4.27s/it]
93%|█████████▎| 3713/4000 [4:21:34<20:55, 4.38s/it]
93%|█████████▎| 3714/4000 [4:21:38<20:31, 4.31s/it]
93%|█████████▎| 3715/4000 [4:21:42<20:10, 4.25s/it]
93%|█████████▎| 3716/4000 [4:21:46<19:55, 4.21s/it]
93%|█████████▎| 3717/4000 [4:21:50<19:49, 4.20s/it]
93%|█████████▎| 3718/4000 [4:21:54<19:39, 4.18s/it]
93%|█████████▎| 3719/4000 [4:21:59<20:01, 4.28s/it]
93%|█████████▎| 3720/4000 [4:22:03<19:44, 4.23s/it]
{'loss': '0.3827', 'grad_norm': '0.1181', 'learning_rate': '4.258e-05', 'epoch': '0.9014'}
93%|█████████▎| 3720/4000 [4:22:03<19:44, 4.23s/it]
93%|█████████▎| 3721/4000 [4:22:07<19:31, 4.20s/it]
93%|█████████▎| 3722/4000 [4:22:11<19:21, 4.18s/it]
93%|█████████▎| 3723/4000 [4:22:16<19:35, 4.24s/it]
93%|█████████▎| 3724/4000 [4:22:20<19:41, 4.28s/it]
93%|█████████▎| 3725/4000 [4:22:24<19:27, 4.25s/it]
93%|█████████▎| 3726/4000 [4:22:29<19:53, 4.36s/it]
93%|█████████▎| 3727/4000 [4:22:33<19:34, 4.30s/it]
93%|█████████▎| 3728/4000 [4:22:37<19:30, 4.30s/it]
93%|█████████▎| 3729/4000 [4:22:41<19:11, 4.25s/it]
93%|█████████▎| 3730/4000 [4:22:46<19:21, 4.30s/it]
{'loss': '0.4119', 'grad_norm': '0.1231', 'learning_rate': '4.106e-05', 'epoch': '0.9038'}
93%|█████████▎| 3730/4000 [4:22:46<19:21, 4.30s/it]
93%|█████████▎| 3731/4000 [4:22:50<19:03, 4.25s/it]
93%|█████████▎| 3732/4000 [4:22:54<18:50, 4.22s/it]
93%|█████████▎| 3733/4000 [4:22:59<19:06, 4.29s/it]
93%|█████████▎| 3734/4000 [4:23:03<18:53, 4.26s/it]
93%|█████████▎| 3735/4000 [4:23:07<18:39, 4.23s/it]
93%|█████████▎| 3736/4000 [4:23:11<18:27, 4.20s/it]
93%|█████████▎| 3737/4000 [4:23:15<18:17, 4.17s/it]
93%|█████████▎| 3738/4000 [4:23:19<18:10, 4.16s/it]
93%|█████████▎| 3739/4000 [4:23:24<18:32, 4.26s/it]
94%|█████████▎| 3740/4000 [4:23:28<18:35, 4.29s/it]
{'loss': '0.3433', 'grad_norm': '0.1236', 'learning_rate': '3.955e-05', 'epoch': '0.9062'}
94%|█████████▎| 3740/4000 [4:23:28<18:35, 4.29s/it]
94%|█████████▎| 3741/4000 [4:23:32<18:35, 4.31s/it]
94%|█████████▎| 3742/4000 [4:23:37<18:18, 4.26s/it]
94%|█████████▎| 3743/4000 [4:23:41<18:15, 4.26s/it]
94%|█████████▎| 3744/4000 [4:23:45<18:01, 4.22s/it]
94%|█████████▎| 3745/4000 [4:23:49<18:04, 4.25s/it]
94%|█████████▎| 3746/4000 [4:23:54<18:17, 4.32s/it]
94%|█████████▎| 3747/4000 [4:23:58<18:23, 4.36s/it]
94%|█████████▎| 3748/4000 [4:24:02<18:04, 4.31s/it]
94%|█████████▎| 3749/4000 [4:24:07<17:47, 4.25s/it]
94%|█████████▍| 3750/4000 [4:24:11<17:34, 4.22s/it]
{'loss': '0.3984', 'grad_norm': '0.1077', 'learning_rate': '3.803e-05', 'epoch': '0.9086'}
94%|█████████▍| 3750/4000 [4:24:11<17:34, 4.22s/it]
94%|█████████▍| 3751/4000 [4:24:15<17:30, 4.22s/it]
94%|█████████▍| 3752/4000 [4:24:19<17:24, 4.21s/it]
94%|█████████▍| 3753/4000 [4:24:24<17:39, 4.29s/it]
94%|█████████▍| 3754/4000 [4:24:28<17:24, 4.25s/it]
94%|█████████▍| 3755/4000 [4:24:32<17:10, 4.21s/it]
94%|█████████▍| 3756/4000 [4:24:36<17:00, 4.18s/it]
94%|█████████▍| 3757/4000 [4:24:40<17:14, 4.26s/it]
94%|█████████▍| 3758/4000 [4:24:45<17:18, 4.29s/it]
94%|█████████▍| 3759/4000 [4:24:49<17:03, 4.24s/it]
94%|█████████▍| 3760/4000 [4:24:54<17:30, 4.38s/it]
{'loss': '0.3751', 'grad_norm': '0.1149', 'learning_rate': '3.652e-05', 'epoch': '0.911'}
94%|█████████▍| 3760/4000 [4:24:54<17:30, 4.38s/it]
94%|█████████▍| 3761/4000 [4:24:58<17:12, 4.32s/it]
94%|█████████▍| 3762/4000 [4:25:02<17:03, 4.30s/it]
94%|█████████▍| 3763/4000 [4:25:06<16:48, 4.25s/it]
94%|█████████▍| 3764/4000 [4:25:10<16:47, 4.27s/it]
94%|█████████▍| 3765/4000 [4:25:15<16:34, 4.23s/it]
94%|█████████▍| 3766/4000 [4:25:19<16:36, 4.26s/it]
94%|█████████▍| 3767/4000 [4:25:23<16:34, 4.27s/it]
94%|█████████▍| 3768/4000 [4:25:27<16:22, 4.24s/it]
94%|█████████▍| 3769/4000 [4:25:32<16:13, 4.21s/it]
94%|█████████▍| 3770/4000 [4:25:36<16:03, 4.19s/it]
{'loss': '0.34', 'grad_norm': '0.1119', 'learning_rate': '3.5e-05', 'epoch': '0.9135'}
94%|█████████▍| 3770/4000 [4:25:36<16:03, 4.19s/it]
94%|█████████▍| 3771/4000 [4:25:40<15:54, 4.17s/it]
94%|█████████▍| 3772/4000 [4:25:44<15:46, 4.15s/it]
94%|█████████▍| 3773/4000 [4:25:48<16:04, 4.25s/it]
94%|█████████▍| 3774/4000 [4:25:53<16:09, 4.29s/it]
94%|█████████▍| 3775/4000 [4:25:57<16:07, 4.30s/it]
94%|█████████▍| 3776/4000 [4:26:01<15:52, 4.25s/it]
94%|█████████▍| 3777/4000 [4:26:06<15:58, 4.30s/it]
94%|█████████▍| 3778/4000 [4:26:10<15:44, 4.25s/it]
94%|█████████▍| 3779/4000 [4:26:14<15:51, 4.30s/it]
94%|█████████▍| 3780/4000 [4:26:19<15:58, 4.36s/it]
{'loss': '0.3786', 'grad_norm': '0.122', 'learning_rate': '3.348e-05', 'epoch': '0.9159'}
94%|█████████▍| 3780/4000 [4:26:19<15:58, 4.36s/it]
95%|█████████▍| 3781/4000 [4:26:23<15:53, 4.36s/it]
95%|█████████▍| 3782/4000 [4:26:27<15:35, 4.29s/it]
95%|█████████▍| 3783/4000 [4:26:31<15:20, 4.24s/it]
95%|█████████▍| 3784/4000 [4:26:35<15:08, 4.21s/it]
95%|█████████▍| 3785/4000 [4:26:40<15:01, 4.19s/it]
95%|█████████▍| 3786/4000 [4:26:44<15:08, 4.25s/it]
95%|█████████▍| 3787/4000 [4:26:48<15:07, 4.26s/it]
95%|█████████▍| 3788/4000 [4:26:52<14:53, 4.22s/it]
95%|█████████▍| 3789/4000 [4:26:56<14:43, 4.19s/it]
95%|█████████▍| 3790/4000 [4:27:01<14:36, 4.17s/it]
{'loss': '0.3527', 'grad_norm': '0.1165', 'learning_rate': '3.197e-05', 'epoch': '0.9183'}
95%|█████████▍| 3790/4000 [4:27:01<14:36, 4.17s/it]
95%|█████████▍| 3791/4000 [4:27:05<14:36, 4.20s/it]
95%|█████████▍| 3792/4000 [4:27:09<14:32, 4.19s/it]
95%|█████████▍| 3793/4000 [4:27:14<14:46, 4.28s/it]
95%|█████████▍| 3794/4000 [4:27:18<14:41, 4.28s/it]
95%|█████████▍| 3795/4000 [4:27:22<14:28, 4.24s/it]
95%|█████████▍| 3796/4000 [4:27:26<14:36, 4.30s/it]
95%|█████████▍| 3797/4000 [4:27:31<14:21, 4.25s/it]
95%|█████████▍| 3798/4000 [4:27:35<14:18, 4.25s/it]
95%|█████████▍| 3799/4000 [4:27:39<14:13, 4.24s/it]
95%|█████████▌| 3800/4000 [4:27:44<14:23, 4.32s/it]
{'loss': '0.3989', 'grad_norm': '0.1184', 'learning_rate': '3.045e-05', 'epoch': '0.9207'}
95%|█████████▌| 3800/4000 [4:27:44<14:23, 4.32s/it]
95%|█████████▌| 3801/4000 [4:27:48<14:07, 4.26s/it]
95%|█████████▌| 3802/4000 [4:27:52<13:56, 4.22s/it]
95%|█████████▌| 3803/4000 [4:27:56<13:49, 4.21s/it]
95%|█████████▌| 3804/4000 [4:28:00<13:40, 4.18s/it]
95%|█████████▌| 3805/4000 [4:28:04<13:32, 4.17s/it]
95%|█████████▌| 3806/4000 [4:28:08<13:26, 4.16s/it]
95%|█████████▌| 3807/4000 [4:28:13<13:41, 4.25s/it]
95%|█████████▌| 3808/4000 [4:28:17<13:44, 4.30s/it]
95%|█████████▌| 3809/4000 [4:28:21<13:35, 4.27s/it]
95%|█████████▌| 3810/4000 [4:28:26<13:23, 4.23s/it]
{'loss': '0.3577', 'grad_norm': '0.1155', 'learning_rate': '2.894e-05', 'epoch': '0.9232'}
95%|█████████▌| 3810/4000 [4:28:26<13:23, 4.23s/it]
95%|█████████▌| 3811/4000 [4:28:30<13:29, 4.28s/it]
95%|█████████▌| 3812/4000 [4:28:34<13:18, 4.25s/it]
95%|█████████▌| 3813/4000 [4:28:39<13:35, 4.36s/it]
95%|█████████▌| 3814/4000 [4:28:43<13:27, 4.34s/it]
95%|█████████▌| 3815/4000 [4:28:47<13:18, 4.31s/it]
95%|█████████▌| 3816/4000 [4:28:51<13:07, 4.28s/it]
95%|█████████▌| 3817/4000 [4:28:56<12:54, 4.23s/it]
95%|█████████▌| 3818/4000 [4:29:00<12:44, 4.20s/it]
95%|█████████▌| 3819/4000 [4:29:04<12:35, 4.18s/it]
96%|█████████▌| 3820/4000 [4:29:08<12:53, 4.30s/it]
{'loss': '0.3699', 'grad_norm': '0.141', 'learning_rate': '2.742e-05', 'epoch': '0.9256'}
96%|█████████▌| 3820/4000 [4:29:08<12:53, 4.30s/it]
96%|█████████▌| 3821/4000 [4:29:13<12:41, 4.25s/it]
96%|█████████▌| 3822/4000 [4:29:17<12:30, 4.22s/it]
96%|█████████▌| 3823/4000 [4:29:21<12:21, 4.19s/it]
96%|█████████▌| 3824/4000 [4:29:25<12:13, 4.17s/it]
96%|█████████▌| 3825/4000 [4:29:29<12:12, 4.19s/it]
96%|█████████▌| 3826/4000 [4:29:33<12:08, 4.18s/it]
96%|█████████▌| 3827/4000 [4:29:38<12:18, 4.27s/it]
96%|█████████▌| 3828/4000 [4:29:42<12:23, 4.32s/it]
96%|█████████▌| 3829/4000 [4:29:46<12:10, 4.27s/it]
96%|█████████▌| 3830/4000 [4:29:51<12:03, 4.26s/it]
{'loss': '0.3517', 'grad_norm': '0.133', 'learning_rate': '2.591e-05', 'epoch': '0.928'}
96%|█████████▌| 3830/4000 [4:29:51<12:03, 4.26s/it]
96%|█████████▌| 3831/4000 [4:29:55<11:53, 4.22s/it]
96%|█████████▌| 3832/4000 [4:29:59<11:45, 4.20s/it]
96%|█████████▌| 3833/4000 [4:30:03<11:51, 4.26s/it]
96%|█████████▌| 3834/4000 [4:30:08<11:58, 4.33s/it]
96%|█████████▌| 3835/4000 [4:30:12<11:43, 4.26s/it]
96%|█████████▌| 3836/4000 [4:30:16<11:34, 4.24s/it]
96%|█████████▌| 3837/4000 [4:30:20<11:28, 4.22s/it]
96%|█████████▌| 3838/4000 [4:30:24<11:19, 4.19s/it]
96%|█████████▌| 3839/4000 [4:30:29<11:12, 4.17s/it]
96%|█████████▌| 3840/4000 [4:30:33<11:14, 4.21s/it]
{'loss': '0.3491', 'grad_norm': '0.128', 'learning_rate': '2.439e-05', 'epoch': '0.9304'}
96%|█████████▌| 3840/4000 [4:30:33<11:14, 4.21s/it]
96%|█████████▌| 3841/4000 [4:30:37<11:14, 4.24s/it]
96%|█████████▌| 3842/4000 [4:30:42<11:19, 4.30s/it]
96%|█████████▌| 3843/4000 [4:30:46<11:11, 4.28s/it]
96%|█████████▌| 3844/4000 [4:30:50<11:02, 4.25s/it]
96%|█████████▌| 3845/4000 [4:30:54<11:04, 4.29s/it]
96%|█████████▌| 3846/4000 [4:30:59<10:54, 4.25s/it]
96%|█████████▌| 3847/4000 [4:31:03<11:08, 4.37s/it]
96%|█████████▌| 3848/4000 [4:31:07<10:54, 4.31s/it]
96%|█████████▌| 3849/4000 [4:31:12<10:43, 4.26s/it]
96%|█████████▋| 3850/4000 [4:31:16<10:40, 4.27s/it]
{'loss': '0.3858', 'grad_norm': '0.1295', 'learning_rate': '2.288e-05', 'epoch': '0.9329'}
96%|█████████▋| 3850/4000 [4:31:16<10:40, 4.27s/it]
96%|█████████▋| 3851/4000 [4:31:20<10:29, 4.23s/it]
96%|█████████▋| 3852/4000 [4:31:24<10:21, 4.20s/it]
96%|█████████▋| 3853/4000 [4:31:28<10:14, 4.18s/it]
96%|█████████▋| 3854/4000 [4:31:33<10:34, 4.35s/it]
96%|█████████▋| 3855/4000 [4:31:37<10:20, 4.28s/it]
96%|█████████▋| 3856/4000 [4:31:41<10:09, 4.24s/it]
96%|█████████▋| 3857/4000 [4:31:45<10:01, 4.21s/it]
96%|█████████▋| 3858/4000 [4:31:49<09:53, 4.18s/it]
96%|█████████▋| 3859/4000 [4:31:54<09:55, 4.22s/it]
96%|█████████▋| 3860/4000 [4:31:58<09:49, 4.21s/it]
{'loss': '0.3705', 'grad_norm': '0.1089', 'learning_rate': '2.136e-05', 'epoch': '0.9353'}
96%|█████████▋| 3860/4000 [4:31:58<09:49, 4.21s/it]
97%|█████████▋| 3861/4000 [4:32:03<09:59, 4.31s/it]
97%|█████████▋| 3862/4000 [4:32:07<10:00, 4.35s/it]
97%|█████████▋| 3863/4000 [4:32:11<09:48, 4.29s/it]
97%|█████████▋| 3864/4000 [4:32:15<09:42, 4.28s/it]
97%|█████████▋| 3865/4000 [4:32:20<09:32, 4.24s/it]
97%|█████████▋| 3866/4000 [4:32:24<09:24, 4.22s/it]
97%|█████████▋| 3867/4000 [4:32:28<09:37, 4.34s/it]
97%|█████████▋| 3868/4000 [4:32:33<09:30, 4.33s/it]
97%|█████████▋| 3869/4000 [4:32:37<09:19, 4.27s/it]
97%|█████████▋| 3870/4000 [4:32:41<09:09, 4.23s/it]
{'loss': '0.3926', 'grad_norm': '0.1193', 'learning_rate': '1.985e-05', 'epoch': '0.9377'}
97%|█████████▋| 3870/4000 [4:32:41<09:09, 4.23s/it]
97%|█████████▋| 3871/4000 [4:32:45<09:05, 4.23s/it]
97%|█████████▋| 3872/4000 [4:32:49<08:57, 4.20s/it]
97%|█████████▋| 3873/4000 [4:32:53<08:50, 4.18s/it]
97%|█████████▋| 3874/4000 [4:32:58<08:58, 4.27s/it]
97%|█████████▋| 3875/4000 [4:33:02<08:48, 4.23s/it]
97%|█████████▋| 3876/4000 [4:33:06<08:45, 4.24s/it]
97%|█████████▋| 3877/4000 [4:33:10<08:39, 4.22s/it]
97%|█████████▋| 3878/4000 [4:33:15<08:35, 4.23s/it]
97%|█████████▋| 3879/4000 [4:33:19<08:36, 4.27s/it]
97%|█████████▋| 3880/4000 [4:33:23<08:28, 4.24s/it]
{'loss': '0.364', 'grad_norm': '0.1004', 'learning_rate': '1.833e-05', 'epoch': '0.9401'}
97%|█████████▋| 3880/4000 [4:33:23<08:28, 4.24s/it]
97%|█████████▋| 3881/4000 [4:33:28<08:40, 4.38s/it]
97%|█████████▋| 3882/4000 [4:33:32<08:28, 4.31s/it]
97%|█████████▋| 3883/4000 [4:33:36<08:21, 4.29s/it]
97%|█████████▋| 3884/4000 [4:33:41<08:18, 4.29s/it]
97%|█████████▋| 3885/4000 [4:33:45<08:12, 4.28s/it]
97%|█████████▋| 3886/4000 [4:33:49<08:02, 4.23s/it]
97%|█████████▋| 3887/4000 [4:33:53<07:54, 4.20s/it]
97%|█████████▋| 3888/4000 [4:33:58<08:04, 4.32s/it]
97%|█████████▋| 3889/4000 [4:34:02<07:53, 4.27s/it]
97%|█████████▋| 3890/4000 [4:34:06<07:44, 4.23s/it]
{'loss': '0.3782', 'grad_norm': '0.1145', 'learning_rate': '1.682e-05', 'epoch': '0.9425'}
97%|█████████▋| 3890/4000 [4:34:06<07:44, 4.23s/it]
97%|█████████▋| 3891/4000 [4:34:10<07:37, 4.20s/it]
97%|█████████▋| 3892/4000 [4:34:14<07:31, 4.18s/it]
97%|█████████▋| 3893/4000 [4:34:19<07:32, 4.23s/it]
97%|█████████▋| 3894/4000 [4:34:23<07:41, 4.35s/it]
97%|█████████▋| 3895/4000 [4:34:27<07:31, 4.30s/it]
97%|█████████▋| 3896/4000 [4:34:32<07:29, 4.32s/it]
97%|█████████▋| 3897/4000 [4:34:36<07:19, 4.26s/it]
97%|█████████▋| 3898/4000 [4:34:40<07:18, 4.30s/it]
97%|█████████▋| 3899/4000 [4:34:44<07:09, 4.25s/it]
98%|█████████▊| 3900/4000 [4:34:49<07:04, 4.24s/it]
{'loss': '0.3661', 'grad_norm': '0.1121', 'learning_rate': '1.53e-05', 'epoch': '0.945'}
98%|█████████▊| 3900/4000 [4:34:49<07:04, 4.24s/it]
98%|█████████▊| 3901/4000 [4:34:53<07:16, 4.41s/it]
98%|█████████▊| 3902/4000 [4:34:58<07:04, 4.33s/it]
98%|█████████▊| 3903/4000 [4:35:02<06:53, 4.27s/it]
98%|█████████▊| 3904/4000 [4:35:06<06:48, 4.26s/it]
98%|█████████▊| 3905/4000 [4:35:10<06:40, 4.21s/it]
98%|█████████▊| 3906/4000 [4:35:14<06:32, 4.18s/it]
98%|█████████▊| 3907/4000 [4:35:18<06:29, 4.19s/it]
98%|█████████▊| 3908/4000 [4:35:23<06:34, 4.28s/it]
98%|█████████▊| 3909/4000 [4:35:27<06:25, 4.23s/it]
98%|█████████▊| 3910/4000 [4:35:31<06:17, 4.19s/it]
{'loss': '0.3507', 'grad_norm': '0.1011', 'learning_rate': '1.379e-05', 'epoch': '0.9474'}
98%|█████████▊| 3910/4000 [4:35:31<06:17, 4.19s/it]
98%|█████████▊| 3911/4000 [4:35:35<06:10, 4.16s/it]
98%|█████████▊| 3912/4000 [4:35:39<06:04, 4.14s/it]
98%|█████████▊| 3913/4000 [4:35:43<05:59, 4.13s/it]
98%|█████████▊| 3914/4000 [4:35:48<05:59, 4.18s/it]
98%|█████████▊| 3915/4000 [4:35:52<05:57, 4.20s/it]
98%|█████████▊| 3916/4000 [4:35:56<05:50, 4.17s/it]
98%|█████████▊| 3917/4000 [4:36:00<05:44, 4.15s/it]
98%|█████████▊| 3918/4000 [4:36:04<05:38, 4.13s/it]
98%|█████████▊| 3919/4000 [4:36:08<05:33, 4.12s/it]
98%|█████████▊| 3920/4000 [4:36:12<05:29, 4.12s/it]
{'loss': '0.3492', 'grad_norm': '0.1289', 'learning_rate': '1.227e-05', 'epoch': '0.9498'}
98%|█████████▊| 3920/4000 [4:36:12<05:29, 4.12s/it]
98%|█████████▊| 3921/4000 [4:36:17<05:33, 4.22s/it]
98%|█████████▊| 3922/4000 [4:36:21<05:26, 4.18s/it]
98%|█████████▊| 3923/4000 [4:36:25<05:20, 4.16s/it]
98%|█████████▊| 3924/4000 [4:36:29<05:14, 4.14s/it]
98%|█████████▊| 3925/4000 [4:36:33<05:09, 4.13s/it]
98%|█████████▊| 3926/4000 [4:36:37<05:04, 4.12s/it]
98%|█████████▊| 3927/4000 [4:36:41<05:00, 4.11s/it]
98%|█████████▊| 3928/4000 [4:36:46<05:02, 4.20s/it]
98%|█████████▊| 3929/4000 [4:36:50<04:57, 4.19s/it]
98%|█████████▊| 3930/4000 [4:36:54<04:52, 4.18s/it]
{'loss': '0.3655', 'grad_norm': '0.1154', 'learning_rate': '1.076e-05', 'epoch': '0.9522'}
98%|█████████▊| 3930/4000 [4:36:54<04:52, 4.18s/it]
98%|█████████▊| 3931/4000 [4:36:58<04:46, 4.16s/it]
98%|█████████▊| 3932/4000 [4:37:02<04:41, 4.14s/it]
98%|█████████▊| 3933/4000 [4:37:07<04:36, 4.13s/it]
98%|█████████▊| 3934/4000 [4:37:11<04:32, 4.13s/it]
98%|█████████▊| 3935/4000 [4:37:15<04:34, 4.22s/it]
98%|█████████▊| 3936/4000 [4:37:19<04:27, 4.19s/it]
98%|█████████▊| 3937/4000 [4:37:23<04:22, 4.16s/it]
98%|█████████▊| 3938/4000 [4:37:27<04:17, 4.15s/it]
98%|█████████▊| 3939/4000 [4:37:32<04:12, 4.14s/it]
98%|█████████▊| 3940/4000 [4:37:36<04:07, 4.13s/it]
{'loss': '0.353', 'grad_norm': '0.1089', 'learning_rate': '9.242e-06', 'epoch': '0.9547'}
98%|█████████▊| 3940/4000 [4:37:36<04:07, 4.13s/it]
99%|█████████▊| 3941/4000 [4:37:40<04:06, 4.18s/it]
99%|█████████▊| 3942/4000 [4:37:44<04:03, 4.20s/it]
99%|█████████▊| 3943/4000 [4:37:48<03:57, 4.17s/it]
99%|█████████▊| 3944/4000 [4:37:52<03:52, 4.15s/it]
99%|█████████▊| 3945/4000 [4:37:56<03:47, 4.13s/it]
99%|█████████▊| 3946/4000 [4:38:01<03:42, 4.12s/it]
99%|█████████▊| 3947/4000 [4:38:05<03:37, 4.11s/it]
99%|█████████▊| 3948/4000 [4:38:09<03:38, 4.21s/it]
99%|█████████▊| 3949/4000 [4:38:13<03:32, 4.18s/it]
99%|█████████▉| 3950/4000 [4:38:17<03:27, 4.15s/it]
{'loss': '0.3895', 'grad_norm': '0.132', 'learning_rate': '7.727e-06', 'epoch': '0.9571'}
99%|█████████▉| 3950/4000 [4:38:17<03:27, 4.15s/it]
99%|█████████▉| 3951/4000 [4:38:21<03:22, 4.13s/it]
99%|█████████▉| 3952/4000 [4:38:25<03:17, 4.12s/it]
99%|█████████▉| 3953/4000 [4:38:30<03:13, 4.11s/it]
99%|█████████▉| 3954/4000 [4:38:34<03:08, 4.10s/it]
99%|█████████▉| 3955/4000 [4:38:38<03:09, 4.20s/it]
99%|█████████▉| 3956/4000 [4:38:42<03:03, 4.17s/it]
99%|█████████▉| 3957/4000 [4:38:46<02:58, 4.15s/it]
99%|█████████▉| 3958/4000 [4:38:50<02:53, 4.14s/it]
99%|█████████▉| 3959/4000 [4:38:54<02:48, 4.12s/it]
99%|█████████▉| 3960/4000 [4:38:59<02:44, 4.11s/it]
{'loss': '0.4014', 'grad_norm': '0.1627', 'learning_rate': '6.212e-06', 'epoch': '0.9595'}
99%|█████████▉| 3960/4000 [4:38:59<02:44, 4.11s/it]
99%|█████████▉| 3961/4000 [4:39:03<02:40, 4.10s/it]
99%|█████████▉| 3962/4000 [4:39:07<02:39, 4.19s/it]
99%|█████████▉| 3963/4000 [4:39:11<02:33, 4.16s/it]
99%|█████████▉| 3964/4000 [4:39:15<02:29, 4.14s/it]
99%|█████████▉| 3965/4000 [4:39:19<02:24, 4.13s/it]
99%|█████████▉| 3966/4000 [4:39:23<02:20, 4.12s/it]
99%|█████████▉| 3967/4000 [4:39:28<02:15, 4.11s/it]
99%|█████████▉| 3968/4000 [4:39:32<02:13, 4.17s/it]
99%|█████████▉| 3969/4000 [4:39:36<02:10, 4.20s/it]
99%|█████████▉| 3970/4000 [4:39:40<02:05, 4.17s/it]
{'loss': '0.3607', 'grad_norm': '0.1194', 'learning_rate': '4.697e-06', 'epoch': '0.9619'}
99%|█████████▉| 3970/4000 [4:39:40<02:05, 4.17s/it]
99%|█████████▉| 3971/4000 [4:39:44<02:00, 4.15s/it]
99%|█████████▉| 3972/4000 [4:39:49<02:00, 4.31s/it]
99%|█████████▉| 3973/4000 [4:39:53<01:54, 4.25s/it]
99%|█████████▉| 3974/4000 [4:39:57<01:49, 4.20s/it]
99%|█████████▉| 3975/4000 [4:40:02<01:51, 4.45s/it]
99%|█████████▉| 3976/4000 [4:40:06<01:44, 4.35s/it]
99%|█████████▉| 3977/4000 [4:40:10<01:38, 4.27s/it]
99%|█████████▉| 3978/4000 [4:40:14<01:32, 4.22s/it]
99%|█████████▉| 3979/4000 [4:40:19<01:27, 4.18s/it]
100%|█████████▉| 3980/4000 [4:40:23<01:23, 4.16s/it]
{'loss': '0.3656', 'grad_norm': '0.1271', 'learning_rate': '3.182e-06', 'epoch': '0.9644'}
100%|█████████▉| 3980/4000 [4:40:23<01:23, 4.16s/it]
100%|█████████▉| 3981/4000 [4:40:27<01:18, 4.14s/it]
100%|█████████▉| 3982/4000 [4:40:31<01:16, 4.24s/it]
100%|█████████▉| 3983/4000 [4:40:35<01:11, 4.20s/it]
100%|█████████▉| 3984/4000 [4:40:39<01:06, 4.16s/it]
100%|█████████▉| 3985/4000 [4:40:44<01:02, 4.14s/it]
100%|█████████▉| 3986/4000 [4:40:48<00:57, 4.13s/it]
100%|█████████▉| 3987/4000 [4:40:52<00:53, 4.12s/it]
100%|█████████▉| 3988/4000 [4:40:56<00:49, 4.11s/it]
100%|█████████▉| 3989/4000 [4:41:00<00:46, 4.22s/it]
100%|█████████▉| 3990/4000 [4:41:04<00:41, 4.18s/it]
{'loss': '0.3475', 'grad_norm': '0.1199', 'learning_rate': '1.667e-06', 'epoch': '0.9668'}
100%|█████████▉| 3990/4000 [4:41:04<00:41, 4.18s/it]
100%|█████████▉| 3991/4000 [4:41:08<00:37, 4.15s/it]
100%|█████████▉| 3992/4000 [4:41:13<00:33, 4.14s/it]
100%|█████████▉| 3993/4000 [4:41:17<00:28, 4.12s/it]
100%|█████████▉| 3994/4000 [4:41:21<00:24, 4.11s/it]
100%|█████████▉| 3995/4000 [4:41:25<00:20, 4.15s/it]
100%|█████████▉| 3996/4000 [4:41:29<00:16, 4.19s/it]
100%|█████████▉| 3997/4000 [4:41:33<00:12, 4.16s/it]
100%|█████████▉| 3998/4000 [4:41:37<00:08, 4.14s/it]
100%|█████████▉| 3999/4000 [4:41:42<00:04, 4.12s/it]
100%|██████████| 4000/4000 [4:41:46<00:00, 4.11s/it]
{'loss': '0.3576', 'grad_norm': '0.1201', 'learning_rate': '1.515e-07', 'epoch': '0.9692'}
100%|██████████| 4000/4000 [4:41:46<00:00, 4.11s/it]
0%| | 0/2 [00:00<?, ?it/s][A
100%|██████████| 2/2 [00:00<00:00, 4.23it/s][A
[A{'eval_loss': '2.231', 'eval_runtime': '1.011', 'eval_samples_per_second': '21.76', 'eval_steps_per_second': '1.978', 'epoch': '0.9692'}
100%|██████████| 4000/4000 [4:41:47<00:00, 4.11s/it]
100%|██████████| 2/2 [00:00<00:00, 4.23it/s][A
[A[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
- If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
- If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
- If you are not the owner of the model architecture class, please contact the model code owner to update it.
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s][A
Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 3.88it/s][A
Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 3.87it/s]
{'train_runtime': '1.691e+04', 'train_samples_per_second': '7.569', 'train_steps_per_second': '0.237', 'train_loss': '0.6708', 'epoch': '0.9692'}
100%|██████████| 4000/4000 [4:41:50<00:00, 4.11s/it]
100%|██████████| 4000/4000 [4:41:50<00:00, 4.23s/it]
[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
- If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
- If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
- If you are not the owner of the model architecture class, please contact the model code owner to update it.
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
[Done] Model saved to outputs/devops-300m-4096-bf16-vast
Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 3.80it/s]
Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 3.80it/s]
[Done] Model saved to outputs/devops-300m-4096-bf16-vast
81d666a3c511:10693:10966 [1] NCCL INFO [Service thread] Connection closed by localRank 1
[rank0]:[W914 21:26:49.450832496 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator())
81d666a3c511:10692:10965 [0] NCCL INFO [Service thread] Connection closed by localRank 0
81d666a3c511:10693:15114 [1] NCCL INFO comm 0x1aa33b30 rank 1 nranks 2 cudaDev 1 busId d6000 - Abort COMPLETE
81d666a3c511:10692:15115 [0] NCCL INFO comm 0x1adc3b90 rank 0 nranks 2 cudaDev 0 busId d2000 - Abort COMPLETE
[2026-09-14 21:26:51,207] [INFO] [launch.py:367:main] Process 10692 exits successfully.
[2026-09-14 21:26:51,208] [INFO] [launch.py:367:main] Process 10693 exits successfully.
=== full run done Mon Sep 14 21:26:53 UTC 2026 exit=0 ===
|