=== full run start Mon Sep 14 16:44:32 UTC 2026 ===
[2026-09-14 16:44:37,556] [WARNING] [runner.py:232:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2026-09-14 16:44:37,556] [INFO] [runner.py:630:main] cmd = /opt/conda/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMV19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None --log_level=info scripts/training/train.py --model_config configs/model/model_300m.yaml --tokenizer_path tokenizer --train_data_dir /workspace/data/pretrain/train --val_data_dir /workspace/data/pretrain/val_small --output_dir outputs/devops-300m-4096-bf16-vast --max_steps 4000 --warmup_steps 40 --learning_rate 6e-4 --batch_size 1 --grad_accum 16 --logging_steps 10 --save_steps 2000 --eval_steps 2000 --deepspeed_config configs/training/deepspeed_zero2_bf16.json --bf16 --seq_length 4096 --run_name devops-300m-4096-bf16-vast
[2026-09-14 16:44:42,440] [INFO] [launch.py:155:main] 0 NCCL_DEBUG=INFO
[2026-09-14 16:44:42,440] [INFO] [launch.py:162:main] WORLD INFO DICT: {'localhost': [0, 1]}
[2026-09-14 16:44:42,440] [INFO] [launch.py:168:main] nnodes=1, num_local_procs=2, node_rank=0
[2026-09-14 16:44:42,440] [INFO] [launch.py:179:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1]})
[2026-09-14 16:44:42,440] [INFO] [launch.py:180:main] dist_world_size=2
[2026-09-14 16:44:42,440] [INFO] [launch.py:184:main] Setting CUDA_VISIBLE_DEVICES=0,1
[2026-09-14 16:44:42,441] [INFO] [launch.py:272:main] process 10692 spawned with command: ['/opt/conda/bin/python', '-u', 'scripts/training/train.py', '--local_rank=0', '--model_config', 'configs/model/model_300m.yaml', '--tokenizer_path', 'tokenizer', '--train_data_dir', '/workspace/data/pretrain/train', '--val_data_dir', '/workspace/data/pretrain/val_small', '--output_dir', 'outputs/devops-300m-4096-bf16-vast', '--max_steps', '4000', '--warmup_steps', '40', '--learning_rate', '6e-4', '--batch_size', '1', '--grad_accum', '16', '--logging_steps', '10', '--save_steps', '2000', '--eval_steps', '2000', '--deepspeed_config', 'configs/training/deepspeed_zero2_bf16.json', '--bf16', '--seq_length', '4096', '--run_name', 'devops-300m-4096-bf16-vast']
[2026-09-14 16:44:42,441] [INFO] [launch.py:272:main] process 10693 spawned with command: ['/opt/conda/bin/python', '-u', 'scripts/training/train.py', '--local_rank=1', '--model_config', 'configs/model/model_300m.yaml', '--tokenizer_path', 'tokenizer', '--train_data_dir', '/workspace/data/pretrain/train', '--val_data_dir', '/workspace/data/pretrain/val_small', '--output_dir', 'outputs/devops-300m-4096-bf16-vast', '--max_steps', '4000', '--warmup_steps', '40', '--learning_rate', '6e-4', '--batch_size', '1', '--grad_accum', '16', '--logging_steps', '10', '--save_steps', '2000', '--eval_steps', '2000', '--deepspeed_config', 'configs/training/deepspeed_zero2_bf16.json', '--bf16', '--seq_length', '4096', '--run_name', 'devops-300m-4096-bf16-vast']
[Tokenizer] vocab_size=65536
[Tokenizer] vocab_size=65536
[Dataset] 1509 files, 132,068 non-overlapping samples, seq_len=4096
[Dataset] 10 files, 22 non-overlapping samples, seq_len=4096
[Dataset] 1509 files, 132,068 non-overlapping samples, seq_len=4096
[Dataset] 10 files, 22 non-overlapping samples, seq_len=4096
[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
  - If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
  - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
  - If you are not the owner of the model architecture class, please contact the model code owner to update it.
[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
  - If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
  - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
  - If you are not the owner of the model architecture class, please contact the model code owner to update it.
[Model] 287.3M parameters
[Env] 2 GPUs detected, using DeepSpeed (launch with: deepspeed --master_port 29500 train.py ...)
[Model] 287.3M parameters
[Env] 2 GPUs detected, using DeepSpeed (launch with: deepspeed --master_port 29500 train.py ...)
81d666a3c511:10692:10692 [0] NCCL INFO Bootstrap : Using eth0:172.17.0.4<0>
81d666a3c511:10692:10692 [0] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
81d666a3c511:10692:10692 [0] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
81d666a3c511:10692:10692 [0] NCCL INFO NET/Plugin: Using internal network plugin.
81d666a3c511:10692:10692 [0] NCCL INFO cudaDriverVersion 13020
NCCL version 2.21.5+cuda12.4
81d666a3c511:10693:10693 [1] NCCL INFO cudaDriverVersion 13020
81d666a3c511:10693:10693 [1] NCCL INFO Bootstrap : Using eth0:172.17.0.4<0>
81d666a3c511:10693:10693 [1] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
81d666a3c511:10693:10693 [1] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
81d666a3c511:10693:10693 [1] NCCL INFO NET/Plugin: Using internal network plugin.
81d666a3c511:10693:10962 [1] NCCL INFO Failed to open libibverbs.so[.1]
81d666a3c511:10693:10962 [1] NCCL INFO NET/Socket : Using [0]eth0:172.17.0.4<0>
81d666a3c511:10693:10962 [1] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10693:10962 [1] NCCL INFO Using network Socket
81d666a3c511:10692:10964 [0] NCCL INFO Failed to open libibverbs.so[.1]
81d666a3c511:10692:10964 [0] NCCL INFO NET/Socket : Using [0]eth0:172.17.0.4<0>
81d666a3c511:10692:10964 [0] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10692:10964 [0] NCCL INFO Using network Socket
81d666a3c511:10692:10964 [0] NCCL INFO ncclCommInitRank comm 0x1adc3b90 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 commId 0x86d35acf72d855e7 - Init START
81d666a3c511:10693:10962 [1] NCCL INFO ncclCommInitRank comm 0x1aa33b30 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 commId 0x86d35acf72d855e7 - Init START
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:10964 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:10962 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10692:10964 [0] NCCL INFO comm 0x1adc3b90 rank 0 nRanks 2 nNodes 1 localRanks 2 localRank 0 MNNVL 0
81d666a3c511:10693:10962 [1] NCCL INFO comm 0x1aa33b30 rank 1 nRanks 2 nNodes 1 localRanks 2 localRank 1 MNNVL 0
81d666a3c511:10692:10964 [0] NCCL INFO Channel 00/02 :    0   1
81d666a3c511:10692:10964 [0] NCCL INFO Channel 01/02 :    0   1
81d666a3c511:10692:10964 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1
81d666a3c511:10692:10964 [0] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10693:10962 [1] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0
81d666a3c511:10693:10962 [1] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10692:10964 [0] NCCL INFO Channel 00 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10692:10964 [0] NCCL INFO Channel 01 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10693:10962 [1] NCCL INFO Channel 00 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10693:10962 [1] NCCL INFO Channel 01 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10692:10964 [0] NCCL INFO Connected all rings
81d666a3c511:10692:10964 [0] NCCL INFO Connected all trees
81d666a3c511:10693:10962 [1] NCCL INFO Connected all rings
81d666a3c511:10693:10962 [1] NCCL INFO Connected all trees
81d666a3c511:10693:10962 [1] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10693:10962 [1] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:10964 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10692:10964 [0] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:10964 [0] NCCL INFO TUNER/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
81d666a3c511:10693:10962 [1] NCCL INFO TUNER/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
81d666a3c511:10692:10964 [0] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
81d666a3c511:10693:10962 [1] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
81d666a3c511:10692:10964 [0] NCCL INFO ncclCommInitRank comm 0x1adc3b90 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 commId 0x86d35acf72d855e7 - Init COMPLETE
81d666a3c511:10693:10962 [1] NCCL INFO ncclCommInitRank comm 0x1aa33b30 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 commId 0x86d35acf72d855e7 - Init COMPLETE

[Train] Starting: 4000 steps, batch=1×16×world_size, lr=0.0006, seq_len=4096

[Train] Starting: 4000 steps, batch=1×16×world_size, lr=0.0006, seq_len=4096
[RANK 0] Gradient accumulation steps mismatch: GradientAccumulationPlugin has 1, DeepSpeed config has 16. Using DeepSpeed's value.
81d666a3c511:10693:11106 [1] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10693:11106 [1] NCCL INFO Using network Socket
81d666a3c511:10692:11110 [0] NCCL INFO Using non-device net plugin version 0
81d666a3c511:10692:11110 [0] NCCL INFO Using network Socket
81d666a3c511:10693:11106 [1] NCCL INFO bootstrapSplit: comm 0xc798460 parent 0x1aa33b30 rank 1 nranks 2 color 1530306504 key 1 prev 0 next 0 - DONE
81d666a3c511:10693:11106 [1] NCCL INFO ncclCommSplit comm 0xc798460 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 parent 0x1aa33b30 color 1530306504 key 1 commId 0xbc296a4dcdede661 - Init START
81d666a3c511:10692:11110 [0] NCCL INFO bootstrapSplit: comm 0xcf26230 parent 0x1adc3b90 rank 0 nranks 2 color 1530306504 key 0 prev 1 next 1 - DONE
81d666a3c511:10692:11110 [0] NCCL INFO ncclCommSplit comm 0xcf26230 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 parent 0x1adc3b90 color 1530306504 key 0 commId 0xbc296a4dcdede661 - Init START
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10693:11106 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 1 and 0. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO P2P is disabled between connected GPUs 0 and 1. You can repress this message with NCCL_IGNORE_DISABLED_P2P=1.
81d666a3c511:10692:11110 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff,00000000
81d666a3c511:10693:11106 [1] NCCL INFO comm 0xc798460 rank 1 nRanks 2 nNodes 1 localRanks 2 localRank 1 MNNVL 0
81d666a3c511:10692:11110 [0] NCCL INFO comm 0xcf26230 rank 0 nRanks 2 nNodes 1 localRanks 2 localRank 0 MNNVL 0
81d666a3c511:10693:11106 [1] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0
81d666a3c511:10693:11106 [1] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10692:11110 [0] NCCL INFO Channel 00/02 :    0   1
81d666a3c511:10692:11110 [0] NCCL INFO Channel 01/02 :    0   1
81d666a3c511:10692:11110 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1
81d666a3c511:10692:11110 [0] NCCL INFO P2P Chunksize set to 131072
81d666a3c511:10693:11106 [1] NCCL INFO Channel 00 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10693:11106 [1] NCCL INFO Channel 01 : 1[1] -> 0[0] via SHM/direct/direct
81d666a3c511:10692:11110 [0] NCCL INFO Channel 00 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10692:11110 [0] NCCL INFO Channel 01 : 0[0] -> 1[1] via SHM/direct/direct
81d666a3c511:10693:11106 [1] NCCL INFO Connected all rings
81d666a3c511:10693:11106 [1] NCCL INFO Connected all trees
81d666a3c511:10693:11106 [1] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10693:11106 [1] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:11110 [0] NCCL INFO Connected all rings
81d666a3c511:10692:11110 [0] NCCL INFO Connected all trees
81d666a3c511:10692:11110 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512
81d666a3c511:10692:11110 [0] NCCL INFO 2 coll channels, 2 collnet channels, 0 nvls channels, 2 p2p channels, 2 p2p channels per peer
81d666a3c511:10692:11110 [0] NCCL INFO ncclCommSplit comm 0xcf26230 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId d2000 parent 0x1adc3b90 color 1530306504 key 0 commId 0xbc296a4dcdede661 - Init COMPLETE
81d666a3c511:10693:11106 [1] NCCL INFO ncclCommSplit comm 0xc798460 rank 1 nranks 2 cudaDev 1 nvmlDev 1 busId d6000 parent 0x1aa33b30 color 1530306504 key 1 commId 0xbc296a4dcdede661 - Init COMPLETE
[2026-09-14 16:44:56,871] [WARNING] [lr_schedules.py:693:get_lr] Attempting to get learning rate from scheduler before it has started
[2026-09-14 16:44:57,091] [WARNING] [lr_schedules.py:693:get_lr] Attempting to get learning rate from scheduler before it has started

  0%|          | 0/4000 [00:00<?, ?it/s][transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!

  0%|          | 1/4000 [00:05<5:36:20,  5.05s/it]
  0%|          | 2/4000 [00:09<5:01:22,  4.52s/it]
  0%|          | 3/4000 [00:13<4:49:48,  4.35s/it]
  0%|          | 4/4000 [00:17<4:43:51,  4.26s/it]
  0%|          | 5/4000 [00:21<4:40:02,  4.21s/it]
  0%|          | 6/4000 [00:25<4:37:30,  4.17s/it]
  0%|          | 7/4000 [00:30<4:45:38,  4.29s/it]
  0%|          | 8/4000 [00:34<4:41:01,  4.22s/it]
  0%|          | 9/4000 [00:38<4:37:14,  4.17s/it]
  0%|          | 10/4000 [00:42<4:34:56,  4.13s/it]
                                                   
{'loss': '9.602', 'grad_norm': '2.286', 'learning_rate': '0.0003745', 'epoch': '0.002423'}

  0%|          | 10/4000 [00:42<4:34:56,  4.13s/it]
  0%|          | 11/4000 [00:46<4:33:45,  4.12s/it]
  0%|          | 12/4000 [00:50<4:33:06,  4.11s/it]
  0%|          | 13/4000 [00:54<4:36:32,  4.16s/it]
  0%|          | 14/4000 [00:59<4:38:38,  4.19s/it]
  0%|          | 15/4000 [01:03<4:36:01,  4.16s/it]
  0%|          | 16/4000 [01:07<4:34:29,  4.13s/it]
  0%|          | 17/4000 [01:11<4:33:13,  4.12s/it]
  0%|          | 18/4000 [01:15<4:32:44,  4.11s/it]
  0%|          | 19/4000 [01:19<4:32:15,  4.10s/it]
  0%|          | 20/4000 [01:24<4:39:36,  4.22s/it]
                                                   
{'loss': '6.495', 'grad_norm': '0.9675', 'learning_rate': '0.0004873', 'epoch': '0.004846'}

  0%|          | 20/4000 [01:24<4:39:36,  4.22s/it]
  1%|          | 21/4000 [01:28<4:36:45,  4.17s/it]
  1%|          | 22/4000 [01:32<4:34:35,  4.14s/it]
  1%|          | 23/4000 [01:36<4:33:19,  4.12s/it]
  1%|          | 24/4000 [01:40<4:32:11,  4.11s/it]
  1%|          | 25/4000 [01:44<4:31:40,  4.10s/it]
  1%|          | 26/4000 [01:48<4:31:10,  4.09s/it]
  1%|          | 27/4000 [01:52<4:38:34,  4.21s/it]
  1%|          | 28/4000 [01:57<4:35:46,  4.17s/it]
  1%|          | 29/4000 [02:01<4:34:07,  4.14s/it]
  1%|          | 30/4000 [02:05<4:33:20,  4.13s/it]
                                                   
{'loss': '5.982', 'grad_norm': '1.023', 'learning_rate': '0.0005532', 'epoch': '0.007269'}

  1%|          | 30/4000 [02:05<4:33:20,  4.13s/it]
  1%|          | 31/4000 [02:09<4:33:04,  4.13s/it]
  1%|          | 32/4000 [02:13<4:33:06,  4.13s/it]
  1%|          | 33/4000 [02:17<4:32:41,  4.12s/it]
  1%|          | 34/4000 [02:22<4:40:00,  4.24s/it]
  1%|          | 35/4000 [02:26<4:36:57,  4.19s/it]
  1%|          | 36/4000 [02:30<4:34:41,  4.16s/it]
  1%|          | 37/4000 [02:34<4:33:18,  4.14s/it]
  1%|          | 38/4000 [02:38<4:32:43,  4.13s/it]
  1%|          | 39/4000 [02:42<4:32:33,  4.13s/it]
  1%|          | 40/4000 [02:46<4:35:49,  4.18s/it]
                                                   
{'loss': '5.683', 'grad_norm': '1.414', 'learning_rate': '0.0006', 'epoch': '0.009692'}

  1%|          | 40/4000 [02:46<4:35:49,  4.18s/it]
  1%|          | 41/4000 [02:51<4:38:33,  4.22s/it]
  1%|          | 42/4000 [02:55<4:37:00,  4.20s/it]
  1%|          | 43/4000 [02:59<4:35:33,  4.18s/it]
  1%|          | 44/4000 [03:03<4:33:54,  4.15s/it]
  1%|          | 45/4000 [03:07<4:32:41,  4.14s/it]
  1%|          | 46/4000 [03:11<4:32:04,  4.13s/it]
  1%|          | 47/4000 [03:16<4:39:07,  4.24s/it]
  1%|          | 48/4000 [03:20<4:36:54,  4.20s/it]
  1%|          | 49/4000 [03:24<4:35:00,  4.18s/it]
  1%|▏         | 50/4000 [03:28<4:33:25,  4.15s/it]
                                                   
{'loss': '5.258', 'grad_norm': '2.436', 'learning_rate': '0.0005986', 'epoch': '0.01211'}

  1%|▏         | 50/4000 [03:28<4:33:25,  4.15s/it]
  1%|▏         | 51/4000 [03:32<4:32:41,  4.14s/it]
  1%|▏         | 52/4000 [03:36<4:31:52,  4.13s/it]
  1%|▏         | 53/4000 [03:40<4:30:50,  4.12s/it]
  1%|▏         | 54/4000 [03:45<4:37:37,  4.22s/it]
  1%|▏         | 55/4000 [03:49<4:34:50,  4.18s/it]
  1%|▏         | 56/4000 [03:53<4:32:55,  4.15s/it]
  1%|▏         | 57/4000 [03:57<4:31:40,  4.13s/it]
  1%|▏         | 58/4000 [04:01<4:30:43,  4.12s/it]
  1%|▏         | 59/4000 [04:05<4:30:07,  4.11s/it]
  2%|▏         | 60/4000 [04:09<4:29:39,  4.11s/it]
                                                   
{'loss': '4.63', 'grad_norm': '1.147', 'learning_rate': '0.0005971', 'epoch': '0.01454'}

  2%|▏         | 60/4000 [04:09<4:29:39,  4.11s/it]
  2%|▏         | 61/4000 [04:14<4:36:15,  4.21s/it]
  2%|▏         | 62/4000 [04:18<4:34:06,  4.18s/it]
  2%|▏         | 63/4000 [04:22<4:32:23,  4.15s/it]
  2%|▏         | 64/4000 [04:26<4:31:10,  4.13s/it]
  2%|▏         | 65/4000 [04:30<4:30:13,  4.12s/it]
  2%|▏         | 66/4000 [04:34<4:29:33,  4.11s/it]
  2%|▏         | 67/4000 [04:39<4:32:41,  4.16s/it]
  2%|▏         | 68/4000 [04:43<4:34:43,  4.19s/it]
  2%|▏         | 69/4000 [04:47<4:32:46,  4.16s/it]
  2%|▏         | 70/4000 [04:51<4:31:28,  4.14s/it]
                                                   
{'loss': '3.956', 'grad_norm': '1.276', 'learning_rate': '0.0005956', 'epoch': '0.01696'}

  2%|▏         | 70/4000 [04:51<4:31:28,  4.14s/it]
  2%|▏         | 71/4000 [04:55<4:30:15,  4.13s/it]
  2%|▏         | 72/4000 [04:59<4:29:14,  4.11s/it]
  2%|▏         | 73/4000 [05:03<4:28:36,  4.10s/it]
  2%|▏         | 74/4000 [05:08<4:35:56,  4.22s/it]
  2%|▏         | 75/4000 [05:12<4:33:28,  4.18s/it]
  2%|▏         | 76/4000 [05:16<4:31:42,  4.15s/it]
  2%|▏         | 77/4000 [05:20<4:30:22,  4.14s/it]
  2%|▏         | 78/4000 [05:24<4:29:19,  4.12s/it]
  2%|▏         | 79/4000 [05:28<4:28:46,  4.11s/it]
  2%|▏         | 80/4000 [05:32<4:28:17,  4.11s/it]
                                                   
{'loss': '3.471', 'grad_norm': '0.7973', 'learning_rate': '0.0005941', 'epoch': '0.01938'}

  2%|▏         | 80/4000 [05:32<4:28:17,  4.11s/it]
  2%|▏         | 81/4000 [05:37<4:33:40,  4.19s/it]
  2%|▏         | 82/4000 [05:41<4:31:34,  4.16s/it]
  2%|▏         | 83/4000 [05:45<4:30:06,  4.14s/it]
  2%|▏         | 84/4000 [05:49<4:29:04,  4.12s/it]
  2%|▏         | 85/4000 [05:53<4:28:28,  4.11s/it]
  2%|▏         | 86/4000 [05:57<4:28:15,  4.11s/it]
  2%|▏         | 87/4000 [06:01<4:28:05,  4.11s/it]
  2%|▏         | 88/4000 [06:06<4:35:19,  4.22s/it]
  2%|▏         | 89/4000 [06:10<4:32:36,  4.18s/it]
  2%|▏         | 90/4000 [06:14<4:31:02,  4.16s/it]
                                                   
{'loss': '3.021', 'grad_norm': '0.7119', 'learning_rate': '0.0005926', 'epoch': '0.02181'}

  2%|▏         | 90/4000 [06:14<4:31:02,  4.16s/it]
  2%|▏         | 91/4000 [06:18<4:29:50,  4.14s/it]
  2%|▏         | 92/4000 [06:22<4:28:49,  4.13s/it]
  2%|▏         | 93/4000 [06:26<4:28:11,  4.12s/it]
  2%|▏         | 94/4000 [06:31<4:33:26,  4.20s/it]
  2%|▏         | 95/4000 [06:35<4:31:19,  4.17s/it]
  2%|▏         | 96/4000 [06:39<4:29:39,  4.14s/it]
  2%|▏         | 97/4000 [06:43<4:28:49,  4.13s/it]
  2%|▏         | 98/4000 [06:47<4:27:56,  4.12s/it]
  2%|▏         | 99/4000 [06:51<4:27:28,  4.11s/it]
  2%|▎         | 100/4000 [06:55<4:26:55,  4.11s/it]
                                                    
{'loss': '2.637', 'grad_norm': '0.6688', 'learning_rate': '0.0005911', 'epoch': '0.02423'}

  2%|▎         | 100/4000 [06:55<4:26:55,  4.11s/it]
  3%|▎         | 101/4000 [07:00<4:34:09,  4.22s/it]
  3%|▎         | 102/4000 [07:04<4:31:45,  4.18s/it]
  3%|▎         | 103/4000 [07:08<4:29:58,  4.16s/it]
  3%|▎         | 104/4000 [07:12<4:28:31,  4.14s/it]
  3%|▎         | 105/4000 [07:16<4:27:33,  4.12s/it]
  3%|▎         | 106/4000 [07:20<4:26:52,  4.11s/it]
  3%|▎         | 107/4000 [07:24<4:26:21,  4.11s/it]
  3%|▎         | 108/4000 [07:29<4:32:45,  4.20s/it]
  3%|▎         | 109/4000 [07:33<4:30:28,  4.17s/it]
  3%|▎         | 110/4000 [07:37<4:28:50,  4.15s/it]
                                                    
{'loss': '2.353', 'grad_norm': '0.5264', 'learning_rate': '0.0005895', 'epoch': '0.02665'}

  3%|▎         | 110/4000 [07:37<4:28:50,  4.15s/it]
  3%|▎         | 111/4000 [07:41<4:27:40,  4.13s/it]
  3%|▎         | 112/4000 [07:45<4:26:59,  4.12s/it]
  3%|▎         | 113/4000 [07:49<4:26:34,  4.11s/it]
  3%|▎         | 114/4000 [07:53<4:26:06,  4.11s/it]
  3%|▎         | 115/4000 [07:58<4:32:41,  4.21s/it]
  3%|▎         | 116/4000 [08:02<4:30:24,  4.18s/it]
  3%|▎         | 117/4000 [08:06<4:28:57,  4.16s/it]
  3%|▎         | 118/4000 [08:10<4:27:45,  4.14s/it]
  3%|▎         | 119/4000 [08:14<4:27:05,  4.13s/it]
  3%|▎         | 120/4000 [08:18<4:26:31,  4.12s/it]
                                                    
{'loss': '2.108', 'grad_norm': '0.4656', 'learning_rate': '0.000588', 'epoch': '0.02908'}

  3%|▎         | 120/4000 [08:18<4:26:31,  4.12s/it]
  3%|▎         | 121/4000 [08:23<4:33:00,  4.22s/it]
  3%|▎         | 122/4000 [08:27<4:30:33,  4.19s/it]
  3%|▎         | 123/4000 [08:31<4:28:35,  4.16s/it]
  3%|▎         | 124/4000 [08:35<4:27:29,  4.14s/it]
  3%|▎         | 125/4000 [08:39<4:26:33,  4.13s/it]
  3%|▎         | 126/4000 [08:43<4:26:02,  4.12s/it]
  3%|▎         | 127/4000 [08:47<4:25:25,  4.11s/it]
  3%|▎         | 128/4000 [08:52<4:30:59,  4.20s/it]
  3%|▎         | 129/4000 [08:56<4:28:58,  4.17s/it]
  3%|▎         | 130/4000 [09:00<4:27:26,  4.15s/it]
                                                    
{'loss': '1.974', 'grad_norm': '0.4384', 'learning_rate': '0.0005865', 'epoch': '0.0315'}

  3%|▎         | 130/4000 [09:00<4:27:26,  4.15s/it]
  3%|▎         | 131/4000 [09:04<4:26:40,  4.14s/it]
  3%|▎         | 132/4000 [09:08<4:25:50,  4.12s/it]
  3%|▎         | 133/4000 [09:12<4:25:21,  4.12s/it]
  3%|▎         | 134/4000 [09:16<4:24:53,  4.11s/it]
  3%|▎         | 135/4000 [09:21<4:31:25,  4.21s/it]
  3%|▎         | 136/4000 [09:25<4:29:01,  4.18s/it]
  3%|▎         | 137/4000 [09:29<4:27:19,  4.15s/it]
  3%|▎         | 138/4000 [09:33<4:26:11,  4.14s/it]
  3%|▎         | 139/4000 [09:37<4:25:21,  4.12s/it]
  4%|▎         | 140/4000 [09:41<4:24:34,  4.11s/it]
                                                    
{'loss': '1.871', 'grad_norm': '0.443', 'learning_rate': '0.000585', 'epoch': '0.03392'}

  4%|▎         | 140/4000 [09:41<4:24:34,  4.11s/it]
  4%|▎         | 141/4000 [09:45<4:27:53,  4.17s/it]
  4%|▎         | 142/4000 [09:50<4:29:39,  4.19s/it]
  4%|▎         | 143/4000 [09:54<4:27:37,  4.16s/it]
  4%|▎         | 144/4000 [09:58<4:26:16,  4.14s/it]
  4%|▎         | 145/4000 [10:02<4:25:30,  4.13s/it]
  4%|▎         | 146/4000 [10:06<4:24:50,  4.12s/it]
  4%|▎         | 147/4000 [10:10<4:24:09,  4.11s/it]
  4%|▎         | 148/4000 [10:15<4:30:09,  4.21s/it]
  4%|▎         | 149/4000 [10:19<4:27:54,  4.17s/it]
  4%|▍         | 150/4000 [10:23<4:26:15,  4.15s/it]
                                                    
{'loss': '1.745', 'grad_norm': '0.3832', 'learning_rate': '0.0005835', 'epoch': '0.03634'}

  4%|▍         | 150/4000 [10:23<4:26:15,  4.15s/it]
  4%|▍         | 151/4000 [10:27<4:25:09,  4.13s/it]
  4%|▍         | 152/4000 [10:31<4:24:16,  4.12s/it]
  4%|▍         | 153/4000 [10:35<4:23:44,  4.11s/it]
  4%|▍         | 154/4000 [10:39<4:23:24,  4.11s/it]
  4%|▍         | 155/4000 [10:44<4:29:38,  4.21s/it]
  4%|▍         | 156/4000 [10:48<4:27:29,  4.18s/it]
  4%|▍         | 157/4000 [10:52<4:26:01,  4.15s/it]
  4%|▍         | 158/4000 [10:56<4:24:47,  4.14s/it]
  4%|▍         | 159/4000 [11:00<4:24:03,  4.12s/it]
  4%|▍         | 160/4000 [11:04<4:23:40,  4.12s/it]
                                                    
{'loss': '1.724', 'grad_norm': '0.3886', 'learning_rate': '0.000582', 'epoch': '0.03877'}

  4%|▍         | 160/4000 [11:04<4:23:40,  4.12s/it]
  4%|▍         | 161/4000 [11:08<4:23:12,  4.11s/it]
  4%|▍         | 162/4000 [11:13<4:28:48,  4.20s/it]
  4%|▍         | 163/4000 [11:17<4:26:48,  4.17s/it]
  4%|▍         | 164/4000 [11:21<4:25:17,  4.15s/it]
  4%|▍         | 165/4000 [11:25<4:24:09,  4.13s/it]
  4%|▍         | 166/4000 [11:29<4:23:26,  4.12s/it]
  4%|▍         | 167/4000 [11:33<4:22:49,  4.11s/it]
  4%|▍         | 168/4000 [11:37<4:25:55,  4.16s/it]
  4%|▍         | 169/4000 [11:42<4:28:16,  4.20s/it]
  4%|▍         | 170/4000 [11:46<4:26:00,  4.17s/it]
                                                    
{'loss': '1.625', 'grad_norm': '0.5112', 'learning_rate': '0.0005805', 'epoch': '0.04119'}

  4%|▍         | 170/4000 [11:46<4:26:00,  4.17s/it]
  4%|▍         | 171/4000 [11:50<4:24:38,  4.15s/it]
  4%|▍         | 172/4000 [11:54<4:23:38,  4.13s/it]
  4%|▍         | 173/4000 [11:58<4:22:43,  4.12s/it]
  4%|▍         | 174/4000 [12:02<4:22:21,  4.11s/it]
  4%|▍         | 175/4000 [12:07<4:28:10,  4.21s/it]
  4%|▍         | 176/4000 [12:11<4:26:01,  4.17s/it]
  4%|▍         | 177/4000 [12:15<4:24:22,  4.15s/it]
  4%|▍         | 178/4000 [12:19<4:23:30,  4.14s/it]
  4%|▍         | 179/4000 [12:23<4:22:46,  4.13s/it]
  4%|▍         | 180/4000 [12:27<4:22:07,  4.12s/it]
                                                    
{'loss': '1.559', 'grad_norm': '0.3595', 'learning_rate': '0.0005789', 'epoch': '0.04361'}

  4%|▍         | 180/4000 [12:27<4:22:07,  4.12s/it]
  5%|▍         | 181/4000 [12:31<4:21:39,  4.11s/it]
  5%|▍         | 182/4000 [12:36<4:27:47,  4.21s/it]
  5%|▍         | 183/4000 [12:40<4:25:34,  4.17s/it]
  5%|▍         | 184/4000 [12:44<4:24:02,  4.15s/it]
  5%|▍         | 185/4000 [12:48<4:22:48,  4.13s/it]
  5%|▍         | 186/4000 [12:52<4:22:00,  4.12s/it]
  5%|▍         | 187/4000 [12:56<4:21:24,  4.11s/it]
  5%|▍         | 188/4000 [13:00<4:21:00,  4.11s/it]
  5%|▍         | 189/4000 [13:05<4:27:27,  4.21s/it]
  5%|▍         | 190/4000 [13:09<4:25:15,  4.18s/it]
                                                    
{'loss': '1.55', 'grad_norm': '0.2786', 'learning_rate': '0.0005774', 'epoch': '0.04604'}

  5%|▍         | 190/4000 [13:09<4:25:15,  4.18s/it]
  5%|▍         | 191/4000 [13:13<4:23:40,  4.15s/it]
  5%|▍         | 192/4000 [13:17<4:22:23,  4.13s/it]
  5%|▍         | 193/4000 [13:21<4:21:38,  4.12s/it]
  5%|▍         | 194/4000 [13:25<4:21:07,  4.12s/it]
  5%|▍         | 195/4000 [13:29<4:23:39,  4.16s/it]
  5%|▍         | 196/4000 [13:34<4:25:30,  4.19s/it]
  5%|▍         | 197/4000 [13:38<4:23:39,  4.16s/it]
  5%|▍         | 198/4000 [13:42<4:22:29,  4.14s/it]
  5%|▍         | 199/4000 [13:46<4:21:33,  4.13s/it]
  5%|▌         | 200/4000 [13:50<4:21:04,  4.12s/it]
                                                    
{'loss': '1.418', 'grad_norm': '0.3144', 'learning_rate': '0.0005759', 'epoch': '0.04846'}

  5%|▌         | 200/4000 [13:50<4:21:04,  4.12s/it]
  5%|▌         | 201/4000 [13:54<4:20:37,  4.12s/it]
  5%|▌         | 202/4000 [13:59<4:27:28,  4.23s/it]
  5%|▌         | 203/4000 [14:03<4:25:03,  4.19s/it]
  5%|▌         | 204/4000 [14:07<4:23:04,  4.16s/it]
  5%|▌         | 205/4000 [14:11<4:21:50,  4.14s/it]
  5%|▌         | 206/4000 [14:15<4:20:49,  4.12s/it]
  5%|▌         | 207/4000 [14:19<4:20:10,  4.12s/it]
  5%|▌         | 208/4000 [14:23<4:19:47,  4.11s/it]
  5%|▌         | 209/4000 [14:28<4:26:08,  4.21s/it]
  5%|▌         | 210/4000 [14:32<4:23:52,  4.18s/it]
                                                    
{'loss': '1.449', 'grad_norm': '0.3036', 'learning_rate': '0.0005744', 'epoch': '0.05088'}

  5%|▌         | 210/4000 [14:32<4:23:52,  4.18s/it]
  5%|▌         | 211/4000 [14:36<4:22:19,  4.15s/it]
  5%|▌         | 212/4000 [14:40<4:21:20,  4.14s/it]
  5%|▌         | 213/4000 [14:44<4:20:31,  4.13s/it]
  5%|▌         | 214/4000 [14:48<4:19:59,  4.12s/it]
  5%|▌         | 215/4000 [14:52<4:19:29,  4.11s/it]
  5%|▌         | 216/4000 [14:57<4:25:25,  4.21s/it]
  5%|▌         | 217/4000 [15:01<4:23:12,  4.17s/it]
  5%|▌         | 218/4000 [15:05<4:21:32,  4.15s/it]
  5%|▌         | 219/4000 [15:09<4:20:36,  4.14s/it]
  6%|▌         | 220/4000 [15:13<4:19:40,  4.12s/it]
                                                    
{'loss': '1.435', 'grad_norm': '0.4483', 'learning_rate': '0.0005729', 'epoch': '0.05331'}

  6%|▌         | 220/4000 [15:13<4:19:40,  4.12s/it]
  6%|▌         | 221/4000 [15:17<4:19:14,  4.12s/it]
  6%|▌         | 222/4000 [15:22<4:25:10,  4.21s/it]
  6%|▌         | 223/4000 [15:26<4:22:49,  4.18s/it]
  6%|▌         | 224/4000 [15:30<4:21:28,  4.15s/it]
  6%|▌         | 225/4000 [15:34<4:20:13,  4.14s/it]
  6%|▌         | 226/4000 [15:38<4:19:24,  4.12s/it]
  6%|▌         | 227/4000 [15:42<4:18:53,  4.12s/it]
  6%|▌         | 228/4000 [15:46<4:18:26,  4.11s/it]
  6%|▌         | 229/4000 [15:51<4:23:41,  4.20s/it]
  6%|▌         | 230/4000 [15:55<4:21:44,  4.17s/it]
                                                    
{'loss': '1.39', 'grad_norm': '0.251', 'learning_rate': '0.0005714', 'epoch': '0.05573'}

  6%|▌         | 230/4000 [15:55<4:21:44,  4.17s/it]
  6%|▌         | 231/4000 [15:59<4:20:26,  4.15s/it]
  6%|▌         | 232/4000 [16:03<4:19:24,  4.13s/it]
  6%|▌         | 233/4000 [16:07<4:18:37,  4.12s/it]
  6%|▌         | 234/4000 [16:11<4:18:08,  4.11s/it]
  6%|▌         | 235/4000 [16:15<4:17:46,  4.11s/it]
  6%|▌         | 236/4000 [16:20<4:24:34,  4.22s/it]
  6%|▌         | 237/4000 [16:24<4:22:22,  4.18s/it]
  6%|▌         | 238/4000 [16:28<4:20:31,  4.16s/it]
  6%|▌         | 239/4000 [16:32<4:19:13,  4.14s/it]
  6%|▌         | 240/4000 [16:36<4:18:23,  4.12s/it]
                                                    
{'loss': '1.308', 'grad_norm': '0.2457', 'learning_rate': '0.0005698', 'epoch': '0.05815'}

  6%|▌         | 240/4000 [16:36<4:18:23,  4.12s/it]
  6%|▌         | 241/4000 [16:40<4:17:50,  4.12s/it]
  6%|▌         | 242/4000 [16:44<4:17:18,  4.11s/it]
  6%|▌         | 243/4000 [16:49<4:23:12,  4.20s/it]
  6%|▌         | 244/4000 [16:53<4:21:07,  4.17s/it]
  6%|▌         | 245/4000 [16:57<4:19:47,  4.15s/it]
  6%|▌         | 246/4000 [17:01<4:18:42,  4.13s/it]
  6%|▌         | 247/4000 [17:05<4:18:06,  4.13s/it]
  6%|▌         | 248/4000 [17:09<4:17:43,  4.12s/it]
  6%|▌         | 249/4000 [17:14<4:23:47,  4.22s/it]
  6%|▋         | 250/4000 [17:18<4:21:24,  4.18s/it]
                                                    
{'loss': '1.299', 'grad_norm': '0.2576', 'learning_rate': '0.0005683', 'epoch': '0.06057'}

  6%|▋         | 250/4000 [17:18<4:21:24,  4.18s/it]
  6%|▋         | 251/4000 [17:22<4:19:46,  4.16s/it]
  6%|▋         | 252/4000 [17:26<4:18:36,  4.14s/it]
  6%|▋         | 253/4000 [17:30<4:17:57,  4.13s/it]
  6%|▋         | 254/4000 [17:34<4:17:16,  4.12s/it]
  6%|▋         | 255/4000 [17:38<4:16:44,  4.11s/it]
  6%|▋         | 256/4000 [17:43<4:22:49,  4.21s/it]
  6%|▋         | 257/4000 [17:47<4:20:36,  4.18s/it]
  6%|▋         | 258/4000 [17:51<4:19:00,  4.15s/it]
  6%|▋         | 259/4000 [17:55<4:17:49,  4.14s/it]
  6%|▋         | 260/4000 [17:59<4:17:05,  4.12s/it]
                                                    
{'loss': '1.242', 'grad_norm': '0.2659', 'learning_rate': '0.0005668', 'epoch': '0.063'}

  6%|▋         | 260/4000 [17:59<4:17:05,  4.12s/it]
  7%|▋         | 261/4000 [18:03<4:16:37,  4.12s/it]
  7%|▋         | 262/4000 [18:07<4:16:13,  4.11s/it]
  7%|▋         | 263/4000 [18:12<4:21:17,  4.20s/it]
  7%|▋         | 264/4000 [18:16<4:19:31,  4.17s/it]
  7%|▋         | 265/4000 [18:20<4:18:02,  4.15s/it]
  7%|▋         | 266/4000 [18:24<4:17:04,  4.13s/it]
  7%|▋         | 267/4000 [18:28<4:16:23,  4.12s/it]
  7%|▋         | 268/4000 [18:32<4:15:58,  4.12s/it]
  7%|▋         | 269/4000 [18:36<4:19:12,  4.17s/it]
  7%|▋         | 270/4000 [18:41<4:21:23,  4.20s/it]
                                                    
{'loss': '1.182', 'grad_norm': '0.286', 'learning_rate': '0.0005653', 'epoch': '0.06542'}

  7%|▋         | 270/4000 [18:41<4:21:23,  4.20s/it]
  7%|▋         | 271/4000 [18:45<4:19:14,  4.17s/it]
  7%|▋         | 272/4000 [18:49<4:17:45,  4.15s/it]
  7%|▋         | 273/4000 [18:53<4:16:32,  4.13s/it]
  7%|▋         | 274/4000 [18:57<4:15:53,  4.12s/it]
  7%|▋         | 275/4000 [19:01<4:15:24,  4.11s/it]
  7%|▋         | 276/4000 [19:06<4:21:28,  4.21s/it]
  7%|▋         | 277/4000 [19:10<4:19:08,  4.18s/it]
  7%|▋         | 278/4000 [19:14<4:17:37,  4.15s/it]
  7%|▋         | 279/4000 [19:18<4:16:21,  4.13s/it]
  7%|▋         | 280/4000 [19:22<4:15:38,  4.12s/it]
                                                    
{'loss': '1.113', 'grad_norm': '0.2803', 'learning_rate': '0.0005638', 'epoch': '0.06784'}

  7%|▋         | 280/4000 [19:22<4:15:38,  4.12s/it]
  7%|▋         | 281/4000 [19:26<4:14:59,  4.11s/it]
  7%|▋         | 282/4000 [19:30<4:14:38,  4.11s/it]
  7%|▋         | 283/4000 [19:35<4:19:35,  4.19s/it]
  7%|▋         | 284/4000 [19:39<4:17:50,  4.16s/it]
  7%|▋         | 285/4000 [19:43<4:16:32,  4.14s/it]
  7%|▋         | 286/4000 [19:47<4:15:42,  4.13s/it]
  7%|▋         | 287/4000 [19:51<4:15:06,  4.12s/it]
  7%|▋         | 288/4000 [19:55<4:14:33,  4.11s/it]
  7%|▋         | 289/4000 [19:59<4:14:12,  4.11s/it]
  7%|▋         | 290/4000 [20:04<4:20:31,  4.21s/it]
                                                    
{'loss': '1.162', 'grad_norm': '0.3629', 'learning_rate': '0.0005623', 'epoch': '0.07027'}

  7%|▋         | 290/4000 [20:04<4:20:31,  4.21s/it]
  7%|▋         | 291/4000 [20:08<4:18:26,  4.18s/it]
  7%|▋         | 292/4000 [20:12<4:16:52,  4.16s/it]
  7%|▋         | 293/4000 [20:16<4:15:44,  4.14s/it]
  7%|▋         | 294/4000 [20:20<4:14:48,  4.13s/it]
  7%|▋         | 295/4000 [20:24<4:14:18,  4.12s/it]
  7%|▋         | 296/4000 [20:28<4:17:19,  4.17s/it]
  7%|▋         | 297/4000 [20:33<4:18:51,  4.19s/it]
  7%|▋         | 298/4000 [20:37<4:17:01,  4.17s/it]
  7%|▋         | 299/4000 [20:41<4:15:36,  4.14s/it]
  8%|▊         | 300/4000 [20:45<4:14:30,  4.13s/it]
                                                    
{'loss': '1.117', 'grad_norm': '0.2806', 'learning_rate': '0.0005608', 'epoch': '0.07269'}

  8%|▊         | 300/4000 [20:45<4:14:30,  4.13s/it]
  8%|▊         | 301/4000 [20:49<4:13:55,  4.12s/it]
  8%|▊         | 302/4000 [20:53<4:13:33,  4.11s/it]
  8%|▊         | 303/4000 [20:58<4:19:20,  4.21s/it]
  8%|▊         | 304/4000 [21:02<4:17:13,  4.18s/it]
  8%|▊         | 305/4000 [21:06<4:15:38,  4.15s/it]
  8%|▊         | 306/4000 [21:10<4:14:41,  4.14s/it]
  8%|▊         | 307/4000 [21:14<4:13:49,  4.12s/it]
  8%|▊         | 308/4000 [21:18<4:13:20,  4.12s/it]
  8%|▊         | 309/4000 [21:22<4:12:46,  4.11s/it]
  8%|▊         | 310/4000 [21:27<4:19:34,  4.22s/it]
                                                    
{'loss': '1.126', 'grad_norm': '0.2639', 'learning_rate': '0.0005592', 'epoch': '0.07511'}

  8%|▊         | 310/4000 [21:27<4:19:34,  4.22s/it]
  8%|▊         | 311/4000 [21:31<4:17:11,  4.18s/it]
  8%|▊         | 312/4000 [21:35<4:15:25,  4.16s/it]
  8%|▊         | 313/4000 [21:39<4:14:03,  4.13s/it]
  8%|▊         | 314/4000 [21:43<4:13:10,  4.12s/it]
  8%|▊         | 315/4000 [21:47<4:12:26,  4.11s/it]
  8%|▊         | 316/4000 [21:51<4:12:01,  4.10s/it]
  8%|▊         | 317/4000 [21:56<4:17:13,  4.19s/it]
  8%|▊         | 318/4000 [22:00<4:15:12,  4.16s/it]
  8%|▊         | 319/4000 [22:04<4:13:41,  4.14s/it]
  8%|▊         | 320/4000 [22:08<4:12:47,  4.12s/it]
                                                    
{'loss': '1.049', 'grad_norm': '0.2683', 'learning_rate': '0.0005577', 'epoch': '0.07754'}

  8%|▊         | 320/4000 [22:08<4:12:47,  4.12s/it]
  8%|▊         | 321/4000 [22:12<4:12:10,  4.11s/it]
  8%|▊         | 322/4000 [22:16<4:11:41,  4.11s/it]
  8%|▊         | 323/4000 [22:20<4:14:10,  4.15s/it]
  8%|▊         | 324/4000 [22:24<4:16:21,  4.18s/it]
  8%|▊         | 325/4000 [22:29<4:14:33,  4.16s/it]
  8%|▊         | 326/4000 [22:33<4:13:19,  4.14s/it]
  8%|▊         | 327/4000 [22:37<4:12:25,  4.12s/it]
  8%|▊         | 328/4000 [22:41<4:11:44,  4.11s/it]
  8%|▊         | 329/4000 [22:45<4:11:12,  4.11s/it]
  8%|▊         | 330/4000 [22:49<4:16:56,  4.20s/it]
                                                    
{'loss': '1.116', 'grad_norm': '0.2458', 'learning_rate': '0.0005562', 'epoch': '0.07996'}

  8%|▊         | 330/4000 [22:49<4:16:56,  4.20s/it]
  8%|▊         | 331/4000 [22:53<4:15:01,  4.17s/it]
  8%|▊         | 332/4000 [22:58<4:13:29,  4.15s/it]
  8%|▊         | 333/4000 [23:02<4:12:41,  4.13s/it]
  8%|▊         | 334/4000 [23:06<4:12:04,  4.13s/it]
  8%|▊         | 335/4000 [23:10<4:11:33,  4.12s/it]
  8%|▊         | 336/4000 [23:14<4:11:08,  4.11s/it]
  8%|▊         | 337/4000 [23:18<4:17:13,  4.21s/it]
  8%|▊         | 338/4000 [23:23<4:15:01,  4.18s/it]
  8%|▊         | 339/4000 [23:27<4:13:30,  4.15s/it]
  8%|▊         | 340/4000 [23:31<4:12:36,  4.14s/it]
                                                    
{'loss': '1.064', 'grad_norm': '0.2021', 'learning_rate': '0.0005547', 'epoch': '0.08238'}

  8%|▊         | 340/4000 [23:31<4:12:36,  4.14s/it]
  9%|▊         | 341/4000 [23:35<4:11:48,  4.13s/it]
  9%|▊         | 342/4000 [23:39<4:11:19,  4.12s/it]
  9%|▊         | 343/4000 [23:43<4:10:43,  4.11s/it]
  9%|▊         | 344/4000 [23:47<4:16:51,  4.22s/it]
  9%|▊         | 345/4000 [23:52<4:14:38,  4.18s/it]
  9%|▊         | 346/4000 [23:56<4:13:07,  4.16s/it]
  9%|▊         | 347/4000 [24:00<4:12:03,  4.14s/it]
  9%|▊         | 348/4000 [24:04<4:11:15,  4.13s/it]
  9%|▊         | 349/4000 [24:08<4:10:32,  4.12s/it]
  9%|▉         | 350/4000 [24:12<4:15:49,  4.21s/it]
                                                    
{'loss': '1.056', 'grad_norm': '0.2233', 'learning_rate': '0.0005532', 'epoch': '0.0848'}

  9%|▉         | 350/4000 [24:12<4:15:49,  4.21s/it]
  9%|▉         | 351/4000 [24:16<4:13:43,  4.17s/it]
  9%|▉         | 352/4000 [24:21<4:12:09,  4.15s/it]
  9%|▉         | 353/4000 [24:25<4:11:05,  4.13s/it]
  9%|▉         | 354/4000 [24:29<4:10:23,  4.12s/it]
  9%|▉         | 355/4000 [24:33<4:09:44,  4.11s/it]
  9%|▉         | 356/4000 [24:37<4:09:24,  4.11s/it]
  9%|▉         | 357/4000 [24:41<4:15:41,  4.21s/it]
  9%|▉         | 358/4000 [24:45<4:13:29,  4.18s/it]
  9%|▉         | 359/4000 [24:50<4:11:42,  4.15s/it]
  9%|▉         | 360/4000 [24:54<4:10:44,  4.13s/it]
                                                    
{'loss': '1.067', 'grad_norm': '0.2488', 'learning_rate': '0.0005517', 'epoch': '0.08723'}

  9%|▉         | 360/4000 [24:54<4:10:44,  4.13s/it]
  9%|▉         | 361/4000 [24:58<4:09:55,  4.12s/it]
  9%|▉         | 362/4000 [25:02<4:09:18,  4.11s/it]
  9%|▉         | 363/4000 [25:06<4:09:08,  4.11s/it]
  9%|▉         | 364/4000 [25:10<4:14:42,  4.20s/it]
  9%|▉         | 365/4000 [25:14<4:12:35,  4.17s/it]
  9%|▉         | 366/4000 [25:19<4:11:01,  4.14s/it]
  9%|▉         | 367/4000 [25:23<4:09:58,  4.13s/it]
  9%|▉         | 368/4000 [25:27<4:09:10,  4.12s/it]
  9%|▉         | 369/4000 [25:31<4:08:32,  4.11s/it]
  9%|▉         | 370/4000 [25:35<4:08:18,  4.10s/it]
                                                    
{'loss': '1.039', 'grad_norm': '0.2644', 'learning_rate': '0.0005502', 'epoch': '0.08965'}

  9%|▉         | 370/4000 [25:35<4:08:18,  4.10s/it]
  9%|▉         | 371/4000 [25:39<4:14:43,  4.21s/it]
  9%|▉         | 372/4000 [25:43<4:12:28,  4.18s/it]
  9%|▉         | 373/4000 [25:48<4:10:47,  4.15s/it]
  9%|▉         | 374/4000 [25:52<4:09:34,  4.13s/it]
  9%|▉         | 375/4000 [25:56<4:08:52,  4.12s/it]
  9%|▉         | 376/4000 [26:00<4:08:07,  4.11s/it]
  9%|▉         | 377/4000 [26:04<4:13:59,  4.21s/it]
  9%|▉         | 378/4000 [26:08<4:11:47,  4.17s/it]
  9%|▉         | 379/4000 [26:12<4:10:30,  4.15s/it]
 10%|▉         | 380/4000 [26:17<4:09:22,  4.13s/it]
                                                    
{'loss': '1.02', 'grad_norm': '0.2621', 'learning_rate': '0.0005486', 'epoch': '0.09207'}

 10%|▉         | 380/4000 [26:17<4:09:22,  4.13s/it]
 10%|▉         | 381/4000 [26:21<4:08:34,  4.12s/it]
 10%|▉         | 382/4000 [26:25<4:07:59,  4.11s/it]
 10%|▉         | 383/4000 [26:29<4:07:43,  4.11s/it]
 10%|▉         | 384/4000 [26:33<4:13:46,  4.21s/it]
 10%|▉         | 385/4000 [26:37<4:11:30,  4.17s/it]
 10%|▉         | 386/4000 [26:41<4:09:58,  4.15s/it]
 10%|▉         | 387/4000 [26:46<4:08:59,  4.13s/it]
 10%|▉         | 388/4000 [26:50<4:08:02,  4.12s/it]
 10%|▉         | 389/4000 [26:54<4:07:20,  4.11s/it]
 10%|▉         | 390/4000 [26:58<4:06:52,  4.10s/it]
                                                    
{'loss': '0.9707', 'grad_norm': '0.2696', 'learning_rate': '0.0005471', 'epoch': '0.0945'}

 10%|▉         | 390/4000 [26:58<4:06:52,  4.10s/it]
 10%|▉         | 391/4000 [27:02<4:12:38,  4.20s/it]
 10%|▉         | 392/4000 [27:06<4:10:38,  4.17s/it]
 10%|▉         | 393/4000 [27:10<4:09:06,  4.14s/it]
 10%|▉         | 394/4000 [27:14<4:07:53,  4.12s/it]
 10%|▉         | 395/4000 [27:19<4:07:10,  4.11s/it]
 10%|▉         | 396/4000 [27:23<4:06:34,  4.11s/it]
 10%|▉         | 397/4000 [27:27<4:09:34,  4.16s/it]
 10%|▉         | 398/4000 [27:31<4:11:09,  4.18s/it]
 10%|▉         | 399/4000 [27:35<4:09:32,  4.16s/it]
 10%|█         | 400/4000 [27:39<4:08:22,  4.14s/it]
                                                    
{'loss': '0.9642', 'grad_norm': '0.2032', 'learning_rate': '0.0005456', 'epoch': '0.09692'}

 10%|█         | 400/4000 [27:39<4:08:22,  4.14s/it]
 10%|█         | 401/4000 [27:43<4:07:21,  4.12s/it]
 10%|█         | 402/4000 [27:48<4:06:41,  4.11s/it]
 10%|█         | 403/4000 [27:52<4:06:12,  4.11s/it]
 10%|█         | 404/4000 [27:56<4:12:35,  4.21s/it]
 10%|█         | 405/4000 [28:00<4:10:15,  4.18s/it]
 10%|█         | 406/4000 [28:04<4:08:28,  4.15s/it]
 10%|█         | 407/4000 [28:08<4:07:28,  4.13s/it]
 10%|█         | 408/4000 [28:12<4:06:42,  4.12s/it]
 10%|█         | 409/4000 [28:17<4:06:11,  4.11s/it]
 10%|█         | 410/4000 [28:21<4:05:39,  4.11s/it]
                                                    
{'loss': '1.005', 'grad_norm': '0.2873', 'learning_rate': '0.0005441', 'epoch': '0.09934'}

 10%|█         | 410/4000 [28:21<4:05:39,  4.11s/it]
 10%|█         | 411/4000 [28:25<4:11:33,  4.21s/it]
 10%|█         | 412/4000 [28:29<4:09:37,  4.17s/it]
 10%|█         | 413/4000 [28:33<4:07:59,  4.15s/it]
 10%|█         | 414/4000 [28:37<4:06:43,  4.13s/it]
 10%|█         | 415/4000 [28:41<4:05:57,  4.12s/it]
 10%|█         | 416/4000 [28:46<4:05:18,  4.11s/it]
 10%|█         | 417/4000 [28:50<4:04:50,  4.10s/it]
 10%|█         | 418/4000 [28:54<4:10:06,  4.19s/it]
 10%|█         | 419/4000 [28:58<4:08:11,  4.16s/it]
 10%|█         | 420/4000 [29:02<4:06:57,  4.14s/it]
                                                    
{'loss': '0.9969', 'grad_norm': '0.1983', 'learning_rate': '0.0005426', 'epoch': '0.1018'}

 10%|█         | 420/4000 [29:02<4:06:57,  4.14s/it]
 11%|█         | 421/4000 [29:06<4:06:02,  4.12s/it]
 11%|█         | 422/4000 [29:10<4:05:27,  4.12s/it]
 11%|█         | 423/4000 [29:15<4:05:16,  4.11s/it]
 11%|█         | 424/4000 [29:19<4:05:00,  4.11s/it]
 11%|█         | 425/4000 [29:23<4:11:23,  4.22s/it]
 11%|█         | 426/4000 [29:27<4:09:24,  4.19s/it]
 11%|█         | 427/4000 [29:31<4:08:09,  4.17s/it]
 11%|█         | 428/4000 [29:35<4:07:06,  4.15s/it]
 11%|█         | 429/4000 [29:40<4:06:24,  4.14s/it]
 11%|█         | 430/4000 [29:44<4:05:55,  4.13s/it]
                                                    
{'loss': '0.9302', 'grad_norm': '0.1899', 'learning_rate': '0.0005411', 'epoch': '0.1042'}

 11%|█         | 430/4000 [29:44<4:05:55,  4.13s/it]
 11%|█         | 431/4000 [29:48<4:10:57,  4.22s/it]
 11%|█         | 432/4000 [29:52<4:08:59,  4.19s/it]
 11%|█         | 433/4000 [29:56<4:07:31,  4.16s/it]
 11%|█         | 434/4000 [30:00<4:06:39,  4.15s/it]
 11%|█         | 435/4000 [30:05<4:06:05,  4.14s/it]
 11%|█         | 436/4000 [30:09<4:05:08,  4.13s/it]
 11%|█         | 437/4000 [30:13<4:04:31,  4.12s/it]
 11%|█         | 438/4000 [30:17<4:12:09,  4.25s/it]
 11%|█         | 439/4000 [30:21<4:09:36,  4.21s/it]
 11%|█         | 440/4000 [30:26<4:08:02,  4.18s/it]
                                                    
{'loss': '0.9665', 'grad_norm': '0.2194', 'learning_rate': '0.0005395', 'epoch': '0.1066'}

 11%|█         | 440/4000 [30:26<4:08:02,  4.18s/it]
 11%|█         | 441/4000 [30:30<4:06:38,  4.16s/it]
 11%|█         | 442/4000 [30:34<4:05:51,  4.15s/it]
 11%|█         | 443/4000 [30:38<4:05:01,  4.13s/it]
 11%|█         | 444/4000 [30:42<4:04:26,  4.12s/it]
 11%|█         | 445/4000 [30:46<4:10:01,  4.22s/it]
 11%|█         | 446/4000 [30:50<4:07:55,  4.19s/it]
 11%|█         | 447/4000 [30:55<4:06:13,  4.16s/it]
 11%|█         | 448/4000 [30:59<4:05:14,  4.14s/it]
 11%|█         | 449/4000 [31:03<4:06:00,  4.16s/it]
 11%|█▏        | 450/4000 [31:07<4:05:34,  4.15s/it]
                                                    
{'loss': '0.9361', 'grad_norm': '0.232', 'learning_rate': '0.000538', 'epoch': '0.109'}

 11%|█▏        | 450/4000 [31:07<4:05:34,  4.15s/it]
 11%|█▏        | 451/4000 [31:11<4:07:10,  4.18s/it]
 11%|█▏        | 452/4000 [31:16<4:09:29,  4.22s/it]
 11%|█▏        | 453/4000 [31:20<4:07:40,  4.19s/it]
 11%|█▏        | 454/4000 [31:24<4:06:09,  4.17s/it]
 11%|█▏        | 455/4000 [31:28<4:05:25,  4.15s/it]
 11%|█▏        | 456/4000 [31:32<4:04:42,  4.14s/it]
 11%|█▏        | 457/4000 [31:36<4:04:31,  4.14s/it]
 11%|█▏        | 458/4000 [31:41<4:11:03,  4.25s/it]
 11%|█▏        | 459/4000 [31:45<4:08:55,  4.22s/it]
 12%|█▏        | 460/4000 [31:49<4:07:20,  4.19s/it]
                                                    
{'loss': '0.9602', 'grad_norm': '0.2184', 'learning_rate': '0.0005365', 'epoch': '0.1115'}

 12%|█▏        | 460/4000 [31:49<4:07:20,  4.19s/it]
 12%|█▏        | 461/4000 [31:53<4:06:08,  4.17s/it]
 12%|█▏        | 462/4000 [31:57<4:05:09,  4.16s/it]
 12%|█▏        | 463/4000 [32:01<4:04:41,  4.15s/it]
 12%|█▏        | 464/4000 [32:05<4:04:11,  4.14s/it]
 12%|█▏        | 465/4000 [32:10<4:09:12,  4.23s/it]
 12%|█▏        | 466/4000 [32:14<4:07:08,  4.20s/it]
 12%|█▏        | 467/4000 [32:18<4:05:40,  4.17s/it]
 12%|█▏        | 468/4000 [32:22<4:05:05,  4.16s/it]
 12%|█▏        | 469/4000 [32:26<4:04:18,  4.15s/it]
 12%|█▏        | 470/4000 [32:31<4:03:56,  4.15s/it]
                                                    
{'loss': '0.8988', 'grad_norm': '0.2376', 'learning_rate': '0.000535', 'epoch': '0.1139'}

 12%|█▏        | 470/4000 [32:31<4:03:56,  4.15s/it]
 12%|█▏        | 471/4000 [32:35<4:03:56,  4.15s/it]
 12%|█▏        | 472/4000 [32:39<4:10:09,  4.25s/it]
 12%|█▏        | 473/4000 [32:43<4:07:55,  4.22s/it]
 12%|█▏        | 474/4000 [32:47<4:06:14,  4.19s/it]
 12%|█▏        | 475/4000 [32:52<4:05:14,  4.17s/it]
 12%|█▏        | 476/4000 [32:56<4:04:08,  4.16s/it]
 12%|█▏        | 477/4000 [33:00<4:03:26,  4.15s/it]
 12%|█▏        | 478/4000 [33:04<4:09:43,  4.25s/it]
 12%|█▏        | 479/4000 [33:08<4:07:33,  4.22s/it]
 12%|█▏        | 480/4000 [33:13<4:05:45,  4.19s/it]
                                                    
{'loss': '0.8198', 'grad_norm': '0.2114', 'learning_rate': '0.0005335', 'epoch': '0.1163'}

 12%|█▏        | 480/4000 [33:13<4:05:45,  4.19s/it]
 12%|█▏        | 481/4000 [33:17<4:04:21,  4.17s/it]
 12%|█▏        | 482/4000 [33:21<4:03:23,  4.15s/it]
 12%|█▏        | 483/4000 [33:25<4:02:42,  4.14s/it]
 12%|█▏        | 484/4000 [33:29<4:02:29,  4.14s/it]
 12%|█▏        | 485/4000 [33:33<4:07:35,  4.23s/it]
 12%|█▏        | 486/4000 [33:38<4:05:56,  4.20s/it]
 12%|█▏        | 487/4000 [33:42<4:04:26,  4.17s/it]
 12%|█▏        | 488/4000 [33:46<4:03:21,  4.16s/it]
 12%|█▏        | 489/4000 [33:50<4:02:54,  4.15s/it]
 12%|█▏        | 490/4000 [33:54<4:02:22,  4.14s/it]
                                                    
{'loss': '0.8315', 'grad_norm': '0.1903', 'learning_rate': '0.000532', 'epoch': '0.1187'}

 12%|█▏        | 490/4000 [33:54<4:02:22,  4.14s/it]
 12%|█▏        | 491/4000 [33:58<4:02:09,  4.14s/it]
 12%|█▏        | 492/4000 [34:03<4:08:39,  4.25s/it]
 12%|█▏        | 493/4000 [34:07<4:06:13,  4.21s/it]
 12%|█▏        | 494/4000 [34:11<4:04:42,  4.19s/it]
 12%|█▏        | 495/4000 [34:15<4:03:32,  4.17s/it]
 12%|█▏        | 496/4000 [34:19<4:02:49,  4.16s/it]
 12%|█▏        | 497/4000 [34:23<4:02:04,  4.15s/it]
 12%|█▏        | 498/4000 [34:28<4:01:56,  4.15s/it]
 12%|█▏        | 499/4000 [34:32<4:07:15,  4.24s/it]
 12%|█▎        | 500/4000 [34:36<4:05:27,  4.21s/it]
                                                    
{'loss': '0.8515', 'grad_norm': '0.2477', 'learning_rate': '0.0005305', 'epoch': '0.1211'}

 12%|█▎        | 500/4000 [34:36<4:05:27,  4.21s/it]
 13%|█▎        | 501/4000 [34:40<4:04:12,  4.19s/it]
 13%|█▎        | 502/4000 [34:44<4:03:22,  4.17s/it]
 13%|█▎        | 503/4000 [34:49<4:02:14,  4.16s/it]
 13%|█▎        | 504/4000 [34:53<4:01:50,  4.15s/it]
 13%|█▎        | 505/4000 [34:57<4:04:23,  4.20s/it]
 13%|█▎        | 506/4000 [35:01<4:06:37,  4.24s/it]
 13%|█▎        | 507/4000 [35:05<4:04:32,  4.20s/it]
 13%|█▎        | 508/4000 [35:10<4:03:22,  4.18s/it]
 13%|█▎        | 509/4000 [35:14<4:02:11,  4.16s/it]
 13%|█▎        | 510/4000 [35:18<4:01:42,  4.16s/it]
                                                    
{'loss': '0.8253', 'grad_norm': '0.1977', 'learning_rate': '0.0005289', 'epoch': '0.1236'}

 13%|█▎        | 510/4000 [35:18<4:01:42,  4.16s/it]
 13%|█▎        | 511/4000 [35:22<4:00:59,  4.14s/it]
 13%|█▎        | 512/4000 [35:26<4:06:49,  4.25s/it]
 13%|█▎        | 513/4000 [35:31<4:04:45,  4.21s/it]
 13%|█▎        | 514/4000 [35:35<4:03:11,  4.19s/it]
 13%|█▎        | 515/4000 [35:39<4:01:53,  4.16s/it]
 13%|█▎        | 516/4000 [35:43<4:01:20,  4.16s/it]
 13%|█▎        | 517/4000 [35:47<4:00:38,  4.15s/it]
 13%|█▎        | 518/4000 [35:51<4:00:28,  4.14s/it]
 13%|█▎        | 519/4000 [35:56<4:05:44,  4.24s/it]
 13%|█▎        | 520/4000 [36:00<4:03:55,  4.21s/it]
                                                    
{'loss': '0.8197', 'grad_norm': '0.2504', 'learning_rate': '0.0005274', 'epoch': '0.126'}

 13%|█▎        | 520/4000 [36:00<4:03:55,  4.21s/it]
 13%|█▎        | 521/4000 [36:04<4:02:24,  4.18s/it]
 13%|█▎        | 522/4000 [36:08<4:01:08,  4.16s/it]
 13%|█▎        | 523/4000 [36:12<4:00:48,  4.16s/it]
 13%|█▎        | 524/4000 [36:16<4:00:30,  4.15s/it]
 13%|█▎        | 525/4000 [36:20<3:59:49,  4.14s/it]
 13%|█▎        | 526/4000 [36:25<4:06:01,  4.25s/it]
 13%|█▎        | 527/4000 [36:29<4:04:08,  4.22s/it]
 13%|█▎        | 528/4000 [36:33<4:02:19,  4.19s/it]
 13%|█▎        | 529/4000 [36:37<4:01:01,  4.17s/it]
 13%|█▎        | 530/4000 [36:41<4:00:23,  4.16s/it]
                                                    
{'loss': '0.8134', 'grad_norm': '0.1738', 'learning_rate': '0.0005259', 'epoch': '0.1284'}

 13%|█▎        | 530/4000 [36:41<4:00:23,  4.16s/it]
 13%|█▎        | 531/4000 [36:46<3:59:42,  4.15s/it]
 13%|█▎        | 532/4000 [36:50<4:04:52,  4.24s/it]
 13%|█▎        | 533/4000 [36:54<4:02:46,  4.20s/it]
 13%|█▎        | 534/4000 [36:58<4:01:35,  4.18s/it]
 13%|█▎        | 535/4000 [37:02<4:00:26,  4.16s/it]
 13%|█▎        | 536/4000 [37:07<4:00:07,  4.16s/it]
 13%|█▎        | 537/4000 [37:11<4:01:48,  4.19s/it]
 13%|█▎        | 538/4000 [37:15<4:01:56,  4.19s/it]
 13%|█▎        | 539/4000 [37:20<4:08:53,  4.31s/it]
 14%|█▎        | 540/4000 [37:24<4:07:56,  4.30s/it]
                                                    
{'loss': '0.7794', 'grad_norm': '0.2064', 'learning_rate': '0.0005244', 'epoch': '0.1308'}

 14%|█▎        | 540/4000 [37:24<4:07:56,  4.30s/it]
 14%|█▎        | 541/4000 [37:28<4:07:28,  4.29s/it]
 14%|█▎        | 542/4000 [37:32<4:04:54,  4.25s/it]
 14%|█▎        | 543/4000 [37:36<4:03:24,  4.22s/it]
 14%|█▎        | 544/4000 [37:41<4:02:25,  4.21s/it]
 14%|█▎        | 545/4000 [37:45<4:01:26,  4.19s/it]
 14%|█▎        | 546/4000 [37:49<4:07:07,  4.29s/it]
 14%|█▎        | 547/4000 [37:53<4:04:39,  4.25s/it]
 14%|█▎        | 548/4000 [37:58<4:02:56,  4.22s/it]
 14%|█▎        | 549/4000 [38:02<4:01:33,  4.20s/it]
 14%|█▍        | 550/4000 [38:06<4:00:33,  4.18s/it]
                                                    
{'loss': '0.8368', 'grad_norm': '0.2211', 'learning_rate': '0.0005229', 'epoch': '0.1333'}

 14%|█▍        | 550/4000 [38:06<4:00:33,  4.18s/it]
 14%|█▍        | 551/4000 [38:10<4:00:12,  4.18s/it]
 14%|█▍        | 552/4000 [38:14<4:02:41,  4.22s/it]
 14%|█▍        | 553/4000 [38:19<4:05:13,  4.27s/it]
 14%|█▍        | 554/4000 [38:23<4:03:03,  4.23s/it]
 14%|█▍        | 555/4000 [38:27<4:01:23,  4.20s/it]
 14%|█▍        | 556/4000 [38:31<4:00:14,  4.19s/it]
 14%|█▍        | 557/4000 [38:35<3:59:36,  4.18s/it]
 14%|█▍        | 558/4000 [38:39<3:59:04,  4.17s/it]
 14%|█▍        | 559/4000 [38:44<4:05:51,  4.29s/it]
 14%|█▍        | 560/4000 [38:48<4:03:14,  4.24s/it]
                                                    
{'loss': '0.7846', 'grad_norm': '0.1814', 'learning_rate': '0.0005214', 'epoch': '0.1357'}

 14%|█▍        | 560/4000 [38:48<4:03:14,  4.24s/it]
 14%|█▍        | 561/4000 [38:52<4:01:12,  4.21s/it]
 14%|█▍        | 562/4000 [38:56<3:59:57,  4.19s/it]
 14%|█▍        | 563/4000 [39:01<3:59:12,  4.18s/it]
 14%|█▍        | 564/4000 [39:05<3:58:20,  4.16s/it]
 14%|█▍        | 565/4000 [39:09<3:57:38,  4.15s/it]
 14%|█▍        | 566/4000 [39:13<4:03:40,  4.26s/it]
 14%|█▍        | 567/4000 [39:18<4:01:43,  4.22s/it]
 14%|█▍        | 568/4000 [39:22<4:00:10,  4.20s/it]
 14%|█▍        | 569/4000 [39:26<3:59:04,  4.18s/it]
 14%|█▍        | 570/4000 [39:30<3:58:16,  4.17s/it]
                                                    
{'loss': '0.8119', 'grad_norm': '0.3592', 'learning_rate': '0.0005198', 'epoch': '0.1381'}

 14%|█▍        | 570/4000 [39:30<3:58:16,  4.17s/it]
 14%|█▍        | 571/4000 [39:34<3:57:28,  4.16s/it]
 14%|█▍        | 572/4000 [39:38<3:57:07,  4.15s/it]
 14%|█▍        | 573/4000 [39:43<4:03:24,  4.26s/it]
 14%|█▍        | 574/4000 [39:47<4:01:06,  4.22s/it]
 14%|█▍        | 575/4000 [39:51<3:59:09,  4.19s/it]
 14%|█▍        | 576/4000 [39:55<3:58:21,  4.18s/it]
 14%|█▍        | 577/4000 [39:59<3:57:39,  4.17s/it]
 14%|█▍        | 578/4000 [40:03<3:57:13,  4.16s/it]
 14%|█▍        | 579/4000 [40:08<4:01:30,  4.24s/it]
 14%|█▍        | 580/4000 [40:12<4:04:47,  4.29s/it]
                                                    
{'loss': '0.8114', 'grad_norm': '0.176', 'learning_rate': '0.0005183', 'epoch': '0.1405'}

 14%|█▍        | 580/4000 [40:12<4:04:47,  4.29s/it]
 15%|█▍        | 581/4000 [40:16<4:03:06,  4.27s/it]
 15%|█▍        | 582/4000 [40:21<4:01:40,  4.24s/it]
 15%|█▍        | 583/4000 [40:25<4:02:36,  4.26s/it]
 15%|█▍        | 584/4000 [40:29<4:02:05,  4.25s/it]
 15%|█▍        | 585/4000 [40:33<3:59:48,  4.21s/it]
 15%|█▍        | 586/4000 [40:38<4:04:40,  4.30s/it]
 15%|█▍        | 587/4000 [40:42<4:01:28,  4.25s/it]
 15%|█▍        | 588/4000 [40:46<3:59:44,  4.22s/it]
 15%|█▍        | 589/4000 [40:50<3:58:20,  4.19s/it]
 15%|█▍        | 590/4000 [40:54<3:57:02,  4.17s/it]
                                                    
{'loss': '0.7472', 'grad_norm': '0.1836', 'learning_rate': '0.0005168', 'epoch': '0.143'}

 15%|█▍        | 590/4000 [40:54<3:57:02,  4.17s/it]
 15%|█▍        | 591/4000 [40:58<3:56:13,  4.16s/it]
 15%|█▍        | 592/4000 [41:03<3:55:38,  4.15s/it]
 15%|█▍        | 593/4000 [41:07<4:01:42,  4.26s/it]
 15%|█▍        | 594/4000 [41:11<3:59:28,  4.22s/it]
 15%|█▍        | 595/4000 [41:15<3:57:42,  4.19s/it]
 15%|█▍        | 596/4000 [41:19<3:56:27,  4.17s/it]
 15%|█▍        | 597/4000 [41:24<3:55:45,  4.16s/it]
 15%|█▍        | 598/4000 [41:28<3:55:19,  4.15s/it]
 15%|█▍        | 599/4000 [41:32<3:54:58,  4.15s/it]
 15%|█▌        | 600/4000 [41:36<4:00:47,  4.25s/it]
                                                    
{'loss': '0.7377', 'grad_norm': '0.1711', 'learning_rate': '0.0005153', 'epoch': '0.1454'}

 15%|█▌        | 600/4000 [41:36<4:00:47,  4.25s/it]
 15%|█▌        | 601/4000 [41:40<3:58:49,  4.22s/it]
 15%|█▌        | 602/4000 [41:45<3:57:23,  4.19s/it]
 15%|█▌        | 603/4000 [41:49<3:56:10,  4.17s/it]
 15%|█▌        | 604/4000 [41:53<3:55:28,  4.16s/it]
 15%|█▌        | 605/4000 [41:57<3:54:50,  4.15s/it]
 15%|█▌        | 606/4000 [42:01<3:57:24,  4.20s/it]
 15%|█▌        | 607/4000 [42:06<3:59:43,  4.24s/it]
 15%|█▌        | 608/4000 [42:10<3:57:55,  4.21s/it]
 15%|█▌        | 609/4000 [42:14<3:56:23,  4.18s/it]
 15%|█▌        | 610/4000 [42:18<3:55:32,  4.17s/it]
                                                    
{'loss': '0.7716', 'grad_norm': '0.2212', 'learning_rate': '0.0005138', 'epoch': '0.1478'}

 15%|█▌        | 610/4000 [42:18<3:55:32,  4.17s/it]
 15%|█▌        | 611/4000 [42:22<3:54:46,  4.16s/it]
 15%|█▌        | 612/4000 [42:26<3:54:12,  4.15s/it]
 15%|█▌        | 613/4000 [42:31<4:00:03,  4.25s/it]
 15%|█▌        | 614/4000 [42:35<3:57:45,  4.21s/it]
 15%|█▌        | 615/4000 [42:39<3:56:20,  4.19s/it]
 15%|█▌        | 616/4000 [42:43<3:55:56,  4.18s/it]
 15%|█▌        | 617/4000 [42:47<3:55:57,  4.19s/it]
 15%|█▌        | 618/4000 [42:52<3:55:24,  4.18s/it]
 15%|█▌        | 619/4000 [42:56<3:55:12,  4.17s/it]
 16%|█▌        | 620/4000 [43:00<4:02:11,  4.30s/it]
                                                    
{'loss': '0.7474', 'grad_norm': '0.1852', 'learning_rate': '0.0005123', 'epoch': '0.1502'}

 16%|█▌        | 620/4000 [43:00<4:02:11,  4.30s/it]
 16%|█▌        | 621/4000 [43:05<4:00:32,  4.27s/it]
 16%|█▌        | 622/4000 [43:09<3:58:00,  4.23s/it]
 16%|█▌        | 623/4000 [43:13<3:56:20,  4.20s/it]
 16%|█▌        | 624/4000 [43:17<3:55:32,  4.19s/it]
 16%|█▌        | 625/4000 [43:21<3:54:36,  4.17s/it]
 16%|█▌        | 626/4000 [43:25<3:53:57,  4.16s/it]
 16%|█▌        | 627/4000 [43:30<3:59:30,  4.26s/it]
 16%|█▌        | 628/4000 [43:34<3:57:18,  4.22s/it]
 16%|█▌        | 629/4000 [43:38<3:55:49,  4.20s/it]
 16%|█▌        | 630/4000 [43:42<3:54:42,  4.18s/it]
                                                    
{'loss': '0.7208', 'grad_norm': '0.1889', 'learning_rate': '0.0005108', 'epoch': '0.1526'}

 16%|█▌        | 630/4000 [43:42<3:54:42,  4.18s/it]
 16%|█▌        | 631/4000 [43:46<3:53:49,  4.16s/it]
 16%|█▌        | 632/4000 [43:50<3:53:17,  4.16s/it]
 16%|█▌        | 633/4000 [43:55<3:58:31,  4.25s/it]
 16%|█▌        | 634/4000 [43:59<3:56:38,  4.22s/it]
 16%|█▌        | 635/4000 [44:03<3:55:25,  4.20s/it]
 16%|█▌        | 636/4000 [44:07<3:54:30,  4.18s/it]
 16%|█▌        | 637/4000 [44:11<3:53:39,  4.17s/it]
 16%|█▌        | 638/4000 [44:16<3:53:14,  4.16s/it]
 16%|█▌        | 639/4000 [44:20<3:54:44,  4.19s/it]
 16%|█▌        | 640/4000 [44:24<4:01:01,  4.30s/it]
                                                    
{'loss': '0.7037', 'grad_norm': '0.1678', 'learning_rate': '0.0005092', 'epoch': '0.1551'}

 16%|█▌        | 640/4000 [44:24<4:01:01,  4.30s/it]
 16%|█▌        | 641/4000 [44:29<3:59:05,  4.27s/it]
 16%|█▌        | 642/4000 [44:33<3:57:50,  4.25s/it]
 16%|█▌        | 643/4000 [44:37<3:57:47,  4.25s/it]
 16%|█▌        | 644/4000 [44:41<3:55:53,  4.22s/it]
 16%|█▌        | 645/4000 [44:45<3:54:35,  4.20s/it]
 16%|█▌        | 646/4000 [44:49<3:53:34,  4.18s/it]
 16%|█▌        | 647/4000 [44:54<3:59:05,  4.28s/it]
 16%|█▌        | 648/4000 [44:58<3:56:55,  4.24s/it]
 16%|█▌        | 649/4000 [45:02<3:55:02,  4.21s/it]
 16%|█▋        | 650/4000 [45:06<3:53:48,  4.19s/it]
                                                    
{'loss': '0.7125', 'grad_norm': '0.2202', 'learning_rate': '0.0005077', 'epoch': '0.1575'}

 16%|█▋        | 650/4000 [45:06<3:53:48,  4.19s/it]
 16%|█▋        | 651/4000 [45:11<3:52:52,  4.17s/it]
 16%|█▋        | 652/4000 [45:15<3:52:16,  4.16s/it]
 16%|█▋        | 653/4000 [45:19<3:52:24,  4.17s/it]
 16%|█▋        | 654/4000 [45:23<3:57:47,  4.26s/it]
 16%|█▋        | 655/4000 [45:27<3:55:34,  4.23s/it]
 16%|█▋        | 656/4000 [45:32<3:54:18,  4.20s/it]
 16%|█▋        | 657/4000 [45:36<3:52:54,  4.18s/it]
 16%|█▋        | 658/4000 [45:40<3:51:52,  4.16s/it]
 16%|█▋        | 659/4000 [45:44<3:51:32,  4.16s/it]
 16%|█▋        | 660/4000 [45:49<3:57:32,  4.27s/it]
                                                    
{'loss': '0.7117', 'grad_norm': '0.1797', 'learning_rate': '0.0005062', 'epoch': '0.1599'}

 16%|█▋        | 660/4000 [45:49<3:57:32,  4.27s/it]
 17%|█▋        | 661/4000 [45:53<3:55:14,  4.23s/it]
 17%|█▋        | 662/4000 [45:57<3:53:35,  4.20s/it]
 17%|█▋        | 663/4000 [46:01<3:52:37,  4.18s/it]
 17%|█▋        | 664/4000 [46:05<3:53:21,  4.20s/it]
 17%|█▋        | 665/4000 [46:09<3:53:34,  4.20s/it]
 17%|█▋        | 666/4000 [46:14<3:53:06,  4.19s/it]
 17%|█▋        | 667/4000 [46:18<3:58:25,  4.29s/it]
 17%|█▋        | 668/4000 [46:22<3:58:32,  4.30s/it]
 17%|█▋        | 669/4000 [46:27<3:57:14,  4.27s/it]
 17%|█▋        | 670/4000 [46:31<3:54:46,  4.23s/it]
                                                    
{'loss': '0.7337', 'grad_norm': '0.1529', 'learning_rate': '0.0005047', 'epoch': '0.1623'}

 17%|█▋        | 670/4000 [46:31<3:54:46,  4.23s/it]
 17%|█▋        | 671/4000 [46:35<3:52:57,  4.20s/it]
 17%|█▋        | 672/4000 [46:39<3:51:51,  4.18s/it]
 17%|█▋        | 673/4000 [46:43<3:50:42,  4.16s/it]
 17%|█▋        | 674/4000 [46:48<3:56:20,  4.26s/it]
 17%|█▋        | 675/4000 [46:52<3:53:51,  4.22s/it]
 17%|█▋        | 676/4000 [46:56<3:52:30,  4.20s/it]
 17%|█▋        | 677/4000 [47:00<3:51:13,  4.17s/it]
 17%|█▋        | 678/4000 [47:04<3:50:25,  4.16s/it]
 17%|█▋        | 679/4000 [47:08<3:50:06,  4.16s/it]
 17%|█▋        | 680/4000 [47:13<3:52:58,  4.21s/it]
                                                    
{'loss': '0.724', 'grad_norm': '0.169', 'learning_rate': '0.0005032', 'epoch': '0.1648'}

 17%|█▋        | 680/4000 [47:13<3:52:58,  4.21s/it]
 17%|█▋        | 681/4000 [47:17<3:55:07,  4.25s/it]
 17%|█▋        | 682/4000 [47:21<3:53:55,  4.23s/it]
 17%|█▋        | 683/4000 [47:25<3:53:19,  4.22s/it]
 17%|█▋        | 684/4000 [47:29<3:51:50,  4.20s/it]
 17%|█▋        | 685/4000 [47:34<3:50:30,  4.17s/it]
 17%|█▋        | 686/4000 [47:38<3:49:53,  4.16s/it]
 17%|█▋        | 687/4000 [47:42<3:54:54,  4.25s/it]
 17%|█▋        | 688/4000 [47:46<3:53:15,  4.23s/it]
 17%|█▋        | 689/4000 [47:51<3:52:18,  4.21s/it]
 17%|█▋        | 690/4000 [47:55<3:51:00,  4.19s/it]
                                                    
{'loss': '0.7363', 'grad_norm': '0.2151', 'learning_rate': '0.0005017', 'epoch': '0.1672'}

 17%|█▋        | 690/4000 [47:55<3:51:00,  4.19s/it]
 17%|█▋        | 691/4000 [47:59<3:50:13,  4.17s/it]
 17%|█▋        | 692/4000 [48:03<3:49:31,  4.16s/it]
 17%|█▋        | 693/4000 [48:07<3:49:16,  4.16s/it]
 17%|█▋        | 694/4000 [48:12<3:55:14,  4.27s/it]
 17%|█▋        | 695/4000 [48:16<3:53:00,  4.23s/it]
 17%|█▋        | 696/4000 [48:20<3:51:10,  4.20s/it]
 17%|█▋        | 697/4000 [48:24<3:50:06,  4.18s/it]
 17%|█▋        | 698/4000 [48:28<3:49:22,  4.17s/it]
 17%|█▋        | 699/4000 [48:32<3:48:59,  4.16s/it]
 18%|█▊        | 700/4000 [48:36<3:48:29,  4.15s/it]
                                                    
{'loss': '0.7571', 'grad_norm': '0.1559', 'learning_rate': '0.0005002', 'epoch': '0.1696'}

 18%|█▊        | 700/4000 [48:36<3:48:29,  4.15s/it]
 18%|█▊        | 701/4000 [48:41<3:53:51,  4.25s/it]
 18%|█▊        | 702/4000 [48:45<3:52:18,  4.23s/it]
 18%|█▊        | 703/4000 [48:49<3:51:23,  4.21s/it]
 18%|█▊        | 704/4000 [48:53<3:50:34,  4.20s/it]
 18%|█▊        | 705/4000 [48:58<3:50:01,  4.19s/it]
 18%|█▊        | 706/4000 [49:02<3:49:43,  4.18s/it]
 18%|█▊        | 707/4000 [49:06<3:52:33,  4.24s/it]
 18%|█▊        | 708/4000 [49:11<3:54:55,  4.28s/it]
 18%|█▊        | 709/4000 [49:15<3:52:57,  4.25s/it]
 18%|█▊        | 710/4000 [49:19<3:51:35,  4.22s/it]
                                                    
{'loss': '0.7024', 'grad_norm': '0.2419', 'learning_rate': '0.0004986', 'epoch': '0.172'}

 18%|█▊        | 710/4000 [49:19<3:51:35,  4.22s/it]
 18%|█▊        | 711/4000 [49:23<3:50:27,  4.20s/it]
 18%|█▊        | 712/4000 [49:27<3:49:56,  4.20s/it]
 18%|█▊        | 713/4000 [49:31<3:50:28,  4.21s/it]
 18%|█▊        | 714/4000 [49:36<3:58:56,  4.36s/it]
 18%|█▊        | 715/4000 [49:40<3:57:38,  4.34s/it]
 18%|█▊        | 716/4000 [49:45<3:56:43,  4.32s/it]
 18%|█▊        | 717/4000 [49:49<3:55:52,  4.31s/it]
 18%|█▊        | 718/4000 [49:53<3:56:14,  4.32s/it]
 18%|█▊        | 719/4000 [49:58<3:54:39,  4.29s/it]
 18%|█▊        | 720/4000 [50:02<3:52:50,  4.26s/it]
                                                    
{'loss': '0.7251', 'grad_norm': '0.2271', 'learning_rate': '0.0004971', 'epoch': '0.1745'}

 18%|█▊        | 720/4000 [50:02<3:52:50,  4.26s/it]
 18%|█▊        | 721/4000 [50:06<3:58:06,  4.36s/it]
 18%|█▊        | 722/4000 [50:11<3:54:59,  4.30s/it]
 18%|█▊        | 723/4000 [50:15<3:52:48,  4.26s/it]
 18%|█▊        | 724/4000 [50:19<3:51:06,  4.23s/it]
 18%|█▊        | 725/4000 [50:23<3:50:04,  4.22s/it]
 18%|█▊        | 726/4000 [50:27<3:49:05,  4.20s/it]
 18%|█▊        | 727/4000 [50:31<3:48:34,  4.19s/it]
 18%|█▊        | 728/4000 [50:36<3:55:31,  4.32s/it]
 18%|█▊        | 729/4000 [50:40<3:55:55,  4.33s/it]
 18%|█▊        | 730/4000 [50:45<3:54:36,  4.30s/it]
                                                    
{'loss': '0.69', 'grad_norm': '0.2366', 'learning_rate': '0.0004956', 'epoch': '0.1769'}

 18%|█▊        | 730/4000 [50:45<3:54:36,  4.30s/it]
 18%|█▊        | 731/4000 [50:49<3:54:34,  4.31s/it]
 18%|█▊        | 732/4000 [50:53<3:54:01,  4.30s/it]
 18%|█▊        | 733/4000 [50:58<3:54:26,  4.31s/it]
 18%|█▊        | 734/4000 [51:02<3:56:16,  4.34s/it]
 18%|█▊        | 735/4000 [51:06<3:56:23,  4.34s/it]
 18%|█▊        | 736/4000 [51:10<3:53:39,  4.30s/it]
 18%|█▊        | 737/4000 [51:15<3:51:32,  4.26s/it]
 18%|█▊        | 738/4000 [51:19<3:50:17,  4.24s/it]
 18%|█▊        | 739/4000 [51:23<3:48:58,  4.21s/it]
 18%|█▊        | 740/4000 [51:27<3:48:02,  4.20s/it]
                                                    
{'loss': '0.6789', 'grad_norm': '0.1401', 'learning_rate': '0.0004941', 'epoch': '0.1793'}

 18%|█▊        | 740/4000 [51:27<3:48:02,  4.20s/it]
 19%|█▊        | 741/4000 [51:32<3:54:16,  4.31s/it]
 19%|█▊        | 742/4000 [51:36<3:54:06,  4.31s/it]
 19%|█▊        | 743/4000 [51:40<3:55:12,  4.33s/it]
 19%|█▊        | 744/4000 [51:45<3:54:03,  4.31s/it]
 19%|█▊        | 745/4000 [51:49<3:53:30,  4.30s/it]
 19%|█▊        | 746/4000 [51:53<3:52:51,  4.29s/it]
 19%|█▊        | 747/4000 [51:57<3:51:36,  4.27s/it]
 19%|█▊        | 748/4000 [52:02<3:57:23,  4.38s/it]
 19%|█▊        | 749/4000 [52:06<3:53:52,  4.32s/it]
 19%|█▉        | 750/4000 [52:10<3:51:38,  4.28s/it]
                                                    
{'loss': '0.6706', 'grad_norm': '0.156', 'learning_rate': '0.0004926', 'epoch': '0.1817'}

 19%|█▉        | 750/4000 [52:10<3:51:38,  4.28s/it]
 19%|█▉        | 751/4000 [52:15<3:52:38,  4.30s/it]
 19%|█▉        | 752/4000 [52:19<3:51:19,  4.27s/it]
 19%|█▉        | 753/4000 [52:23<3:53:05,  4.31s/it]
 19%|█▉        | 754/4000 [52:28<3:51:13,  4.27s/it]
 19%|█▉        | 755/4000 [52:32<3:56:01,  4.36s/it]
 19%|█▉        | 756/4000 [52:36<3:53:07,  4.31s/it]
 19%|█▉        | 757/4000 [52:41<3:52:00,  4.29s/it]
 19%|█▉        | 758/4000 [52:45<3:51:40,  4.29s/it]
 19%|█▉        | 759/4000 [52:49<3:51:37,  4.29s/it]
 19%|█▉        | 760/4000 [52:53<3:51:36,  4.29s/it]
                                                    
{'loss': '0.6586', 'grad_norm': '0.1442', 'learning_rate': '0.0004911', 'epoch': '0.1841'}

 19%|█▉        | 760/4000 [52:53<3:51:36,  4.29s/it]
 19%|█▉        | 761/4000 [52:58<3:59:55,  4.44s/it]
 19%|█▉        | 762/4000 [53:02<3:55:39,  4.37s/it]
 19%|█▉        | 763/4000 [53:07<3:52:27,  4.31s/it]
 19%|█▉        | 764/4000 [53:11<3:50:29,  4.27s/it]
 19%|█▉        | 765/4000 [53:15<3:48:50,  4.24s/it]
 19%|█▉        | 766/4000 [53:19<3:47:35,  4.22s/it]
 19%|█▉        | 767/4000 [53:23<3:46:43,  4.21s/it]
 19%|█▉        | 768/4000 [53:28<3:52:43,  4.32s/it]
 19%|█▉        | 769/4000 [53:32<3:50:13,  4.28s/it]
 19%|█▉        | 770/4000 [53:36<3:49:21,  4.26s/it]
                                                    
{'loss': '0.6648', 'grad_norm': '0.1553', 'learning_rate': '0.0004895', 'epoch': '0.1866'}

 19%|█▉        | 770/4000 [53:36<3:49:21,  4.26s/it]
 19%|█▉        | 771/4000 [53:41<3:49:26,  4.26s/it]
 19%|█▉        | 772/4000 [53:45<3:50:41,  4.29s/it]
 19%|█▉        | 773/4000 [53:49<3:51:25,  4.30s/it]
 19%|█▉        | 774/4000 [53:54<3:52:23,  4.32s/it]
 19%|█▉        | 775/4000 [53:59<4:03:02,  4.52s/it]
 19%|█▉        | 776/4000 [54:03<3:58:00,  4.43s/it]
 19%|█▉        | 777/4000 [54:07<3:54:20,  4.36s/it]
 19%|█▉        | 778/4000 [54:11<3:52:39,  4.33s/it]
 19%|█▉        | 779/4000 [54:16<3:50:37,  4.30s/it]
 20%|█▉        | 780/4000 [54:20<3:49:12,  4.27s/it]
                                                    
{'loss': '0.7217', 'grad_norm': '0.1571', 'learning_rate': '0.000488', 'epoch': '0.189'}

 20%|█▉        | 780/4000 [54:20<3:49:12,  4.27s/it]
 20%|█▉        | 781/4000 [54:24<3:48:20,  4.26s/it]
 20%|█▉        | 782/4000 [54:29<3:54:48,  4.38s/it]
 20%|█▉        | 783/4000 [54:33<3:55:09,  4.39s/it]
 20%|█▉        | 784/4000 [54:37<3:53:17,  4.35s/it]
 20%|█▉        | 785/4000 [54:42<3:52:56,  4.35s/it]
 20%|█▉        | 786/4000 [54:46<3:53:42,  4.36s/it]
 20%|█▉        | 787/4000 [54:50<3:54:26,  4.38s/it]
 20%|█▉        | 788/4000 [54:55<4:02:20,  4.53s/it]
 20%|█▉        | 789/4000 [55:00<3:57:32,  4.44s/it]
 20%|█▉        | 790/4000 [55:04<3:54:51,  4.39s/it]
                                                    
{'loss': '0.656', 'grad_norm': '0.1701', 'learning_rate': '0.0004865', 'epoch': '0.1914'}

 20%|█▉        | 790/4000 [55:04<3:54:51,  4.39s/it]
 20%|█▉        | 791/4000 [55:08<3:52:38,  4.35s/it]
 20%|█▉        | 792/4000 [55:12<3:51:46,  4.33s/it]
 20%|█▉        | 793/4000 [55:17<3:49:52,  4.30s/it]
 20%|█▉        | 794/4000 [55:21<3:48:11,  4.27s/it]
 20%|█▉        | 795/4000 [55:25<3:53:31,  4.37s/it]
 20%|█▉        | 796/4000 [55:30<3:50:38,  4.32s/it]
 20%|█▉        | 797/4000 [55:34<3:51:04,  4.33s/it]
 20%|█▉        | 798/4000 [55:38<3:51:17,  4.33s/it]
 20%|█▉        | 799/4000 [55:43<3:50:39,  4.32s/it]
 20%|██        | 800/4000 [55:47<3:50:36,  4.32s/it]
                                                    
{'loss': '0.6637', 'grad_norm': '0.1514', 'learning_rate': '0.000485', 'epoch': '0.1938'}

 20%|██        | 800/4000 [55:47<3:50:36,  4.32s/it]
 20%|██        | 801/4000 [55:51<3:51:50,  4.35s/it]
 20%|██        | 802/4000 [55:56<3:56:26,  4.44s/it]
 20%|██        | 803/4000 [56:00<3:52:28,  4.36s/it]
 20%|██        | 804/4000 [56:04<3:49:47,  4.31s/it]
 20%|██        | 805/4000 [56:09<3:49:33,  4.31s/it]
 20%|██        | 806/4000 [56:13<3:48:36,  4.29s/it]
 20%|██        | 807/4000 [56:17<3:46:47,  4.26s/it]
 20%|██        | 808/4000 [56:22<3:49:21,  4.31s/it]
 20%|██        | 809/4000 [56:26<3:50:54,  4.34s/it]
 20%|██        | 810/4000 [56:30<3:52:48,  4.38s/it]
                                                    
{'loss': '0.648', 'grad_norm': '0.1341', 'learning_rate': '0.0004835', 'epoch': '0.1963'}

 20%|██        | 810/4000 [56:30<3:52:48,  4.38s/it]
 20%|██        | 811/4000 [56:35<3:52:57,  4.38s/it]
 20%|██        | 812/4000 [56:39<3:51:45,  4.36s/it]
 20%|██        | 813/4000 [56:43<3:50:28,  4.34s/it]
 20%|██        | 814/4000 [56:48<3:51:27,  4.36s/it]
 20%|██        | 815/4000 [56:52<3:56:44,  4.46s/it]
 20%|██        | 816/4000 [56:57<3:52:43,  4.39s/it]
 20%|██        | 817/4000 [57:01<3:50:02,  4.34s/it]
 20%|██        | 818/4000 [57:05<3:47:47,  4.30s/it]
 20%|██        | 819/4000 [57:09<3:46:11,  4.27s/it]
 20%|██        | 820/4000 [57:14<3:45:59,  4.26s/it]
                                                    
{'loss': '0.6421', 'grad_norm': '0.1593', 'learning_rate': '0.000482', 'epoch': '0.1987'}

 20%|██        | 820/4000 [57:14<3:45:59,  4.26s/it]
 21%|██        | 821/4000 [57:18<3:46:30,  4.28s/it]
 21%|██        | 822/4000 [57:23<3:52:15,  4.38s/it]
 21%|██        | 823/4000 [57:27<3:50:27,  4.35s/it]
 21%|██        | 824/4000 [57:31<3:49:54,  4.34s/it]
 21%|██        | 825/4000 [57:35<3:48:31,  4.32s/it]
 21%|██        | 826/4000 [57:40<3:48:30,  4.32s/it]
 21%|██        | 827/4000 [57:44<3:47:52,  4.31s/it]
 21%|██        | 828/4000 [57:48<3:46:05,  4.28s/it]
 21%|██        | 829/4000 [57:53<3:56:48,  4.48s/it]
 21%|██        | 830/4000 [57:57<3:52:37,  4.40s/it]
                                                    
{'loss': '0.6431', 'grad_norm': '0.1672', 'learning_rate': '0.0004805', 'epoch': '0.2011'}

 21%|██        | 830/4000 [57:57<3:52:37,  4.40s/it]
 21%|██        | 831/4000 [58:02<3:49:07,  4.34s/it]
 21%|██        | 832/4000 [58:06<3:46:14,  4.28s/it]
 21%|██        | 833/4000 [58:10<3:44:27,  4.25s/it]
 21%|██        | 834/4000 [58:14<3:43:39,  4.24s/it]
 21%|██        | 835/4000 [58:19<3:50:44,  4.37s/it]
 21%|██        | 836/4000 [58:23<3:51:36,  4.39s/it]
 21%|██        | 837/4000 [58:27<3:49:27,  4.35s/it]
 21%|██        | 838/4000 [58:32<3:49:00,  4.35s/it]
 21%|██        | 839/4000 [58:36<3:48:07,  4.33s/it]
 21%|██        | 840/4000 [58:40<3:48:48,  4.34s/it]
                                                    
{'loss': '0.6408', 'grad_norm': '0.1539', 'learning_rate': '0.0004789', 'epoch': '0.2035'}

 21%|██        | 840/4000 [58:40<3:48:48,  4.34s/it]
 21%|██        | 841/4000 [58:45<3:46:56,  4.31s/it]
 21%|██        | 842/4000 [58:49<3:52:12,  4.41s/it]
 21%|██        | 843/4000 [58:54<3:52:03,  4.41s/it]
 21%|██        | 844/4000 [58:58<3:48:45,  4.35s/it]
 21%|██        | 845/4000 [59:02<3:46:20,  4.30s/it]
 21%|██        | 846/4000 [59:06<3:44:45,  4.28s/it]
 21%|██        | 847/4000 [59:11<3:43:36,  4.26s/it]
 21%|██        | 848/4000 [59:15<3:43:06,  4.25s/it]
 21%|██        | 849/4000 [59:20<3:53:34,  4.45s/it]
 21%|██▏       | 850/4000 [59:24<3:50:59,  4.40s/it]
                                                    
{'loss': '0.5943', 'grad_norm': '0.14', 'learning_rate': '0.0004774', 'epoch': '0.206'}

 21%|██▏       | 850/4000 [59:24<3:50:59,  4.40s/it]
 21%|██▏       | 851/4000 [59:28<3:49:34,  4.37s/it]
 21%|██▏       | 852/4000 [59:33<3:50:19,  4.39s/it]
 21%|██▏       | 853/4000 [59:37<3:50:07,  4.39s/it]
 21%|██▏       | 854/4000 [59:41<3:48:16,  4.35s/it]
 21%|██▏       | 855/4000 [59:46<3:46:32,  4.32s/it]
 21%|██▏       | 856/4000 [59:50<3:50:33,  4.40s/it]
 21%|██▏       | 857/4000 [59:54<3:47:19,  4.34s/it]
 21%|██▏       | 858/4000 [59:59<3:48:42,  4.37s/it]
 21%|██▏       | 859/4000 [1:00:03<3:45:32,  4.31s/it]
 22%|██▏       | 860/4000 [1:00:07<3:43:28,  4.27s/it]
                                                      
{'loss': '0.6131', 'grad_norm': '0.136', 'learning_rate': '0.0004759', 'epoch': '0.2084'}

 22%|██▏       | 860/4000 [1:00:07<3:43:28,  4.27s/it]
 22%|██▏       | 861/4000 [1:00:11<3:42:03,  4.24s/it]
 22%|██▏       | 862/4000 [1:00:16<3:49:05,  4.38s/it]
 22%|██▏       | 863/4000 [1:00:21<3:52:01,  4.44s/it]
 22%|██▏       | 864/4000 [1:00:25<3:49:57,  4.40s/it]
 22%|██▏       | 865/4000 [1:00:29<3:48:36,  4.38s/it]
 22%|██▏       | 866/4000 [1:00:34<3:46:59,  4.35s/it]
 22%|██▏       | 867/4000 [1:00:38<3:44:40,  4.30s/it]
 22%|██▏       | 868/4000 [1:00:42<3:42:56,  4.27s/it]
 22%|██▏       | 869/4000 [1:00:47<3:48:39,  4.38s/it]
 22%|██▏       | 870/4000 [1:00:51<3:46:05,  4.33s/it]
                                                      
{'loss': '0.6394', 'grad_norm': '0.1506', 'learning_rate': '0.0004744', 'epoch': '0.2108'}

 22%|██▏       | 870/4000 [1:00:51<3:46:05,  4.33s/it]
 22%|██▏       | 871/4000 [1:00:55<3:43:53,  4.29s/it]
 22%|██▏       | 872/4000 [1:00:59<3:44:02,  4.30s/it]
 22%|██▏       | 873/4000 [1:01:04<3:42:18,  4.27s/it]
 22%|██▏       | 874/4000 [1:01:08<3:41:38,  4.25s/it]
 22%|██▏       | 875/4000 [1:01:12<3:43:16,  4.29s/it]
 22%|██▏       | 876/4000 [1:01:17<3:51:04,  4.44s/it]
 22%|██▏       | 877/4000 [1:01:21<3:48:46,  4.40s/it]
 22%|██▏       | 878/4000 [1:01:26<3:47:29,  4.37s/it]
 22%|██▏       | 879/4000 [1:01:30<3:45:50,  4.34s/it]
 22%|██▏       | 880/4000 [1:01:34<3:45:03,  4.33s/it]
                                                      
{'loss': '0.6229', 'grad_norm': '0.1507', 'learning_rate': '0.0004729', 'epoch': '0.2132'}

 22%|██▏       | 880/4000 [1:01:34<3:45:03,  4.33s/it]
 22%|██▏       | 881/4000 [1:01:38<3:43:23,  4.30s/it]
 22%|██▏       | 882/4000 [1:01:43<3:41:35,  4.26s/it]
 22%|██▏       | 883/4000 [1:01:47<3:47:20,  4.38s/it]
 22%|██▏       | 884/4000 [1:01:51<3:44:30,  4.32s/it]
 22%|██▏       | 885/4000 [1:01:56<3:42:15,  4.28s/it]
 22%|██▏       | 886/4000 [1:02:00<3:41:53,  4.28s/it]
 22%|██▏       | 887/4000 [1:02:04<3:40:53,  4.26s/it]
 22%|██▏       | 888/4000 [1:02:08<3:41:00,  4.26s/it]
 22%|██▏       | 889/4000 [1:02:13<3:49:07,  4.42s/it]
 22%|██▏       | 890/4000 [1:02:17<3:46:40,  4.37s/it]
                                                      
{'loss': '0.6182', 'grad_norm': '0.1306', 'learning_rate': '0.0004714', 'epoch': '0.2156'}

 22%|██▏       | 890/4000 [1:02:17<3:46:40,  4.37s/it]
 22%|██▏       | 891/4000 [1:02:22<3:46:04,  4.36s/it]
 22%|██▏       | 892/4000 [1:02:26<3:44:27,  4.33s/it]
 22%|██▏       | 893/4000 [1:02:30<3:44:48,  4.34s/it]
 22%|██▏       | 894/4000 [1:02:35<3:42:27,  4.30s/it]
 22%|██▏       | 895/4000 [1:02:39<3:40:49,  4.27s/it]
 22%|██▏       | 896/4000 [1:02:43<3:46:28,  4.38s/it]
 22%|██▏       | 897/4000 [1:02:48<3:43:20,  4.32s/it]
 22%|██▏       | 898/4000 [1:02:52<3:40:54,  4.27s/it]
 22%|██▏       | 899/4000 [1:02:56<3:39:16,  4.24s/it]
 22%|██▎       | 900/4000 [1:03:00<3:38:25,  4.23s/it]
                                                      
{'loss': '0.614', 'grad_norm': '0.149', 'learning_rate': '0.0004698', 'epoch': '0.2181'}

 22%|██▎       | 900/4000 [1:03:00<3:38:25,  4.23s/it]
 23%|██▎       | 901/4000 [1:03:04<3:37:29,  4.21s/it]
 23%|██▎       | 902/4000 [1:03:08<3:37:28,  4.21s/it]
 23%|██▎       | 903/4000 [1:03:13<3:48:08,  4.42s/it]
 23%|██▎       | 904/4000 [1:03:18<3:46:45,  4.39s/it]
 23%|██▎       | 905/4000 [1:03:22<3:44:57,  4.36s/it]
 23%|██▎       | 906/4000 [1:03:26<3:44:40,  4.36s/it]
 23%|██▎       | 907/4000 [1:03:31<3:43:33,  4.34s/it]
 23%|██▎       | 908/4000 [1:03:35<3:41:25,  4.30s/it]
 23%|██▎       | 909/4000 [1:03:39<3:39:36,  4.26s/it]
 23%|██▎       | 910/4000 [1:03:44<3:44:52,  4.37s/it]
                                                      
{'loss': '0.6052', 'grad_norm': '0.152', 'learning_rate': '0.0004683', 'epoch': '0.2205'}

 23%|██▎       | 910/4000 [1:03:44<3:44:52,  4.37s/it]
 23%|██▎       | 911/4000 [1:03:48<3:42:41,  4.33s/it]
 23%|██▎       | 912/4000 [1:03:52<3:40:24,  4.28s/it]
 23%|██▎       | 913/4000 [1:03:56<3:38:34,  4.25s/it]
 23%|██▎       | 914/4000 [1:04:00<3:37:35,  4.23s/it]
 23%|██▎       | 915/4000 [1:04:05<3:41:48,  4.31s/it]
 23%|██▎       | 916/4000 [1:04:10<3:47:35,  4.43s/it]
 23%|██▎       | 917/4000 [1:04:14<3:44:55,  4.38s/it]
 23%|██▎       | 918/4000 [1:04:18<3:43:22,  4.35s/it]
 23%|██▎       | 919/4000 [1:04:22<3:42:54,  4.34s/it]
 23%|██▎       | 920/4000 [1:04:27<3:42:23,  4.33s/it]
                                                      
{'loss': '0.6114', 'grad_norm': '0.1508', 'learning_rate': '0.0004668', 'epoch': '0.2229'}

 23%|██▎       | 920/4000 [1:04:27<3:42:23,  4.33s/it]
 23%|██▎       | 921/4000 [1:04:31<3:40:01,  4.29s/it]
 23%|██▎       | 922/4000 [1:04:35<3:38:17,  4.26s/it]
 23%|██▎       | 923/4000 [1:04:40<3:48:44,  4.46s/it]
 23%|██▎       | 924/4000 [1:04:44<3:46:53,  4.43s/it]
 23%|██▎       | 925/4000 [1:04:49<3:43:38,  4.36s/it]
 23%|██▎       | 926/4000 [1:04:53<3:40:53,  4.31s/it]
 23%|██▎       | 927/4000 [1:04:57<3:38:46,  4.27s/it]
 23%|██▎       | 928/4000 [1:05:01<3:37:28,  4.25s/it]
 23%|██▎       | 929/4000 [1:05:05<3:37:18,  4.25s/it]
 23%|██▎       | 930/4000 [1:05:10<3:44:37,  4.39s/it]
                                                      
{'loss': '0.6094', 'grad_norm': '0.1683', 'learning_rate': '0.0004653', 'epoch': '0.2253'}

 23%|██▎       | 930/4000 [1:05:10<3:44:37,  4.39s/it]
 23%|██▎       | 931/4000 [1:05:14<3:43:07,  4.36s/it]
 23%|██▎       | 932/4000 [1:05:19<3:42:08,  4.34s/it]
 23%|██▎       | 933/4000 [1:05:23<3:43:06,  4.36s/it]
 23%|██▎       | 934/4000 [1:05:27<3:41:42,  4.34s/it]
 23%|██▎       | 935/4000 [1:05:32<3:39:51,  4.30s/it]
 23%|██▎       | 936/4000 [1:05:36<3:41:43,  4.34s/it]
 23%|██▎       | 937/4000 [1:05:40<3:42:18,  4.35s/it]
 23%|██▎       | 938/4000 [1:05:45<3:39:46,  4.31s/it]
 23%|██▎       | 939/4000 [1:05:49<3:38:25,  4.28s/it]
 24%|██▎       | 940/4000 [1:05:53<3:37:47,  4.27s/it]
                                                      
{'loss': '0.6596', 'grad_norm': '0.1666', 'learning_rate': '0.0004638', 'epoch': '0.2278'}

 24%|██▎       | 940/4000 [1:05:53<3:37:47,  4.27s/it]
 24%|██▎       | 941/4000 [1:05:57<3:37:03,  4.26s/it]
 24%|██▎       | 942/4000 [1:06:02<3:40:17,  4.32s/it]
 24%|██▎       | 943/4000 [1:06:07<3:47:54,  4.47s/it]
 24%|██▎       | 944/4000 [1:06:11<3:46:16,  4.44s/it]
 24%|██▎       | 945/4000 [1:06:15<3:44:41,  4.41s/it]
 24%|██▎       | 946/4000 [1:06:20<3:41:15,  4.35s/it]
 24%|██▎       | 947/4000 [1:06:24<3:38:45,  4.30s/it]
 24%|██▎       | 948/4000 [1:06:28<3:36:44,  4.26s/it]
 24%|██▎       | 949/4000 [1:06:32<3:35:24,  4.24s/it]
 24%|██▍       | 950/4000 [1:06:37<3:40:55,  4.35s/it]
                                                      
{'loss': '0.5961', 'grad_norm': '0.1414', 'learning_rate': '0.0004623', 'epoch': '0.2302'}

 24%|██▍       | 950/4000 [1:06:37<3:40:55,  4.35s/it]
 24%|██▍       | 951/4000 [1:06:41<3:39:27,  4.32s/it]
 24%|██▍       | 952/4000 [1:06:45<3:37:29,  4.28s/it]
 24%|██▍       | 953/4000 [1:06:49<3:36:54,  4.27s/it]
 24%|██▍       | 954/4000 [1:06:54<3:36:52,  4.27s/it]
 24%|██▍       | 955/4000 [1:06:58<3:37:24,  4.28s/it]
 24%|██▍       | 956/4000 [1:07:02<3:38:12,  4.30s/it]
 24%|██▍       | 957/4000 [1:07:07<3:44:41,  4.43s/it]
 24%|██▍       | 958/4000 [1:07:11<3:41:04,  4.36s/it]
 24%|██▍       | 959/4000 [1:07:15<3:38:25,  4.31s/it]
 24%|██▍       | 960/4000 [1:07:20<3:39:04,  4.32s/it]
                                                      
{'loss': '0.6108', 'grad_norm': '0.1701', 'learning_rate': '0.0004608', 'epoch': '0.2326'}

 24%|██▍       | 960/4000 [1:07:20<3:39:04,  4.32s/it]
 24%|██▍       | 961/4000 [1:07:24<3:37:16,  4.29s/it]
 24%|██▍       | 962/4000 [1:07:28<3:36:34,  4.28s/it]
 24%|██▍       | 963/4000 [1:07:32<3:35:28,  4.26s/it]
 24%|██▍       | 964/4000 [1:07:37<3:40:45,  4.36s/it]
 24%|██▍       | 965/4000 [1:07:41<3:38:05,  4.31s/it]
 24%|██▍       | 966/4000 [1:07:46<3:38:42,  4.33s/it]
 24%|██▍       | 967/4000 [1:07:50<3:37:51,  4.31s/it]
 24%|██▍       | 968/4000 [1:07:54<3:37:41,  4.31s/it]
 24%|██▍       | 969/4000 [1:07:59<3:39:12,  4.34s/it]
 24%|██▍       | 970/4000 [1:08:03<3:45:44,  4.47s/it]
                                                      
{'loss': '0.5866', 'grad_norm': '0.1793', 'learning_rate': '0.0004592', 'epoch': '0.235'}

 24%|██▍       | 970/4000 [1:08:03<3:45:44,  4.47s/it]
 24%|██▍       | 971/4000 [1:08:08<3:41:43,  4.39s/it]
 24%|██▍       | 972/4000 [1:08:12<3:39:32,  4.35s/it]
 24%|██▍       | 973/4000 [1:08:16<3:37:06,  4.30s/it]
 24%|██▍       | 974/4000 [1:08:20<3:35:54,  4.28s/it]
 24%|██▍       | 975/4000 [1:08:24<3:34:29,  4.25s/it]
 24%|██▍       | 976/4000 [1:08:29<3:33:21,  4.23s/it]
 24%|██▍       | 977/4000 [1:08:33<3:39:25,  4.36s/it]
 24%|██▍       | 978/4000 [1:08:38<3:39:12,  4.35s/it]
 24%|██▍       | 979/4000 [1:08:42<3:39:31,  4.36s/it]
 24%|██▍       | 980/4000 [1:08:46<3:38:16,  4.34s/it]
                                                      
{'loss': '0.5984', 'grad_norm': '0.1629', 'learning_rate': '0.0004577', 'epoch': '0.2375'}

 24%|██▍       | 980/4000 [1:08:46<3:38:16,  4.34s/it]
 25%|██▍       | 981/4000 [1:08:51<3:37:43,  4.33s/it]
 25%|██▍       | 982/4000 [1:08:55<3:36:46,  4.31s/it]
 25%|██▍       | 983/4000 [1:08:59<3:35:29,  4.29s/it]
 25%|██▍       | 984/4000 [1:09:04<3:40:05,  4.38s/it]
 25%|██▍       | 985/4000 [1:09:08<3:37:05,  4.32s/it]
 25%|██▍       | 986/4000 [1:09:12<3:34:49,  4.28s/it]
 25%|██▍       | 987/4000 [1:09:16<3:34:28,  4.27s/it]
 25%|██▍       | 988/4000 [1:09:21<3:36:21,  4.31s/it]
 25%|██▍       | 989/4000 [1:09:25<3:35:07,  4.29s/it]
 25%|██▍       | 990/4000 [1:09:29<3:36:32,  4.32s/it]
                                                      
{'loss': '0.6663', 'grad_norm': '0.1632', 'learning_rate': '0.0004562', 'epoch': '0.2399'}

 25%|██▍       | 990/4000 [1:09:29<3:36:32,  4.32s/it]
 25%|██▍       | 991/4000 [1:09:34<3:40:24,  4.40s/it]
 25%|██▍       | 992/4000 [1:09:38<3:37:22,  4.34s/it]
 25%|██▍       | 993/4000 [1:09:42<3:34:34,  4.28s/it]
 25%|██▍       | 994/4000 [1:09:46<3:32:50,  4.25s/it]
 25%|██▍       | 995/4000 [1:09:51<3:31:39,  4.23s/it]
 25%|██▍       | 996/4000 [1:09:55<3:30:56,  4.21s/it]
 25%|██▍       | 997/4000 [1:09:59<3:36:43,  4.33s/it]
 25%|██▍       | 998/4000 [1:10:04<3:34:22,  4.28s/it]
 25%|██▍       | 999/4000 [1:10:08<3:32:22,  4.25s/it]
 25%|██▌       | 1000/4000 [1:10:12<3:31:16,  4.23s/it]
                                                       
{'loss': '0.5965', 'grad_norm': '0.1698', 'learning_rate': '0.0004547', 'epoch': '0.2423'}

 25%|██▌       | 1000/4000 [1:10:12<3:31:16,  4.23s/it]
 25%|██▌       | 1001/4000 [1:10:16<3:30:00,  4.20s/it]
 25%|██▌       | 1002/4000 [1:10:20<3:29:21,  4.19s/it]
 25%|██▌       | 1003/4000 [1:10:25<3:31:30,  4.23s/it]
 25%|██▌       | 1004/4000 [1:10:29<3:35:56,  4.32s/it]
 25%|██▌       | 1005/4000 [1:10:33<3:33:28,  4.28s/it]
 25%|██▌       | 1006/4000 [1:10:37<3:31:30,  4.24s/it]
 25%|██▌       | 1007/4000 [1:10:42<3:30:08,  4.21s/it]
 25%|██▌       | 1008/4000 [1:10:46<3:29:17,  4.20s/it]
 25%|██▌       | 1009/4000 [1:10:50<3:28:38,  4.19s/it]
 25%|██▌       | 1010/4000 [1:10:54<3:28:24,  4.18s/it]
                                                       
{'loss': '0.6223', 'grad_norm': '0.1658', 'learning_rate': '0.0004532', 'epoch': '0.2447'}

 25%|██▌       | 1010/4000 [1:10:54<3:28:24,  4.18s/it]
 25%|██▌       | 1011/4000 [1:10:59<3:34:11,  4.30s/it]
 25%|██▌       | 1012/4000 [1:11:03<3:32:00,  4.26s/it]
 25%|██▌       | 1013/4000 [1:11:07<3:30:35,  4.23s/it]
 25%|██▌       | 1014/4000 [1:11:11<3:29:18,  4.21s/it]
 25%|██▌       | 1015/4000 [1:11:15<3:28:14,  4.19s/it]
 25%|██▌       | 1016/4000 [1:11:19<3:27:55,  4.18s/it]
 25%|██▌       | 1017/4000 [1:11:24<3:34:00,  4.30s/it]
 25%|██▌       | 1018/4000 [1:11:28<3:31:23,  4.25s/it]
 25%|██▌       | 1019/4000 [1:11:32<3:30:00,  4.23s/it]
 26%|██▌       | 1020/4000 [1:11:36<3:28:57,  4.21s/it]
                                                       
{'loss': '0.5967', 'grad_norm': '0.1394', 'learning_rate': '0.0004517', 'epoch': '0.2471'}

 26%|██▌       | 1020/4000 [1:11:36<3:28:57,  4.21s/it]
 26%|██▌       | 1021/4000 [1:11:41<3:28:26,  4.20s/it]
 26%|██▌       | 1022/4000 [1:11:45<3:28:07,  4.19s/it]
 26%|██▌       | 1023/4000 [1:11:49<3:27:41,  4.19s/it]
 26%|██▌       | 1024/4000 [1:11:54<3:33:08,  4.30s/it]
 26%|██▌       | 1025/4000 [1:11:58<3:30:59,  4.26s/it]
 26%|██▌       | 1026/4000 [1:12:02<3:29:34,  4.23s/it]
 26%|██▌       | 1027/4000 [1:12:06<3:28:48,  4.21s/it]
 26%|██▌       | 1028/4000 [1:12:10<3:27:57,  4.20s/it]
 26%|██▌       | 1029/4000 [1:12:14<3:27:52,  4.20s/it]
 26%|██▌       | 1030/4000 [1:12:19<3:27:18,  4.19s/it]
                                                       
{'loss': '0.6391', 'grad_norm': '0.1589', 'learning_rate': '0.0004502', 'epoch': '0.2496'}

 26%|██▌       | 1030/4000 [1:12:19<3:27:18,  4.19s/it]
 26%|██▌       | 1031/4000 [1:12:23<3:32:38,  4.30s/it]
 26%|██▌       | 1032/4000 [1:12:27<3:30:38,  4.26s/it]
 26%|██▌       | 1033/4000 [1:12:31<3:29:06,  4.23s/it]
 26%|██▌       | 1034/4000 [1:12:36<3:28:13,  4.21s/it]
 26%|██▌       | 1035/4000 [1:12:40<3:27:22,  4.20s/it]
 26%|██▌       | 1036/4000 [1:12:44<3:26:48,  4.19s/it]
 26%|██▌       | 1037/4000 [1:12:48<3:26:25,  4.18s/it]
 26%|██▌       | 1038/4000 [1:12:53<3:31:27,  4.28s/it]
 26%|██▌       | 1039/4000 [1:12:57<3:29:32,  4.25s/it]
 26%|██▌       | 1040/4000 [1:13:01<3:28:20,  4.22s/it]
                                                       
{'loss': '0.5962', 'grad_norm': '0.1483', 'learning_rate': '0.0004486', 'epoch': '0.252'}

 26%|██▌       | 1040/4000 [1:13:01<3:28:20,  4.22s/it]
 26%|██▌       | 1041/4000 [1:13:05<3:28:02,  4.22s/it]
 26%|██▌       | 1042/4000 [1:13:09<3:27:18,  4.21s/it]
 26%|██▌       | 1043/4000 [1:13:14<3:26:32,  4.19s/it]
 26%|██▌       | 1044/4000 [1:13:18<3:29:26,  4.25s/it]
 26%|██▌       | 1045/4000 [1:13:22<3:30:58,  4.28s/it]
 26%|██▌       | 1046/4000 [1:13:26<3:29:14,  4.25s/it]
 26%|██▌       | 1047/4000 [1:13:31<3:30:29,  4.28s/it]
 26%|██▌       | 1048/4000 [1:13:35<3:28:37,  4.24s/it]
 26%|██▌       | 1049/4000 [1:13:39<3:27:17,  4.21s/it]
 26%|██▋       | 1050/4000 [1:13:43<3:26:28,  4.20s/it]
                                                       
{'loss': '0.634', 'grad_norm': '0.1758', 'learning_rate': '0.0004471', 'epoch': '0.2544'}

 26%|██▋       | 1050/4000 [1:13:43<3:26:28,  4.20s/it]
 26%|██▋       | 1051/4000 [1:13:48<3:31:17,  4.30s/it]
 26%|██▋       | 1052/4000 [1:13:52<3:29:30,  4.26s/it]
 26%|██▋       | 1053/4000 [1:13:56<3:28:15,  4.24s/it]
 26%|██▋       | 1054/4000 [1:14:00<3:27:14,  4.22s/it]
 26%|██▋       | 1055/4000 [1:14:05<3:28:07,  4.24s/it]
 26%|██▋       | 1056/4000 [1:14:09<3:28:43,  4.25s/it]
 26%|██▋       | 1057/4000 [1:14:13<3:27:08,  4.22s/it]
 26%|██▋       | 1058/4000 [1:14:18<3:32:01,  4.32s/it]
 26%|██▋       | 1059/4000 [1:14:22<3:29:40,  4.28s/it]
 26%|██▋       | 1060/4000 [1:14:26<3:27:55,  4.24s/it]
                                                       
{'loss': '0.5492', 'grad_norm': '0.1251', 'learning_rate': '0.0004456', 'epoch': '0.2568'}

 26%|██▋       | 1060/4000 [1:14:26<3:27:55,  4.24s/it]
 27%|██▋       | 1061/4000 [1:14:30<3:26:32,  4.22s/it]
 27%|██▋       | 1062/4000 [1:14:34<3:25:39,  4.20s/it]
 27%|██▋       | 1063/4000 [1:14:38<3:25:30,  4.20s/it]
 27%|██▋       | 1064/4000 [1:14:43<3:24:50,  4.19s/it]
 27%|██▋       | 1065/4000 [1:14:47<3:30:17,  4.30s/it]
 27%|██▋       | 1066/4000 [1:14:51<3:28:07,  4.26s/it]
 27%|██▋       | 1067/4000 [1:14:55<3:26:47,  4.23s/it]
 27%|██▋       | 1068/4000 [1:15:00<3:25:46,  4.21s/it]
 27%|██▋       | 1069/4000 [1:15:04<3:24:54,  4.19s/it]
 27%|██▋       | 1070/4000 [1:15:08<3:24:18,  4.18s/it]
                                                       
{'loss': '0.5665', 'grad_norm': '0.1419', 'learning_rate': '0.0004441', 'epoch': '0.2593'}

 27%|██▋       | 1070/4000 [1:15:08<3:24:18,  4.18s/it]
 27%|██▋       | 1071/4000 [1:15:13<3:29:25,  4.29s/it]
 27%|██▋       | 1072/4000 [1:15:17<3:27:34,  4.25s/it]
 27%|██▋       | 1073/4000 [1:15:21<3:26:40,  4.24s/it]
 27%|██▋       | 1074/4000 [1:15:25<3:25:27,  4.21s/it]
 27%|██▋       | 1075/4000 [1:15:29<3:24:49,  4.20s/it]
 27%|██▋       | 1076/4000 [1:15:33<3:24:21,  4.19s/it]
 27%|██▋       | 1077/4000 [1:15:38<3:24:22,  4.20s/it]
 27%|██▋       | 1078/4000 [1:15:42<3:29:11,  4.30s/it]
 27%|██▋       | 1079/4000 [1:15:46<3:27:32,  4.26s/it]
 27%|██▋       | 1080/4000 [1:15:50<3:25:56,  4.23s/it]
                                                       
{'loss': '0.5894', 'grad_norm': '0.1408', 'learning_rate': '0.0004426', 'epoch': '0.2617'}

 27%|██▋       | 1080/4000 [1:15:50<3:25:56,  4.23s/it]
 27%|██▋       | 1081/4000 [1:15:55<3:24:53,  4.21s/it]
 27%|██▋       | 1082/4000 [1:15:59<3:23:50,  4.19s/it]
 27%|██▋       | 1083/4000 [1:16:03<3:23:21,  4.18s/it]
 27%|██▋       | 1084/4000 [1:16:07<3:23:32,  4.19s/it]
 27%|██▋       | 1085/4000 [1:16:12<3:28:26,  4.29s/it]
 27%|██▋       | 1086/4000 [1:16:16<3:26:16,  4.25s/it]
 27%|██▋       | 1087/4000 [1:16:20<3:24:48,  4.22s/it]
 27%|██▋       | 1088/4000 [1:16:24<3:23:57,  4.20s/it]
 27%|██▋       | 1089/4000 [1:16:28<3:23:45,  4.20s/it]
 27%|██▋       | 1090/4000 [1:16:33<3:23:45,  4.20s/it]
                                                       
{'loss': '0.5948', 'grad_norm': '0.1558', 'learning_rate': '0.0004411', 'epoch': '0.2641'}

 27%|██▋       | 1090/4000 [1:16:33<3:23:45,  4.20s/it]
 27%|██▋       | 1091/4000 [1:16:37<3:26:06,  4.25s/it]
 27%|██▋       | 1092/4000 [1:16:41<3:27:25,  4.28s/it]
 27%|██▋       | 1093/4000 [1:16:45<3:25:39,  4.24s/it]
 27%|██▋       | 1094/4000 [1:16:50<3:24:21,  4.22s/it]
 27%|██▋       | 1095/4000 [1:16:54<3:23:10,  4.20s/it]
 27%|██▋       | 1096/4000 [1:16:58<3:22:48,  4.19s/it]
 27%|██▋       | 1097/4000 [1:17:02<3:22:44,  4.19s/it]
 27%|██▋       | 1098/4000 [1:17:07<3:28:17,  4.31s/it]
 27%|██▋       | 1099/4000 [1:17:11<3:26:16,  4.27s/it]
 28%|██▊       | 1100/4000 [1:17:15<3:24:41,  4.24s/it]
                                                       
{'loss': '0.5363', 'grad_norm': '0.2345', 'learning_rate': '0.0004395', 'epoch': '0.2665'}

 28%|██▊       | 1100/4000 [1:17:15<3:24:41,  4.24s/it]
 28%|██▊       | 1101/4000 [1:17:19<3:23:34,  4.21s/it]
 28%|██▊       | 1102/4000 [1:17:23<3:22:49,  4.20s/it]
 28%|██▊       | 1103/4000 [1:17:27<3:22:32,  4.19s/it]
 28%|██▊       | 1104/4000 [1:17:32<3:22:08,  4.19s/it]
 28%|██▊       | 1105/4000 [1:17:36<3:26:39,  4.28s/it]
 28%|██▊       | 1106/4000 [1:17:40<3:24:41,  4.24s/it]
 28%|██▊       | 1107/4000 [1:17:44<3:23:28,  4.22s/it]
 28%|██▊       | 1108/4000 [1:17:49<3:22:22,  4.20s/it]
 28%|██▊       | 1109/4000 [1:17:53<3:21:30,  4.18s/it]
 28%|██▊       | 1110/4000 [1:17:57<3:21:02,  4.17s/it]
                                                       
{'loss': '0.5761', 'grad_norm': '0.1357', 'learning_rate': '0.000438', 'epoch': '0.269'}

 28%|██▊       | 1110/4000 [1:17:57<3:21:02,  4.17s/it]
 28%|██▊       | 1111/4000 [1:18:01<3:20:43,  4.17s/it]
 28%|██▊       | 1112/4000 [1:18:06<3:26:13,  4.28s/it]
 28%|██▊       | 1113/4000 [1:18:10<3:24:13,  4.24s/it]
 28%|██▊       | 1114/4000 [1:18:14<3:23:01,  4.22s/it]
 28%|██▊       | 1115/4000 [1:18:18<3:22:10,  4.20s/it]
 28%|██▊       | 1116/4000 [1:18:22<3:21:41,  4.20s/it]
 28%|██▊       | 1117/4000 [1:18:26<3:21:20,  4.19s/it]
 28%|██▊       | 1118/4000 [1:18:31<3:23:45,  4.24s/it]
 28%|██▊       | 1119/4000 [1:18:35<3:25:22,  4.28s/it]
 28%|██▊       | 1120/4000 [1:18:39<3:23:31,  4.24s/it]
                                                       
{'loss': '0.5042', 'grad_norm': '0.119', 'learning_rate': '0.0004365', 'epoch': '0.2714'}

 28%|██▊       | 1120/4000 [1:18:39<3:23:31,  4.24s/it]
 28%|██▊       | 1121/4000 [1:18:44<3:22:14,  4.21s/it]
 28%|██▊       | 1122/4000 [1:18:48<3:21:13,  4.20s/it]
 28%|██▊       | 1123/4000 [1:18:52<3:20:30,  4.18s/it]
 28%|██▊       | 1124/4000 [1:18:56<3:23:00,  4.24s/it]
 28%|██▊       | 1125/4000 [1:19:01<3:27:01,  4.32s/it]
 28%|██▊       | 1126/4000 [1:19:05<3:24:39,  4.27s/it]
 28%|██▊       | 1127/4000 [1:19:09<3:22:43,  4.23s/it]
 28%|██▊       | 1128/4000 [1:19:13<3:21:34,  4.21s/it]
 28%|██▊       | 1129/4000 [1:19:17<3:20:55,  4.20s/it]
 28%|██▊       | 1130/4000 [1:19:22<3:20:33,  4.19s/it]
                                                       
{'loss': '0.6019', 'grad_norm': '0.1415', 'learning_rate': '0.000435', 'epoch': '0.2738'}

 28%|██▊       | 1130/4000 [1:19:22<3:20:33,  4.19s/it]
 28%|██▊       | 1131/4000 [1:19:26<3:19:55,  4.18s/it]
 28%|██▊       | 1132/4000 [1:19:30<3:25:01,  4.29s/it]
 28%|██▊       | 1133/4000 [1:19:34<3:23:10,  4.25s/it]
 28%|██▊       | 1134/4000 [1:19:39<3:21:44,  4.22s/it]
 28%|██▊       | 1135/4000 [1:19:43<3:20:48,  4.21s/it]
 28%|██▊       | 1136/4000 [1:19:47<3:19:42,  4.18s/it]
 28%|██▊       | 1137/4000 [1:19:51<3:19:15,  4.18s/it]
 28%|██▊       | 1138/4000 [1:19:55<3:19:04,  4.17s/it]
 28%|██▊       | 1139/4000 [1:20:00<3:24:02,  4.28s/it]
 28%|██▊       | 1140/4000 [1:20:04<3:22:04,  4.24s/it]
                                                       
{'loss': '0.5809', 'grad_norm': '0.1289', 'learning_rate': '0.0004335', 'epoch': '0.2762'}

 28%|██▊       | 1140/4000 [1:20:04<3:22:04,  4.24s/it]
 29%|██▊       | 1141/4000 [1:20:08<3:20:53,  4.22s/it]
 29%|██▊       | 1142/4000 [1:20:12<3:19:56,  4.20s/it]
 29%|██▊       | 1143/4000 [1:20:16<3:18:57,  4.18s/it]
 29%|██▊       | 1144/4000 [1:20:20<3:18:23,  4.17s/it]
 29%|██▊       | 1145/4000 [1:20:25<3:20:43,  4.22s/it]
 29%|██▊       | 1146/4000 [1:20:29<3:22:21,  4.25s/it]
 29%|██▊       | 1147/4000 [1:20:33<3:20:44,  4.22s/it]
 29%|██▊       | 1148/4000 [1:20:37<3:19:33,  4.20s/it]
 29%|██▊       | 1149/4000 [1:20:42<3:18:51,  4.18s/it]
 29%|██▉       | 1150/4000 [1:20:46<3:18:19,  4.18s/it]
                                                       
{'loss': '0.5696', 'grad_norm': '0.1419', 'learning_rate': '0.000432', 'epoch': '0.2786'}

 29%|██▉       | 1150/4000 [1:20:46<3:18:19,  4.18s/it]
 29%|██▉       | 1151/4000 [1:20:50<3:17:59,  4.17s/it]
 29%|██▉       | 1152/4000 [1:20:54<3:23:10,  4.28s/it]
 29%|██▉       | 1153/4000 [1:20:59<3:21:13,  4.24s/it]
 29%|██▉       | 1154/4000 [1:21:03<3:19:54,  4.21s/it]
 29%|██▉       | 1155/4000 [1:21:07<3:18:54,  4.19s/it]
 29%|██▉       | 1156/4000 [1:21:11<3:18:08,  4.18s/it]
 29%|██▉       | 1157/4000 [1:21:15<3:17:38,  4.17s/it]
 29%|██▉       | 1158/4000 [1:21:19<3:17:23,  4.17s/it]
 29%|██▉       | 1159/4000 [1:21:24<3:22:44,  4.28s/it]
 29%|██▉       | 1160/4000 [1:21:28<3:20:43,  4.24s/it]
                                                       
{'loss': '0.5416', 'grad_norm': '0.1512', 'learning_rate': '0.0004305', 'epoch': '0.2811'}

 29%|██▉       | 1160/4000 [1:21:28<3:20:43,  4.24s/it]
 29%|██▉       | 1161/4000 [1:21:32<3:19:19,  4.21s/it]
 29%|██▉       | 1162/4000 [1:21:36<3:18:27,  4.20s/it]
 29%|██▉       | 1163/4000 [1:21:40<3:17:48,  4.18s/it]
 29%|██▉       | 1164/4000 [1:21:45<3:17:17,  4.17s/it]
 29%|██▉       | 1165/4000 [1:21:49<3:16:48,  4.17s/it]
 29%|██▉       | 1166/4000 [1:21:53<3:22:26,  4.29s/it]
 29%|██▉       | 1167/4000 [1:21:57<3:20:54,  4.25s/it]
 29%|██▉       | 1168/4000 [1:22:02<3:19:58,  4.24s/it]
 29%|██▉       | 1169/4000 [1:22:06<3:18:47,  4.21s/it]
 29%|██▉       | 1170/4000 [1:22:10<3:17:49,  4.19s/it]
                                                       
{'loss': '0.5622', 'grad_norm': '0.1497', 'learning_rate': '0.0004289', 'epoch': '0.2835'}

 29%|██▉       | 1170/4000 [1:22:10<3:17:49,  4.19s/it]
 29%|██▉       | 1171/4000 [1:22:14<3:17:13,  4.18s/it]
 29%|██▉       | 1172/4000 [1:22:19<3:21:58,  4.29s/it]
 29%|██▉       | 1173/4000 [1:22:23<3:20:18,  4.25s/it]
 29%|██▉       | 1174/4000 [1:22:27<3:18:46,  4.22s/it]
 29%|██▉       | 1175/4000 [1:22:31<3:17:43,  4.20s/it]
 29%|██▉       | 1176/4000 [1:22:35<3:17:01,  4.19s/it]
 29%|██▉       | 1177/4000 [1:22:39<3:16:36,  4.18s/it]
 29%|██▉       | 1178/4000 [1:22:44<3:16:44,  4.18s/it]
 29%|██▉       | 1179/4000 [1:22:48<3:22:28,  4.31s/it]
 30%|██▉       | 1180/4000 [1:22:52<3:20:41,  4.27s/it]
                                                       
{'loss': '0.6044', 'grad_norm': '0.1439', 'learning_rate': '0.0004274', 'epoch': '0.2859'}

 30%|██▉       | 1180/4000 [1:22:52<3:20:41,  4.27s/it]
 30%|██▉       | 1181/4000 [1:22:57<3:18:56,  4.23s/it]
 30%|██▉       | 1182/4000 [1:23:01<3:17:52,  4.21s/it]
 30%|██▉       | 1183/4000 [1:23:05<3:17:08,  4.20s/it]
 30%|██▉       | 1184/4000 [1:23:09<3:16:52,  4.19s/it]
 30%|██▉       | 1185/4000 [1:23:13<3:16:52,  4.20s/it]
 30%|██▉       | 1186/4000 [1:23:18<3:21:37,  4.30s/it]
 30%|██▉       | 1187/4000 [1:23:22<3:19:35,  4.26s/it]
 30%|██▉       | 1188/4000 [1:23:26<3:17:59,  4.22s/it]
 30%|██▉       | 1189/4000 [1:23:30<3:16:59,  4.20s/it]
 30%|██▉       | 1190/4000 [1:23:34<3:16:08,  4.19s/it]
                                                       
{'loss': '0.5693', 'grad_norm': '0.153', 'learning_rate': '0.0004259', 'epoch': '0.2883'}

 30%|██▉       | 1190/4000 [1:23:34<3:16:08,  4.19s/it]
 30%|██▉       | 1191/4000 [1:23:39<3:15:39,  4.18s/it]
 30%|██▉       | 1192/4000 [1:23:43<3:15:28,  4.18s/it]
 30%|██▉       | 1193/4000 [1:23:47<3:20:37,  4.29s/it]
 30%|██▉       | 1194/4000 [1:23:51<3:18:51,  4.25s/it]
 30%|██▉       | 1195/4000 [1:23:56<3:17:26,  4.22s/it]
 30%|██▉       | 1196/4000 [1:24:00<3:16:29,  4.20s/it]
 30%|██▉       | 1197/4000 [1:24:04<3:15:49,  4.19s/it]
 30%|██▉       | 1198/4000 [1:24:08<3:15:22,  4.18s/it]
 30%|██▉       | 1199/4000 [1:24:13<3:20:43,  4.30s/it]
 30%|███       | 1200/4000 [1:24:17<3:18:41,  4.26s/it]
                                                       
{'loss': '0.5716', 'grad_norm': '0.1361', 'learning_rate': '0.0004244', 'epoch': '0.2908'}

 30%|███       | 1200/4000 [1:24:17<3:18:41,  4.26s/it]
 30%|███       | 1201/4000 [1:24:21<3:17:18,  4.23s/it]
 30%|███       | 1202/4000 [1:24:25<3:16:15,  4.21s/it]
 30%|███       | 1203/4000 [1:24:29<3:15:29,  4.19s/it]
 30%|███       | 1204/4000 [1:24:33<3:14:50,  4.18s/it]
 30%|███       | 1205/4000 [1:24:38<3:14:28,  4.17s/it]
 30%|███       | 1206/4000 [1:24:42<3:19:28,  4.28s/it]
 30%|███       | 1207/4000 [1:24:46<3:17:47,  4.25s/it]
 30%|███       | 1208/4000 [1:24:51<3:16:19,  4.22s/it]
 30%|███       | 1209/4000 [1:24:55<3:15:17,  4.20s/it]
 30%|███       | 1210/4000 [1:24:59<3:14:36,  4.18s/it]
                                                       
{'loss': '0.5381', 'grad_norm': '0.1753', 'learning_rate': '0.0004229', 'epoch': '0.2932'}

 30%|███       | 1210/4000 [1:24:59<3:14:36,  4.18s/it]
 30%|███       | 1211/4000 [1:25:03<3:14:05,  4.18s/it]
 30%|███       | 1212/4000 [1:25:07<3:13:43,  4.17s/it]
 30%|███       | 1213/4000 [1:25:12<3:18:58,  4.28s/it]
 30%|███       | 1214/4000 [1:25:16<3:17:14,  4.25s/it]
 30%|███       | 1215/4000 [1:25:20<3:15:55,  4.22s/it]
 30%|███       | 1216/4000 [1:25:24<3:15:07,  4.21s/it]
 30%|███       | 1217/4000 [1:25:28<3:14:18,  4.19s/it]
 30%|███       | 1218/4000 [1:25:32<3:13:48,  4.18s/it]
 30%|███       | 1219/4000 [1:25:37<3:16:05,  4.23s/it]
 30%|███       | 1220/4000 [1:25:41<3:17:36,  4.27s/it]
                                                       
{'loss': '0.5334', 'grad_norm': '0.1509', 'learning_rate': '0.0004214', 'epoch': '0.2956'}

 30%|███       | 1220/4000 [1:25:41<3:17:36,  4.27s/it]
 31%|███       | 1221/4000 [1:25:45<3:16:02,  4.23s/it]
 31%|███       | 1222/4000 [1:25:50<3:15:23,  4.22s/it]
 31%|███       | 1223/4000 [1:25:54<3:14:59,  4.21s/it]
 31%|███       | 1224/4000 [1:25:58<3:13:58,  4.19s/it]
 31%|███       | 1225/4000 [1:26:02<3:13:32,  4.18s/it]
 31%|███       | 1226/4000 [1:26:07<3:18:29,  4.29s/it]
 31%|███       | 1227/4000 [1:26:11<3:16:28,  4.25s/it]
 31%|███       | 1228/4000 [1:26:15<3:15:00,  4.22s/it]
 31%|███       | 1229/4000 [1:26:19<3:13:43,  4.19s/it]
 31%|███       | 1230/4000 [1:26:23<3:12:57,  4.18s/it]
                                                       
{'loss': '0.5645', 'grad_norm': '0.132', 'learning_rate': '0.0004198', 'epoch': '0.298'}

 31%|███       | 1230/4000 [1:26:23<3:12:57,  4.18s/it]
 31%|███       | 1231/4000 [1:26:27<3:12:27,  4.17s/it]
 31%|███       | 1232/4000 [1:26:31<3:12:04,  4.16s/it]
 31%|███       | 1233/4000 [1:26:36<3:16:45,  4.27s/it]
 31%|███       | 1234/4000 [1:26:40<3:15:06,  4.23s/it]
 31%|███       | 1235/4000 [1:26:44<3:13:55,  4.21s/it]
 31%|███       | 1236/4000 [1:26:48<3:12:59,  4.19s/it]
 31%|███       | 1237/4000 [1:26:53<3:12:23,  4.18s/it]
 31%|███       | 1238/4000 [1:26:57<3:12:04,  4.17s/it]
 31%|███       | 1239/4000 [1:27:01<3:11:49,  4.17s/it]
 31%|███       | 1240/4000 [1:27:05<3:16:50,  4.28s/it]
                                                       
{'loss': '0.5638', 'grad_norm': '0.1434', 'learning_rate': '0.0004183', 'epoch': '0.3005'}

 31%|███       | 1240/4000 [1:27:05<3:16:50,  4.28s/it]
 31%|███       | 1241/4000 [1:27:10<3:15:12,  4.25s/it]
 31%|███       | 1242/4000 [1:27:14<3:14:02,  4.22s/it]
 31%|███       | 1243/4000 [1:27:18<3:13:06,  4.20s/it]
 31%|███       | 1244/4000 [1:27:22<3:12:21,  4.19s/it]
 31%|███       | 1245/4000 [1:27:26<3:11:58,  4.18s/it]
 31%|███       | 1246/4000 [1:27:31<3:14:23,  4.24s/it]
 31%|███       | 1247/4000 [1:27:35<3:15:48,  4.27s/it]
 31%|███       | 1248/4000 [1:27:39<3:14:10,  4.23s/it]
 31%|███       | 1249/4000 [1:27:43<3:13:11,  4.21s/it]
 31%|███▏      | 1250/4000 [1:27:47<3:12:14,  4.19s/it]
                                                       
{'loss': '0.5903', 'grad_norm': '0.135', 'learning_rate': '0.0004168', 'epoch': '0.3029'}

 31%|███▏      | 1250/4000 [1:27:47<3:12:14,  4.19s/it]
 31%|███▏      | 1251/4000 [1:27:52<3:11:32,  4.18s/it]
 31%|███▏      | 1252/4000 [1:27:56<3:11:20,  4.18s/it]
 31%|███▏      | 1253/4000 [1:28:00<3:16:29,  4.29s/it]
 31%|███▏      | 1254/4000 [1:28:04<3:14:34,  4.25s/it]
 31%|███▏      | 1255/4000 [1:28:09<3:13:09,  4.22s/it]
 31%|███▏      | 1256/4000 [1:28:13<3:12:15,  4.20s/it]
 31%|███▏      | 1257/4000 [1:28:17<3:11:23,  4.19s/it]
 31%|███▏      | 1258/4000 [1:28:21<3:10:46,  4.17s/it]
 31%|███▏      | 1259/4000 [1:28:25<3:10:13,  4.16s/it]
 32%|███▏      | 1260/4000 [1:28:30<3:15:17,  4.28s/it]
                                                       
{'loss': '0.5562', 'grad_norm': '0.1464', 'learning_rate': '0.0004153', 'epoch': '0.3053'}

 32%|███▏      | 1260/4000 [1:28:30<3:15:17,  4.28s/it]
 32%|███▏      | 1261/4000 [1:28:34<3:13:37,  4.24s/it]
 32%|███▏      | 1262/4000 [1:28:38<3:12:33,  4.22s/it]
 32%|███▏      | 1263/4000 [1:28:42<3:11:38,  4.20s/it]
 32%|███▏      | 1264/4000 [1:28:46<3:11:11,  4.19s/it]
 32%|███▏      | 1265/4000 [1:28:51<3:10:40,  4.18s/it]
 32%|███▏      | 1266/4000 [1:28:55<3:10:19,  4.18s/it]
 32%|███▏      | 1267/4000 [1:28:59<3:15:34,  4.29s/it]
 32%|███▏      | 1268/4000 [1:29:03<3:13:34,  4.25s/it]
 32%|███▏      | 1269/4000 [1:29:08<3:11:59,  4.22s/it]
 32%|███▏      | 1270/4000 [1:29:12<3:11:14,  4.20s/it]
                                                       
{'loss': '0.5484', 'grad_norm': '0.1255', 'learning_rate': '0.0004138', 'epoch': '0.3077'}

 32%|███▏      | 1270/4000 [1:29:12<3:11:14,  4.20s/it]
 32%|███▏      | 1271/4000 [1:29:16<3:10:39,  4.19s/it]
 32%|███▏      | 1272/4000 [1:29:20<3:10:05,  4.18s/it]
 32%|███▏      | 1273/4000 [1:29:24<3:12:21,  4.23s/it]
 32%|███▏      | 1274/4000 [1:29:29<3:13:37,  4.26s/it]
 32%|███▏      | 1275/4000 [1:29:33<3:12:02,  4.23s/it]
 32%|███▏      | 1276/4000 [1:29:37<3:10:52,  4.20s/it]
 32%|███▏      | 1277/4000 [1:29:41<3:10:16,  4.19s/it]
 32%|███▏      | 1278/4000 [1:29:45<3:09:38,  4.18s/it]
 32%|███▏      | 1279/4000 [1:29:50<3:09:15,  4.17s/it]
 32%|███▏      | 1280/4000 [1:29:54<3:14:27,  4.29s/it]
                                                       
{'loss': '0.5281', 'grad_norm': '0.1772', 'learning_rate': '0.0004123', 'epoch': '0.3101'}

 32%|███▏      | 1280/4000 [1:29:54<3:14:27,  4.29s/it]
 32%|███▏      | 1281/4000 [1:29:58<3:12:35,  4.25s/it]
 32%|███▏      | 1282/4000 [1:30:02<3:11:19,  4.22s/it]
 32%|███▏      | 1283/4000 [1:30:07<3:10:09,  4.20s/it]
 32%|███▏      | 1284/4000 [1:30:11<3:09:26,  4.18s/it]
 32%|███▏      | 1285/4000 [1:30:15<3:09:02,  4.18s/it]
 32%|███▏      | 1286/4000 [1:30:19<3:08:39,  4.17s/it]
 32%|███▏      | 1287/4000 [1:30:24<3:13:15,  4.27s/it]
 32%|███▏      | 1288/4000 [1:30:28<3:11:25,  4.24s/it]
 32%|███▏      | 1289/4000 [1:30:32<3:10:37,  4.22s/it]
 32%|███▏      | 1290/4000 [1:30:36<3:09:49,  4.20s/it]
                                                       
{'loss': '0.5709', 'grad_norm': '0.1399', 'learning_rate': '0.0004108', 'epoch': '0.3126'}

 32%|███▏      | 1290/4000 [1:30:36<3:09:49,  4.20s/it]
 32%|███▏      | 1291/4000 [1:30:40<3:08:58,  4.19s/it]
 32%|███▏      | 1292/4000 [1:30:44<3:08:50,  4.18s/it]
 32%|███▏      | 1293/4000 [1:30:48<3:08:37,  4.18s/it]
 32%|███▏      | 1294/4000 [1:30:53<3:13:00,  4.28s/it]
 32%|███▏      | 1295/4000 [1:30:57<3:11:08,  4.24s/it]
 32%|███▏      | 1296/4000 [1:31:01<3:09:49,  4.21s/it]
 32%|███▏      | 1297/4000 [1:31:05<3:09:06,  4.20s/it]
 32%|███▏      | 1298/4000 [1:31:10<3:08:33,  4.19s/it]
 32%|███▏      | 1299/4000 [1:31:14<3:08:03,  4.18s/it]
 32%|███▎      | 1300/4000 [1:31:18<3:13:12,  4.29s/it]
                                                       
{'loss': '0.5659', 'grad_norm': '0.1292', 'learning_rate': '0.0004092', 'epoch': '0.315'}

 32%|███▎      | 1300/4000 [1:31:18<3:13:12,  4.29s/it]
 33%|███▎      | 1301/4000 [1:31:22<3:11:02,  4.25s/it]
 33%|███▎      | 1302/4000 [1:31:27<3:09:43,  4.22s/it]
 33%|███▎      | 1303/4000 [1:31:31<3:08:41,  4.20s/it]
 33%|███▎      | 1304/4000 [1:31:35<3:07:56,  4.18s/it]
 33%|███▎      | 1305/4000 [1:31:39<3:07:20,  4.17s/it]
 33%|███▎      | 1306/4000 [1:31:43<3:07:05,  4.17s/it]
 33%|███▎      | 1307/4000 [1:31:48<3:11:26,  4.27s/it]
 33%|███▎      | 1308/4000 [1:31:52<3:09:45,  4.23s/it]
 33%|███▎      | 1309/4000 [1:31:56<3:08:50,  4.21s/it]
 33%|███▎      | 1310/4000 [1:32:00<3:08:12,  4.20s/it]
                                                       
{'loss': '0.5598', 'grad_norm': '0.1478', 'learning_rate': '0.0004077', 'epoch': '0.3174'}

 33%|███▎      | 1310/4000 [1:32:00<3:08:12,  4.20s/it]
 33%|███▎      | 1311/4000 [1:32:04<3:07:36,  4.19s/it]
 33%|███▎      | 1312/4000 [1:32:09<3:06:56,  4.17s/it]
 33%|███▎      | 1313/4000 [1:32:13<3:06:32,  4.17s/it]
 33%|███▎      | 1314/4000 [1:32:17<3:11:26,  4.28s/it]
 33%|███▎      | 1315/4000 [1:32:21<3:09:48,  4.24s/it]
 33%|███▎      | 1316/4000 [1:32:26<3:08:41,  4.22s/it]
 33%|███▎      | 1317/4000 [1:32:30<3:08:08,  4.21s/it]
 33%|███▎      | 1318/4000 [1:32:34<3:07:37,  4.20s/it]
 33%|███▎      | 1319/4000 [1:32:38<3:07:28,  4.20s/it]
 33%|███▎      | 1320/4000 [1:32:42<3:07:16,  4.19s/it]
                                                       
{'loss': '0.5577', 'grad_norm': '0.1276', 'learning_rate': '0.0004062', 'epoch': '0.3198'}

 33%|███▎      | 1320/4000 [1:32:42<3:07:16,  4.19s/it]
 33%|███▎      | 1321/4000 [1:32:47<3:12:35,  4.31s/it]
 33%|███▎      | 1322/4000 [1:32:51<3:10:50,  4.28s/it]
 33%|███▎      | 1323/4000 [1:32:55<3:09:02,  4.24s/it]
 33%|███▎      | 1324/4000 [1:32:59<3:08:02,  4.22s/it]
 33%|███▎      | 1325/4000 [1:33:04<3:07:08,  4.20s/it]
 33%|███▎      | 1326/4000 [1:33:08<3:06:27,  4.18s/it]
 33%|███▎      | 1327/4000 [1:33:12<3:10:42,  4.28s/it]
 33%|███▎      | 1328/4000 [1:33:16<3:08:56,  4.24s/it]
 33%|███▎      | 1329/4000 [1:33:20<3:07:26,  4.21s/it]
 33%|███▎      | 1330/4000 [1:33:25<3:06:31,  4.19s/it]
                                                       
{'loss': '0.5051', 'grad_norm': '0.1239', 'learning_rate': '0.0004047', 'epoch': '0.3223'}

 33%|███▎      | 1330/4000 [1:33:25<3:06:31,  4.19s/it]
 33%|███▎      | 1331/4000 [1:33:29<3:05:53,  4.18s/it]
 33%|███▎      | 1332/4000 [1:33:33<3:05:29,  4.17s/it]
 33%|███▎      | 1333/4000 [1:33:37<3:05:08,  4.17s/it]
 33%|███▎      | 1334/4000 [1:33:42<3:10:18,  4.28s/it]
 33%|███▎      | 1335/4000 [1:33:46<3:08:34,  4.25s/it]
 33%|███▎      | 1336/4000 [1:33:50<3:07:39,  4.23s/it]
 33%|███▎      | 1337/4000 [1:33:54<3:06:46,  4.21s/it]
 33%|███▎      | 1338/4000 [1:33:58<3:06:05,  4.19s/it]
 33%|███▎      | 1339/4000 [1:34:02<3:05:37,  4.19s/it]
 34%|███▎      | 1340/4000 [1:34:07<3:05:37,  4.19s/it]
                                                       
{'loss': '0.5823', 'grad_norm': '0.1593', 'learning_rate': '0.0004032', 'epoch': '0.3247'}

 34%|███▎      | 1340/4000 [1:34:07<3:05:37,  4.19s/it]
 34%|███▎      | 1341/4000 [1:34:11<3:10:08,  4.29s/it]
 34%|███▎      | 1342/4000 [1:34:15<3:08:29,  4.25s/it]
 34%|███▎      | 1343/4000 [1:34:20<3:07:10,  4.23s/it]
 34%|███▎      | 1344/4000 [1:34:24<3:06:25,  4.21s/it]
 34%|███▎      | 1345/4000 [1:34:28<3:05:38,  4.20s/it]
 34%|███▎      | 1346/4000 [1:34:32<3:04:57,  4.18s/it]
 34%|███▎      | 1347/4000 [1:34:36<3:07:04,  4.23s/it]
 34%|███▎      | 1348/4000 [1:34:41<3:08:25,  4.26s/it]
 34%|███▎      | 1349/4000 [1:34:45<3:06:51,  4.23s/it]
 34%|███▍      | 1350/4000 [1:34:49<3:05:57,  4.21s/it]
                                                       
{'loss': '0.5374', 'grad_norm': '0.1937', 'learning_rate': '0.0004017', 'epoch': '0.3271'}

 34%|███▍      | 1350/4000 [1:34:49<3:05:57,  4.21s/it]
 34%|███▍      | 1351/4000 [1:34:53<3:05:16,  4.20s/it]
 34%|███▍      | 1352/4000 [1:34:57<3:04:56,  4.19s/it]
 34%|███▍      | 1353/4000 [1:35:01<3:04:29,  4.18s/it]
 34%|███▍      | 1354/4000 [1:35:06<3:09:49,  4.30s/it]
 34%|███▍      | 1355/4000 [1:35:10<3:08:05,  4.27s/it]
 34%|███▍      | 1356/4000 [1:35:14<3:06:53,  4.24s/it]
 34%|███▍      | 1357/4000 [1:35:19<3:05:48,  4.22s/it]
 34%|███▍      | 1358/4000 [1:35:23<3:04:49,  4.20s/it]
 34%|███▍      | 1359/4000 [1:35:27<3:04:28,  4.19s/it]
 34%|███▍      | 1360/4000 [1:35:31<3:04:04,  4.18s/it]
                                                       
{'loss': '0.5148', 'grad_norm': '0.1512', 'learning_rate': '0.0004002', 'epoch': '0.3295'}

 34%|███▍      | 1360/4000 [1:35:31<3:04:04,  4.18s/it]
 34%|███▍      | 1361/4000 [1:35:36<3:08:28,  4.29s/it]
 34%|███▍      | 1362/4000 [1:35:40<3:06:54,  4.25s/it]
 34%|███▍      | 1363/4000 [1:35:44<3:05:48,  4.23s/it]
 34%|███▍      | 1364/4000 [1:35:48<3:04:53,  4.21s/it]
 34%|███▍      | 1365/4000 [1:35:52<3:04:20,  4.20s/it]
 34%|███▍      | 1366/4000 [1:35:56<3:04:03,  4.19s/it]
 34%|███▍      | 1367/4000 [1:36:01<3:03:27,  4.18s/it]
 34%|███▍      | 1368/4000 [1:36:05<3:08:30,  4.30s/it]
 34%|███▍      | 1369/4000 [1:36:09<3:06:32,  4.25s/it]
 34%|███▍      | 1370/4000 [1:36:14<3:05:21,  4.23s/it]
                                                       
{'loss': '0.5666', 'grad_norm': '0.1293', 'learning_rate': '0.0003986', 'epoch': '0.332'}

 34%|███▍      | 1370/4000 [1:36:14<3:05:21,  4.23s/it]
 34%|███▍      | 1371/4000 [1:36:18<3:04:24,  4.21s/it]
 34%|███▍      | 1372/4000 [1:36:22<3:03:47,  4.20s/it]
 34%|███▍      | 1373/4000 [1:36:26<3:03:15,  4.19s/it]
 34%|███▍      | 1374/4000 [1:36:30<3:05:20,  4.23s/it]
 34%|███▍      | 1375/4000 [1:36:35<3:06:30,  4.26s/it]
 34%|███▍      | 1376/4000 [1:36:39<3:05:02,  4.23s/it]
 34%|███▍      | 1377/4000 [1:36:43<3:04:09,  4.21s/it]
 34%|███▍      | 1378/4000 [1:36:47<3:03:36,  4.20s/it]
 34%|███▍      | 1379/4000 [1:36:51<3:03:09,  4.19s/it]
 34%|███▍      | 1380/4000 [1:36:56<3:02:41,  4.18s/it]
                                                       
{'loss': '0.5444', 'grad_norm': '0.1459', 'learning_rate': '0.0003971', 'epoch': '0.3344'}

 34%|███▍      | 1380/4000 [1:36:56<3:02:41,  4.18s/it]
 35%|███▍      | 1381/4000 [1:37:00<3:07:06,  4.29s/it]
 35%|███▍      | 1382/4000 [1:37:04<3:05:36,  4.25s/it]
 35%|███▍      | 1383/4000 [1:37:08<3:04:19,  4.23s/it]
 35%|███▍      | 1384/4000 [1:37:13<3:03:30,  4.21s/it]
 35%|███▍      | 1385/4000 [1:37:17<3:02:49,  4.19s/it]
 35%|███▍      | 1386/4000 [1:37:21<3:02:34,  4.19s/it]
 35%|███▍      | 1387/4000 [1:37:25<3:02:11,  4.18s/it]
 35%|███▍      | 1388/4000 [1:37:30<3:07:31,  4.31s/it]
 35%|███▍      | 1389/4000 [1:37:34<3:05:43,  4.27s/it]
 35%|███▍      | 1390/4000 [1:37:38<3:04:17,  4.24s/it]
                                                       
{'loss': '0.5412', 'grad_norm': '0.1493', 'learning_rate': '0.0003956', 'epoch': '0.3368'}

 35%|███▍      | 1390/4000 [1:37:38<3:04:17,  4.24s/it]
 35%|███▍      | 1391/4000 [1:37:42<3:03:28,  4.22s/it]
 35%|███▍      | 1392/4000 [1:37:46<3:02:50,  4.21s/it]
 35%|███▍      | 1393/4000 [1:37:51<3:02:13,  4.19s/it]
 35%|███▍      | 1394/4000 [1:37:55<3:01:49,  4.19s/it]
 35%|███▍      | 1395/4000 [1:37:59<3:06:19,  4.29s/it]
 35%|███▍      | 1396/4000 [1:38:03<3:04:32,  4.25s/it]
 35%|███▍      | 1397/4000 [1:38:08<3:03:17,  4.23s/it]
 35%|███▍      | 1398/4000 [1:38:12<3:03:07,  4.22s/it]
 35%|███▍      | 1399/4000 [1:38:16<3:02:24,  4.21s/it]
 35%|███▌      | 1400/4000 [1:38:20<3:01:40,  4.19s/it]
                                                       
{'loss': '0.4633', 'grad_norm': '0.1313', 'learning_rate': '0.0003941', 'epoch': '0.3392'}

 35%|███▌      | 1400/4000 [1:38:20<3:01:40,  4.19s/it]
 35%|███▌      | 1401/4000 [1:38:24<3:03:57,  4.25s/it]
 35%|███▌      | 1402/4000 [1:38:29<3:05:01,  4.27s/it]
 35%|███▌      | 1403/4000 [1:38:33<3:03:36,  4.24s/it]
 35%|███▌      | 1404/4000 [1:38:37<3:02:30,  4.22s/it]
 35%|███▌      | 1405/4000 [1:38:41<3:01:36,  4.20s/it]
 35%|███▌      | 1406/4000 [1:38:45<3:01:06,  4.19s/it]
 35%|███▌      | 1407/4000 [1:38:50<3:00:54,  4.19s/it]
 35%|███▌      | 1408/4000 [1:38:54<3:05:35,  4.30s/it]
 35%|███▌      | 1409/4000 [1:38:58<3:03:52,  4.26s/it]
 35%|███▌      | 1410/4000 [1:39:03<3:02:44,  4.23s/it]
                                                       
{'loss': '0.5398', 'grad_norm': '0.1156', 'learning_rate': '0.0003926', 'epoch': '0.3416'}

 35%|███▌      | 1410/4000 [1:39:03<3:02:44,  4.23s/it]
 35%|███▌      | 1411/4000 [1:39:07<3:01:54,  4.22s/it]
 35%|███▌      | 1412/4000 [1:39:11<3:01:19,  4.20s/it]
 35%|███▌      | 1413/4000 [1:39:15<3:00:37,  4.19s/it]
 35%|███▌      | 1414/4000 [1:39:19<3:00:07,  4.18s/it]
 35%|███▌      | 1415/4000 [1:39:24<3:04:24,  4.28s/it]
 35%|███▌      | 1416/4000 [1:39:28<3:03:04,  4.25s/it]
 35%|███▌      | 1417/4000 [1:39:32<3:02:15,  4.23s/it]
 35%|███▌      | 1418/4000 [1:39:36<3:01:25,  4.22s/it]
 35%|███▌      | 1419/4000 [1:39:40<3:00:52,  4.20s/it]
 36%|███▌      | 1420/4000 [1:39:45<3:00:18,  4.19s/it]
                                                       
{'loss': '0.5687', 'grad_norm': '0.1501', 'learning_rate': '0.0003911', 'epoch': '0.3441'}

 36%|███▌      | 1420/4000 [1:39:45<3:00:18,  4.19s/it]
 36%|███▌      | 1421/4000 [1:39:49<2:59:46,  4.18s/it]
 36%|███▌      | 1422/4000 [1:39:53<3:04:07,  4.29s/it]
 36%|███▌      | 1423/4000 [1:39:57<3:02:43,  4.25s/it]
 36%|███▌      | 1424/4000 [1:40:02<3:01:45,  4.23s/it]
 36%|███▌      | 1425/4000 [1:40:06<3:00:46,  4.21s/it]
 36%|███▌      | 1426/4000 [1:40:10<3:00:27,  4.21s/it]
 36%|███▌      | 1427/4000 [1:40:14<2:59:49,  4.19s/it]
 36%|███▌      | 1428/4000 [1:40:19<3:04:35,  4.31s/it]
 36%|███▌      | 1429/4000 [1:40:23<3:03:11,  4.28s/it]
 36%|███▌      | 1430/4000 [1:40:27<3:01:50,  4.25s/it]
                                                       
{'loss': '0.5372', 'grad_norm': '0.2107', 'learning_rate': '0.0003895', 'epoch': '0.3465'}

 36%|███▌      | 1430/4000 [1:40:27<3:01:50,  4.25s/it]
 36%|███▌      | 1431/4000 [1:40:31<3:00:48,  4.22s/it]
 36%|███▌      | 1432/4000 [1:40:35<3:00:05,  4.21s/it]
 36%|███▌      | 1433/4000 [1:40:40<2:59:52,  4.20s/it]
 36%|███▌      | 1434/4000 [1:40:44<2:59:05,  4.19s/it]
 36%|███▌      | 1435/4000 [1:40:48<3:03:21,  4.29s/it]
 36%|███▌      | 1436/4000 [1:40:53<3:01:49,  4.25s/it]
 36%|███▌      | 1437/4000 [1:40:57<3:01:09,  4.24s/it]
 36%|███▌      | 1438/4000 [1:41:01<3:00:21,  4.22s/it]
 36%|███▌      | 1439/4000 [1:41:05<2:59:43,  4.21s/it]
 36%|███▌      | 1440/4000 [1:41:09<2:59:14,  4.20s/it]
                                                       
{'loss': '0.5215', 'grad_norm': '0.1397', 'learning_rate': '0.000388', 'epoch': '0.3489'}

 36%|███▌      | 1440/4000 [1:41:09<2:59:14,  4.20s/it]
 36%|███▌      | 1441/4000 [1:41:13<2:59:09,  4.20s/it]
 36%|███▌      | 1442/4000 [1:41:18<3:03:28,  4.30s/it]
 36%|███▌      | 1443/4000 [1:41:22<3:01:43,  4.26s/it]
 36%|███▌      | 1444/4000 [1:41:26<3:00:29,  4.24s/it]
 36%|███▌      | 1445/4000 [1:41:31<2:59:48,  4.22s/it]
 36%|███▌      | 1446/4000 [1:41:35<2:59:05,  4.21s/it]
 36%|███▌      | 1447/4000 [1:41:39<2:58:33,  4.20s/it]
 36%|███▌      | 1448/4000 [1:41:43<2:57:53,  4.18s/it]
 36%|███▌      | 1449/4000 [1:41:48<3:02:24,  4.29s/it]
 36%|███▋      | 1450/4000 [1:41:52<3:00:33,  4.25s/it]
                                                       
{'loss': '0.5467', 'grad_norm': '0.1459', 'learning_rate': '0.0003865', 'epoch': '0.3513'}

 36%|███▋      | 1450/4000 [1:41:52<3:00:33,  4.25s/it]
 36%|███▋      | 1451/4000 [1:41:56<2:59:14,  4.22s/it]
 36%|███▋      | 1452/4000 [1:42:00<2:58:14,  4.20s/it]
 36%|███▋      | 1453/4000 [1:42:04<2:58:03,  4.19s/it]
 36%|███▋      | 1454/4000 [1:42:08<2:57:53,  4.19s/it]
 36%|███▋      | 1455/4000 [1:42:13<3:03:02,  4.32s/it]
 36%|███▋      | 1456/4000 [1:42:17<3:01:06,  4.27s/it]
 36%|███▋      | 1457/4000 [1:42:21<2:59:43,  4.24s/it]
 36%|███▋      | 1458/4000 [1:42:26<2:58:39,  4.22s/it]
 36%|███▋      | 1459/4000 [1:42:30<2:57:46,  4.20s/it]
 36%|███▋      | 1460/4000 [1:42:34<2:57:20,  4.19s/it]
                                                       
{'loss': '0.533', 'grad_norm': '0.1424', 'learning_rate': '0.000385', 'epoch': '0.3538'}

 36%|███▋      | 1460/4000 [1:42:34<2:57:20,  4.19s/it]
 37%|███▋      | 1461/4000 [1:42:38<2:57:02,  4.18s/it]
 37%|███▋      | 1462/4000 [1:42:42<3:00:43,  4.27s/it]
 37%|███▋      | 1463/4000 [1:42:47<2:58:53,  4.23s/it]
 37%|███▋      | 1464/4000 [1:42:51<2:57:42,  4.20s/it]
 37%|███▋      | 1465/4000 [1:42:55<2:56:48,  4.18s/it]
 37%|███▋      | 1466/4000 [1:42:59<2:56:23,  4.18s/it]
 37%|███▋      | 1467/4000 [1:43:03<2:55:52,  4.17s/it]
 37%|███▋      | 1468/4000 [1:43:07<2:55:33,  4.16s/it]
 37%|███▋      | 1469/4000 [1:43:12<3:00:10,  4.27s/it]
 37%|███▋      | 1470/4000 [1:43:16<2:58:41,  4.24s/it]
                                                       
{'loss': '0.5525', 'grad_norm': '0.1409', 'learning_rate': '0.0003835', 'epoch': '0.3562'}

 37%|███▋      | 1470/4000 [1:43:16<2:58:41,  4.24s/it]
 37%|███▋      | 1471/4000 [1:43:20<2:57:29,  4.21s/it]
 37%|███▋      | 1472/4000 [1:43:24<2:56:41,  4.19s/it]
 37%|███▋      | 1473/4000 [1:43:28<2:56:03,  4.18s/it]
 37%|███▋      | 1474/4000 [1:43:33<2:55:55,  4.18s/it]
 37%|███▋      | 1475/4000 [1:43:37<2:58:04,  4.23s/it]
 37%|███▋      | 1476/4000 [1:43:41<2:59:05,  4.26s/it]
 37%|███▋      | 1477/4000 [1:43:45<2:57:52,  4.23s/it]
 37%|███▋      | 1478/4000 [1:43:50<2:56:55,  4.21s/it]
 37%|███▋      | 1479/4000 [1:43:54<2:56:22,  4.20s/it]
 37%|███▋      | 1480/4000 [1:43:58<2:56:01,  4.19s/it]
                                                       
{'loss': '0.5515', 'grad_norm': '0.1436', 'learning_rate': '0.000382', 'epoch': '0.3586'}

 37%|███▋      | 1480/4000 [1:43:58<2:56:01,  4.19s/it]
 37%|███▋      | 1481/4000 [1:44:02<2:55:44,  4.19s/it]
 37%|███▋      | 1482/4000 [1:44:07<2:59:57,  4.29s/it]
 37%|███▋      | 1483/4000 [1:44:11<2:58:25,  4.25s/it]
 37%|███▋      | 1484/4000 [1:44:15<2:57:21,  4.23s/it]
 37%|███▋      | 1485/4000 [1:44:19<2:56:15,  4.20s/it]
 37%|███▋      | 1486/4000 [1:44:23<2:55:21,  4.18s/it]
 37%|███▋      | 1487/4000 [1:44:27<2:54:59,  4.18s/it]
 37%|███▋      | 1488/4000 [1:44:32<2:54:30,  4.17s/it]
 37%|███▋      | 1489/4000 [1:44:36<2:58:47,  4.27s/it]
 37%|███▋      | 1490/4000 [1:44:40<2:57:02,  4.23s/it]
                                                       
{'loss': '0.499', 'grad_norm': '0.1194', 'learning_rate': '0.0003805', 'epoch': '0.361'}

 37%|███▋      | 1490/4000 [1:44:40<2:57:02,  4.23s/it]
 37%|███▋      | 1491/4000 [1:44:44<2:56:11,  4.21s/it]
 37%|███▋      | 1492/4000 [1:44:49<2:55:21,  4.20s/it]
 37%|███▋      | 1493/4000 [1:44:53<2:54:29,  4.18s/it]
 37%|███▋      | 1494/4000 [1:44:57<2:54:00,  4.17s/it]
 37%|███▋      | 1495/4000 [1:45:01<2:53:47,  4.16s/it]
 37%|███▋      | 1496/4000 [1:45:06<2:58:02,  4.27s/it]
 37%|███▋      | 1497/4000 [1:45:10<2:56:20,  4.23s/it]
 37%|███▋      | 1498/4000 [1:45:14<2:55:11,  4.20s/it]
 37%|███▋      | 1499/4000 [1:45:18<2:54:31,  4.19s/it]
 38%|███▊      | 1500/4000 [1:45:22<2:53:50,  4.17s/it]
                                                       
{'loss': '0.5304', 'grad_norm': '0.1319', 'learning_rate': '0.0003789', 'epoch': '0.3634'}

 38%|███▊      | 1500/4000 [1:45:22<2:53:50,  4.17s/it]
 38%|███▊      | 1501/4000 [1:45:26<2:53:21,  4.16s/it]
 38%|███▊      | 1502/4000 [1:45:30<2:52:59,  4.16s/it]
 38%|███▊      | 1503/4000 [1:45:35<2:57:05,  4.26s/it]
 38%|███▊      | 1504/4000 [1:45:39<2:55:47,  4.23s/it]
 38%|███▊      | 1505/4000 [1:45:43<2:54:44,  4.20s/it]
 38%|███▊      | 1506/4000 [1:45:47<2:54:01,  4.19s/it]
 38%|███▊      | 1507/4000 [1:45:51<2:53:26,  4.17s/it]
 38%|███▊      | 1508/4000 [1:45:56<2:53:09,  4.17s/it]
 38%|███▊      | 1509/4000 [1:46:00<2:57:26,  4.27s/it]
 38%|███▊      | 1510/4000 [1:46:04<2:55:41,  4.23s/it]
                                                       
{'loss': '0.4838', 'grad_norm': '0.1603', 'learning_rate': '0.0003774', 'epoch': '0.3659'}

 38%|███▊      | 1510/4000 [1:46:04<2:55:41,  4.23s/it]
 38%|███▊      | 1511/4000 [1:46:08<2:54:32,  4.21s/it]
 38%|███▊      | 1512/4000 [1:46:13<2:53:58,  4.20s/it]
 38%|███▊      | 1513/4000 [1:46:17<2:53:09,  4.18s/it]
 38%|███▊      | 1514/4000 [1:46:21<2:52:39,  4.17s/it]
 38%|███▊      | 1515/4000 [1:46:25<2:52:20,  4.16s/it]
 38%|███▊      | 1516/4000 [1:46:30<2:56:48,  4.27s/it]
 38%|███▊      | 1517/4000 [1:46:34<2:54:57,  4.23s/it]
 38%|███▊      | 1518/4000 [1:46:38<2:53:46,  4.20s/it]
 38%|███▊      | 1519/4000 [1:46:42<2:52:58,  4.18s/it]
 38%|███▊      | 1520/4000 [1:46:46<2:52:23,  4.17s/it]
                                                       
{'loss': '0.526', 'grad_norm': '0.1302', 'learning_rate': '0.0003759', 'epoch': '0.3683'}

 38%|███▊      | 1520/4000 [1:46:46<2:52:23,  4.17s/it]
 38%|███▊      | 1521/4000 [1:46:50<2:52:09,  4.17s/it]
 38%|███▊      | 1522/4000 [1:46:54<2:51:36,  4.15s/it]
 38%|███▊      | 1523/4000 [1:46:59<2:55:43,  4.26s/it]
 38%|███▊      | 1524/4000 [1:47:03<2:54:07,  4.22s/it]
 38%|███▊      | 1525/4000 [1:47:07<2:53:19,  4.20s/it]
 38%|███▊      | 1526/4000 [1:47:11<2:52:28,  4.18s/it]
 38%|███▊      | 1527/4000 [1:47:15<2:51:56,  4.17s/it]
 38%|███▊      | 1528/4000 [1:47:20<2:51:35,  4.16s/it]
 38%|███▊      | 1529/4000 [1:47:24<2:53:40,  4.22s/it]
 38%|███▊      | 1530/4000 [1:47:28<2:55:05,  4.25s/it]
                                                       
{'loss': '0.5107', 'grad_norm': '0.1193', 'learning_rate': '0.0003744', 'epoch': '0.3707'}

 38%|███▊      | 1530/4000 [1:47:28<2:55:05,  4.25s/it]
 38%|███▊      | 1531/4000 [1:47:32<2:53:45,  4.22s/it]
 38%|███▊      | 1532/4000 [1:47:37<2:52:52,  4.20s/it]
 38%|███▊      | 1533/4000 [1:47:41<2:52:19,  4.19s/it]
 38%|███▊      | 1534/4000 [1:47:45<2:51:37,  4.18s/it]
 38%|███▊      | 1535/4000 [1:47:49<2:51:21,  4.17s/it]
 38%|███▊      | 1536/4000 [1:47:54<2:55:36,  4.28s/it]
 38%|███▊      | 1537/4000 [1:47:58<2:54:31,  4.25s/it]
 38%|███▊      | 1538/4000 [1:48:02<2:53:15,  4.22s/it]
 38%|███▊      | 1539/4000 [1:48:06<2:52:28,  4.21s/it]
 38%|███▊      | 1540/4000 [1:48:10<2:51:50,  4.19s/it]
                                                       
{'loss': '0.4817', 'grad_norm': '0.1222', 'learning_rate': '0.0003729', 'epoch': '0.3731'}

 38%|███▊      | 1540/4000 [1:48:10<2:51:50,  4.19s/it]
 39%|███▊      | 1541/4000 [1:48:14<2:51:21,  4.18s/it]
 39%|███▊      | 1542/4000 [1:48:19<2:51:08,  4.18s/it]
 39%|███▊      | 1543/4000 [1:48:23<2:55:14,  4.28s/it]
 39%|███▊      | 1544/4000 [1:48:27<2:53:45,  4.24s/it]
 39%|███▊      | 1545/4000 [1:48:31<2:52:31,  4.22s/it]
 39%|███▊      | 1546/4000 [1:48:36<2:51:49,  4.20s/it]
 39%|███▊      | 1547/4000 [1:48:40<2:51:25,  4.19s/it]
 39%|███▊      | 1548/4000 [1:48:44<2:50:41,  4.18s/it]
 39%|███▊      | 1549/4000 [1:48:48<2:50:12,  4.17s/it]
 39%|███▉      | 1550/4000 [1:48:53<2:54:56,  4.28s/it]
                                                       
{'loss': '0.4882', 'grad_norm': '0.1181', 'learning_rate': '0.0003714', 'epoch': '0.3756'}

 39%|███▉      | 1550/4000 [1:48:53<2:54:56,  4.28s/it]
 39%|███▉      | 1551/4000 [1:48:57<2:53:26,  4.25s/it]
 39%|███▉      | 1552/4000 [1:49:01<2:52:13,  4.22s/it]
 39%|███▉      | 1553/4000 [1:49:05<2:51:08,  4.20s/it]
 39%|███▉      | 1554/4000 [1:49:09<2:50:39,  4.19s/it]
 39%|███▉      | 1555/4000 [1:49:13<2:50:09,  4.18s/it]
 39%|███▉      | 1556/4000 [1:49:18<2:54:35,  4.29s/it]
 39%|███▉      | 1557/4000 [1:49:22<2:53:04,  4.25s/it]
 39%|███▉      | 1558/4000 [1:49:26<2:51:54,  4.22s/it]
 39%|███▉      | 1559/4000 [1:49:30<2:51:07,  4.21s/it]
 39%|███▉      | 1560/4000 [1:49:35<2:50:37,  4.20s/it]
                                                       
{'loss': '0.5387', 'grad_norm': '0.1452', 'learning_rate': '0.0003698', 'epoch': '0.378'}

 39%|███▉      | 1560/4000 [1:49:35<2:50:37,  4.20s/it]
 39%|███▉      | 1561/4000 [1:49:39<2:50:13,  4.19s/it]
 39%|███▉      | 1562/4000 [1:49:43<2:49:47,  4.18s/it]
 39%|███▉      | 1563/4000 [1:49:47<2:53:03,  4.26s/it]
 39%|███▉      | 1564/4000 [1:49:52<2:51:45,  4.23s/it]
 39%|███▉      | 1565/4000 [1:49:56<2:50:53,  4.21s/it]
 39%|███▉      | 1566/4000 [1:50:00<2:50:10,  4.20s/it]
 39%|███▉      | 1567/4000 [1:50:04<2:49:34,  4.18s/it]
 39%|███▉      | 1568/4000 [1:50:08<2:49:00,  4.17s/it]
 39%|███▉      | 1569/4000 [1:50:12<2:48:35,  4.16s/it]
 39%|███▉      | 1570/4000 [1:50:17<2:53:18,  4.28s/it]
                                                       
{'loss': '0.5409', 'grad_norm': '0.146', 'learning_rate': '0.0003683', 'epoch': '0.3804'}

 39%|███▉      | 1570/4000 [1:50:17<2:53:18,  4.28s/it]
 39%|███▉      | 1571/4000 [1:50:21<2:51:44,  4.24s/it]
 39%|███▉      | 1572/4000 [1:50:25<2:50:34,  4.22s/it]
 39%|███▉      | 1573/4000 [1:50:29<2:49:43,  4.20s/it]
 39%|███▉      | 1574/4000 [1:50:33<2:49:01,  4.18s/it]
 39%|███▉      | 1575/4000 [1:50:38<2:48:29,  4.17s/it]
 39%|███▉      | 1576/4000 [1:50:42<2:48:14,  4.16s/it]
 39%|███▉      | 1577/4000 [1:50:46<2:52:17,  4.27s/it]
 39%|███▉      | 1578/4000 [1:50:50<2:50:40,  4.23s/it]
 39%|███▉      | 1579/4000 [1:50:55<2:49:38,  4.20s/it]
 40%|███▉      | 1580/4000 [1:50:59<2:49:27,  4.20s/it]
                                                       
{'loss': '0.5167', 'grad_norm': '0.1351', 'learning_rate': '0.0003668', 'epoch': '0.3828'}

 40%|███▉      | 1580/4000 [1:50:59<2:49:27,  4.20s/it]
 40%|███▉      | 1581/4000 [1:51:03<2:48:52,  4.19s/it]
 40%|███▉      | 1582/4000 [1:51:07<2:48:43,  4.19s/it]
 40%|███▉      | 1583/4000 [1:51:11<2:50:52,  4.24s/it]
 40%|███▉      | 1584/4000 [1:51:16<2:51:54,  4.27s/it]
 40%|███▉      | 1585/4000 [1:51:20<2:50:23,  4.23s/it]
 40%|███▉      | 1586/4000 [1:51:24<2:49:28,  4.21s/it]
 40%|███▉      | 1587/4000 [1:51:28<2:48:58,  4.20s/it]
 40%|███▉      | 1588/4000 [1:51:32<2:48:34,  4.19s/it]
 40%|███▉      | 1589/4000 [1:51:37<2:48:10,  4.19s/it]
 40%|███▉      | 1590/4000 [1:51:41<2:52:44,  4.30s/it]
                                                       
{'loss': '0.5061', 'grad_norm': '0.1514', 'learning_rate': '0.0003653', 'epoch': '0.3853'}

 40%|███▉      | 1590/4000 [1:51:41<2:52:44,  4.30s/it]
 40%|███▉      | 1591/4000 [1:51:45<2:51:08,  4.26s/it]
 40%|███▉      | 1592/4000 [1:51:50<2:50:03,  4.24s/it]
 40%|███▉      | 1593/4000 [1:51:54<2:49:09,  4.22s/it]
 40%|███▉      | 1594/4000 [1:51:58<2:48:26,  4.20s/it]
 40%|███▉      | 1595/4000 [1:52:02<2:47:52,  4.19s/it]
 40%|███▉      | 1596/4000 [1:52:06<2:47:36,  4.18s/it]
 40%|███▉      | 1597/4000 [1:52:11<2:51:32,  4.28s/it]
 40%|███▉      | 1598/4000 [1:52:15<2:50:16,  4.25s/it]
 40%|███▉      | 1599/4000 [1:52:19<2:49:06,  4.23s/it]
 40%|████      | 1600/4000 [1:52:23<2:48:45,  4.22s/it]
                                                       
{'loss': '0.4952', 'grad_norm': '0.1838', 'learning_rate': '0.0003638', 'epoch': '0.3877'}

 40%|████      | 1600/4000 [1:52:23<2:48:45,  4.22s/it]
 40%|████      | 1601/4000 [1:52:27<2:48:25,  4.21s/it]
 40%|████      | 1602/4000 [1:52:32<2:47:50,  4.20s/it]
 40%|████      | 1603/4000 [1:52:36<2:47:32,  4.19s/it]
 40%|████      | 1604/4000 [1:52:40<2:51:40,  4.30s/it]
 40%|████      | 1605/4000 [1:52:45<2:49:58,  4.26s/it]
 40%|████      | 1606/4000 [1:52:49<2:48:46,  4.23s/it]
 40%|████      | 1607/4000 [1:52:53<2:48:03,  4.21s/it]
 40%|████      | 1608/4000 [1:52:57<2:47:17,  4.20s/it]
 40%|████      | 1609/4000 [1:53:01<2:46:55,  4.19s/it]
 40%|████      | 1610/4000 [1:53:06<2:51:21,  4.30s/it]
                                                       
{'loss': '0.5098', 'grad_norm': '0.1327', 'learning_rate': '0.0003623', 'epoch': '0.3901'}

 40%|████      | 1610/4000 [1:53:06<2:51:21,  4.30s/it]
 40%|████      | 1611/4000 [1:53:10<2:49:40,  4.26s/it]
 40%|████      | 1612/4000 [1:53:14<2:48:28,  4.23s/it]
 40%|████      | 1613/4000 [1:53:18<2:47:42,  4.22s/it]
 40%|████      | 1614/4000 [1:53:22<2:47:12,  4.20s/it]
 40%|████      | 1615/4000 [1:53:27<2:46:36,  4.19s/it]
 40%|████      | 1616/4000 [1:53:31<2:46:16,  4.18s/it]
 40%|████      | 1617/4000 [1:53:35<2:50:41,  4.30s/it]
 40%|████      | 1618/4000 [1:53:39<2:49:09,  4.26s/it]
 40%|████      | 1619/4000 [1:53:44<2:47:56,  4.23s/it]
 40%|████      | 1620/4000 [1:53:48<2:47:01,  4.21s/it]
                                                       
{'loss': '0.4661', 'grad_norm': '0.1156', 'learning_rate': '0.0003608', 'epoch': '0.3925'}

 40%|████      | 1620/4000 [1:53:48<2:47:01,  4.21s/it]
 41%|████      | 1621/4000 [1:53:52<2:46:22,  4.20s/it]
 41%|████      | 1622/4000 [1:53:56<2:46:04,  4.19s/it]
 41%|████      | 1623/4000 [1:54:00<2:45:44,  4.18s/it]
 41%|████      | 1624/4000 [1:54:05<2:49:56,  4.29s/it]
 41%|████      | 1625/4000 [1:54:09<2:48:28,  4.26s/it]
 41%|████      | 1626/4000 [1:54:13<2:47:26,  4.23s/it]
 41%|████      | 1627/4000 [1:54:17<2:46:54,  4.22s/it]
 41%|████      | 1628/4000 [1:54:22<2:46:15,  4.21s/it]
 41%|████      | 1629/4000 [1:54:26<2:45:42,  4.19s/it]
 41%|████      | 1630/4000 [1:54:30<2:47:24,  4.24s/it]
                                                       
{'loss': '0.4574', 'grad_norm': '0.1199', 'learning_rate': '0.0003592', 'epoch': '0.3949'}

 41%|████      | 1630/4000 [1:54:30<2:47:24,  4.24s/it]
 41%|████      | 1631/4000 [1:54:34<2:48:32,  4.27s/it]
 41%|████      | 1632/4000 [1:54:39<2:47:20,  4.24s/it]
 41%|████      | 1633/4000 [1:54:43<2:46:25,  4.22s/it]
 41%|████      | 1634/4000 [1:54:47<2:45:50,  4.21s/it]
 41%|████      | 1635/4000 [1:54:51<2:45:24,  4.20s/it]
 41%|████      | 1636/4000 [1:54:55<2:44:59,  4.19s/it]
 41%|████      | 1637/4000 [1:55:00<2:49:52,  4.31s/it]
 41%|████      | 1638/4000 [1:55:04<2:48:14,  4.27s/it]
 41%|████      | 1639/4000 [1:55:08<2:47:02,  4.25s/it]
 41%|████      | 1640/4000 [1:55:12<2:46:20,  4.23s/it]
                                                       
{'loss': '0.4998', 'grad_norm': '0.2282', 'learning_rate': '0.0003577', 'epoch': '0.3974'}

 41%|████      | 1640/4000 [1:55:12<2:46:20,  4.23s/it]
 41%|████      | 1641/4000 [1:55:17<2:45:41,  4.21s/it]
 41%|████      | 1642/4000 [1:55:21<2:45:08,  4.20s/it]
 41%|████      | 1643/4000 [1:55:25<2:44:36,  4.19s/it]
 41%|████      | 1644/4000 [1:55:29<2:48:21,  4.29s/it]
 41%|████      | 1645/4000 [1:55:34<2:47:04,  4.26s/it]
 41%|████      | 1646/4000 [1:55:38<2:45:59,  4.23s/it]
 41%|████      | 1647/4000 [1:55:42<2:45:12,  4.21s/it]
 41%|████      | 1648/4000 [1:55:46<2:44:26,  4.19s/it]
 41%|████      | 1649/4000 [1:55:50<2:43:55,  4.18s/it]
 41%|████▏     | 1650/4000 [1:55:54<2:43:36,  4.18s/it]
                                                       
{'loss': '0.5091', 'grad_norm': '0.1964', 'learning_rate': '0.0003562', 'epoch': '0.3998'}

 41%|████▏     | 1650/4000 [1:55:54<2:43:36,  4.18s/it]
 41%|████▏     | 1651/4000 [1:55:59<2:47:47,  4.29s/it]
 41%|████▏     | 1652/4000 [1:56:03<2:46:11,  4.25s/it]
 41%|████▏     | 1653/4000 [1:56:07<2:45:02,  4.22s/it]
 41%|████▏     | 1654/4000 [1:56:11<2:44:23,  4.20s/it]
 41%|████▏     | 1655/4000 [1:56:16<2:44:10,  4.20s/it]
 41%|████▏     | 1656/4000 [1:56:20<2:44:14,  4.20s/it]
 41%|████▏     | 1657/4000 [1:56:24<2:46:26,  4.26s/it]
 41%|████▏     | 1658/4000 [1:56:29<2:47:59,  4.30s/it]
 41%|████▏     | 1659/4000 [1:56:33<2:46:20,  4.26s/it]
 42%|████▏     | 1660/4000 [1:56:37<2:45:39,  4.25s/it]
                                                       
{'loss': '0.483', 'grad_norm': '0.1286', 'learning_rate': '0.0003547', 'epoch': '0.4022'}

 42%|████▏     | 1660/4000 [1:56:37<2:45:39,  4.25s/it]
 42%|████▏     | 1661/4000 [1:56:41<2:44:39,  4.22s/it]
 42%|████▏     | 1662/4000 [1:56:45<2:44:12,  4.21s/it]
 42%|████▏     | 1663/4000 [1:56:50<2:43:37,  4.20s/it]
 42%|████▏     | 1664/4000 [1:56:54<2:47:16,  4.30s/it]
 42%|████▏     | 1665/4000 [1:56:58<2:45:41,  4.26s/it]
 42%|████▏     | 1666/4000 [1:57:02<2:44:28,  4.23s/it]
 42%|████▏     | 1667/4000 [1:57:07<2:43:47,  4.21s/it]
 42%|████▏     | 1668/4000 [1:57:11<2:43:24,  4.20s/it]
 42%|████▏     | 1669/4000 [1:57:15<2:43:14,  4.20s/it]
 42%|████▏     | 1670/4000 [1:57:19<2:43:14,  4.20s/it]
                                                       
{'loss': '0.4756', 'grad_norm': '0.1426', 'learning_rate': '0.0003532', 'epoch': '0.4046'}

 42%|████▏     | 1670/4000 [1:57:19<2:43:14,  4.20s/it]
 42%|████▏     | 1671/4000 [1:57:24<2:47:13,  4.31s/it]
 42%|████▏     | 1672/4000 [1:57:28<2:45:51,  4.27s/it]
 42%|████▏     | 1673/4000 [1:57:32<2:44:47,  4.25s/it]
 42%|████▏     | 1674/4000 [1:57:36<2:44:10,  4.23s/it]
 42%|████▏     | 1675/4000 [1:57:41<2:43:14,  4.21s/it]
 42%|████▏     | 1676/4000 [1:57:45<2:42:46,  4.20s/it]
 42%|████▏     | 1677/4000 [1:57:49<2:42:16,  4.19s/it]
 42%|████▏     | 1678/4000 [1:57:53<2:46:19,  4.30s/it]
 42%|████▏     | 1679/4000 [1:57:58<2:44:33,  4.25s/it]
 42%|████▏     | 1680/4000 [1:58:02<2:43:17,  4.22s/it]
                                                       
{'loss': '0.5059', 'grad_norm': '0.1299', 'learning_rate': '0.0003517', 'epoch': '0.4071'}

 42%|████▏     | 1680/4000 [1:58:02<2:43:17,  4.22s/it]
 42%|████▏     | 1681/4000 [1:58:06<2:42:37,  4.21s/it]
 42%|████▏     | 1682/4000 [1:58:10<2:42:04,  4.20s/it]
 42%|████▏     | 1683/4000 [1:58:14<2:41:25,  4.18s/it]
 42%|████▏     | 1684/4000 [1:58:19<2:43:05,  4.23s/it]
 42%|████▏     | 1685/4000 [1:58:23<2:44:37,  4.27s/it]
 42%|████▏     | 1686/4000 [1:58:27<2:43:12,  4.23s/it]
 42%|████▏     | 1687/4000 [1:58:31<2:42:22,  4.21s/it]
 42%|████▏     | 1688/4000 [1:58:35<2:41:53,  4.20s/it]
 42%|████▏     | 1689/4000 [1:58:40<2:41:32,  4.19s/it]
 42%|████▏     | 1690/4000 [1:58:44<2:41:12,  4.19s/it]
                                                       
{'loss': '0.5031', 'grad_norm': '0.1176', 'learning_rate': '0.0003502', 'epoch': '0.4095'}

 42%|████▏     | 1690/4000 [1:58:44<2:41:12,  4.19s/it]
 42%|████▏     | 1691/4000 [1:58:48<2:45:12,  4.29s/it]
 42%|████▏     | 1692/4000 [1:58:52<2:43:40,  4.25s/it]
 42%|████▏     | 1693/4000 [1:58:57<2:42:20,  4.22s/it]
 42%|████▏     | 1694/4000 [1:59:01<2:41:36,  4.21s/it]
 42%|████▏     | 1695/4000 [1:59:05<2:41:00,  4.19s/it]
 42%|████▏     | 1696/4000 [1:59:09<2:40:41,  4.18s/it]
 42%|████▏     | 1697/4000 [1:59:13<2:40:38,  4.19s/it]
 42%|████▏     | 1698/4000 [1:59:18<2:44:54,  4.30s/it]
 42%|████▏     | 1699/4000 [1:59:22<2:43:25,  4.26s/it]
 42%|████▎     | 1700/4000 [1:59:26<2:42:22,  4.24s/it]
                                                       
{'loss': '0.4895', 'grad_norm': '0.1596', 'learning_rate': '0.0003486', 'epoch': '0.4119'}

 42%|████▎     | 1700/4000 [1:59:26<2:42:22,  4.24s/it]
 43%|████▎     | 1701/4000 [1:59:30<2:41:51,  4.22s/it]
 43%|████▎     | 1702/4000 [1:59:35<2:41:48,  4.22s/it]
 43%|████▎     | 1703/4000 [1:59:39<2:41:19,  4.21s/it]
 43%|████▎     | 1704/4000 [1:59:43<2:40:46,  4.20s/it]
 43%|████▎     | 1705/4000 [1:59:47<2:44:36,  4.30s/it]
 43%|████▎     | 1706/4000 [1:59:52<2:43:00,  4.26s/it]
 43%|████▎     | 1707/4000 [1:59:56<2:42:04,  4.24s/it]
 43%|████▎     | 1708/4000 [2:00:00<2:41:44,  4.23s/it]
 43%|████▎     | 1709/4000 [2:00:04<2:40:57,  4.22s/it]
 43%|████▎     | 1710/4000 [2:00:08<2:40:21,  4.20s/it]
                                                       
{'loss': '0.5079', 'grad_norm': '0.122', 'learning_rate': '0.0003471', 'epoch': '0.4143'}

 43%|████▎     | 1710/4000 [2:00:08<2:40:21,  4.20s/it]
 43%|████▎     | 1711/4000 [2:00:13<2:43:37,  4.29s/it]
 43%|████▎     | 1712/4000 [2:00:17<2:42:24,  4.26s/it]
 43%|████▎     | 1713/4000 [2:00:21<2:41:18,  4.23s/it]
 43%|████▎     | 1714/4000 [2:00:25<2:40:47,  4.22s/it]
 43%|████▎     | 1715/4000 [2:00:30<2:40:00,  4.20s/it]
 43%|████▎     | 1716/4000 [2:00:34<2:39:21,  4.19s/it]
 43%|████▎     | 1717/4000 [2:00:38<2:39:27,  4.19s/it]
 43%|████▎     | 1718/4000 [2:00:43<2:43:37,  4.30s/it]
 43%|████▎     | 1719/4000 [2:00:47<2:41:57,  4.26s/it]
 43%|████▎     | 1720/4000 [2:00:51<2:40:46,  4.23s/it]
                                                       
{'loss': '0.5074', 'grad_norm': '0.1177', 'learning_rate': '0.0003456', 'epoch': '0.4168'}

 43%|████▎     | 1720/4000 [2:00:51<2:40:46,  4.23s/it]
 43%|████▎     | 1721/4000 [2:00:55<2:40:08,  4.22s/it]
 43%|████▎     | 1722/4000 [2:00:59<2:39:50,  4.21s/it]
 43%|████▎     | 1723/4000 [2:01:03<2:39:39,  4.21s/it]
 43%|████▎     | 1724/4000 [2:01:08<2:39:32,  4.21s/it]
 43%|████▎     | 1725/4000 [2:01:12<2:43:34,  4.31s/it]
 43%|████▎     | 1726/4000 [2:01:16<2:41:50,  4.27s/it]
 43%|████▎     | 1727/4000 [2:01:21<2:40:29,  4.24s/it]
 43%|████▎     | 1728/4000 [2:01:25<2:39:27,  4.21s/it]
 43%|████▎     | 1729/4000 [2:01:29<2:38:48,  4.20s/it]
 43%|████▎     | 1730/4000 [2:01:33<2:38:21,  4.19s/it]
                                                       
{'loss': '0.5266', 'grad_norm': '0.1315', 'learning_rate': '0.0003441', 'epoch': '0.4192'}

 43%|████▎     | 1730/4000 [2:01:33<2:38:21,  4.19s/it]
 43%|████▎     | 1731/4000 [2:01:37<2:38:00,  4.18s/it]
 43%|████▎     | 1732/4000 [2:01:42<2:41:57,  4.28s/it]
 43%|████▎     | 1733/4000 [2:01:46<2:40:43,  4.25s/it]
 43%|████▎     | 1734/4000 [2:01:50<2:39:53,  4.23s/it]
 43%|████▎     | 1735/4000 [2:01:54<2:39:15,  4.22s/it]
 43%|████▎     | 1736/4000 [2:01:58<2:38:55,  4.21s/it]
 43%|████▎     | 1737/4000 [2:02:03<2:38:15,  4.20s/it]
 43%|████▎     | 1738/4000 [2:02:07<2:42:17,  4.31s/it]
 43%|████▎     | 1739/4000 [2:02:11<2:40:44,  4.27s/it]
 44%|████▎     | 1740/4000 [2:02:16<2:39:53,  4.24s/it]
                                                       
{'loss': '0.5235', 'grad_norm': '0.1266', 'learning_rate': '0.0003426', 'epoch': '0.4216'}

 44%|████▎     | 1740/4000 [2:02:16<2:39:53,  4.24s/it]
 44%|████▎     | 1741/4000 [2:02:20<2:38:56,  4.22s/it]
 44%|████▎     | 1742/4000 [2:02:24<2:38:27,  4.21s/it]
 44%|████▎     | 1743/4000 [2:02:28<2:37:55,  4.20s/it]
 44%|████▎     | 1744/4000 [2:02:32<2:37:33,  4.19s/it]
 44%|████▎     | 1745/4000 [2:02:37<2:41:18,  4.29s/it]
 44%|████▎     | 1746/4000 [2:02:41<2:39:51,  4.26s/it]
 44%|████▎     | 1747/4000 [2:02:45<2:38:47,  4.23s/it]
 44%|████▎     | 1748/4000 [2:02:49<2:38:06,  4.21s/it]
 44%|████▎     | 1749/4000 [2:02:53<2:37:37,  4.20s/it]
 44%|████▍     | 1750/4000 [2:02:58<2:37:12,  4.19s/it]
                                                       
{'loss': '0.489', 'grad_norm': '0.1482', 'learning_rate': '0.0003411', 'epoch': '0.424'}

 44%|████▍     | 1750/4000 [2:02:58<2:37:12,  4.19s/it]
 44%|████▍     | 1751/4000 [2:03:02<2:36:47,  4.18s/it]
 44%|████▍     | 1752/4000 [2:03:06<2:40:49,  4.29s/it]
 44%|████▍     | 1753/4000 [2:03:10<2:39:27,  4.26s/it]
 44%|████▍     | 1754/4000 [2:03:15<2:38:25,  4.23s/it]
 44%|████▍     | 1755/4000 [2:03:19<2:37:37,  4.21s/it]
 44%|████▍     | 1756/4000 [2:03:23<2:37:22,  4.21s/it]
 44%|████▍     | 1757/4000 [2:03:27<2:36:51,  4.20s/it]
 44%|████▍     | 1758/4000 [2:03:32<2:38:31,  4.24s/it]
 44%|████▍     | 1759/4000 [2:03:36<2:40:17,  4.29s/it]
 44%|████▍     | 1760/4000 [2:03:40<2:39:01,  4.26s/it]
                                                       
{'loss': '0.529', 'grad_norm': '0.1371', 'learning_rate': '0.0003395', 'epoch': '0.4264'}

 44%|████▍     | 1760/4000 [2:03:40<2:39:01,  4.26s/it]
 44%|████▍     | 1761/4000 [2:03:44<2:38:16,  4.24s/it]
 44%|████▍     | 1762/4000 [2:03:48<2:37:08,  4.21s/it]
 44%|████▍     | 1763/4000 [2:03:53<2:36:45,  4.20s/it]
 44%|████▍     | 1764/4000 [2:03:57<2:36:14,  4.19s/it]
 44%|████▍     | 1765/4000 [2:04:01<2:39:59,  4.29s/it]
 44%|████▍     | 1766/4000 [2:04:06<2:38:28,  4.26s/it]
 44%|████▍     | 1767/4000 [2:04:10<2:37:19,  4.23s/it]
 44%|████▍     | 1768/4000 [2:04:14<2:36:47,  4.21s/it]
 44%|████▍     | 1769/4000 [2:04:18<2:36:14,  4.20s/it]
 44%|████▍     | 1770/4000 [2:04:22<2:35:43,  4.19s/it]
                                                       
{'loss': '0.4912', 'grad_norm': '0.1379', 'learning_rate': '0.000338', 'epoch': '0.4289'}

 44%|████▍     | 1770/4000 [2:04:22<2:35:43,  4.19s/it]
 44%|████▍     | 1771/4000 [2:04:26<2:35:28,  4.19s/it]
 44%|████▍     | 1772/4000 [2:04:31<2:39:20,  4.29s/it]
 44%|████▍     | 1773/4000 [2:04:35<2:37:50,  4.25s/it]
 44%|████▍     | 1774/4000 [2:04:39<2:36:54,  4.23s/it]
 44%|████▍     | 1775/4000 [2:04:43<2:36:04,  4.21s/it]
 44%|████▍     | 1776/4000 [2:04:48<2:35:31,  4.20s/it]
 44%|████▍     | 1777/4000 [2:04:52<2:35:03,  4.19s/it]
 44%|████▍     | 1778/4000 [2:04:56<2:34:51,  4.18s/it]
 44%|████▍     | 1779/4000 [2:05:00<2:38:48,  4.29s/it]
 44%|████▍     | 1780/4000 [2:05:05<2:37:28,  4.26s/it]
                                                       
{'loss': '0.4496', 'grad_norm': '0.1109', 'learning_rate': '0.0003365', 'epoch': '0.4313'}

 44%|████▍     | 1780/4000 [2:05:05<2:37:28,  4.26s/it]
 45%|████▍     | 1781/4000 [2:05:09<2:36:28,  4.23s/it]
 45%|████▍     | 1782/4000 [2:05:13<2:35:42,  4.21s/it]
 45%|████▍     | 1783/4000 [2:05:17<2:35:08,  4.20s/it]
 45%|████▍     | 1784/4000 [2:05:21<2:34:35,  4.19s/it]
 45%|████▍     | 1785/4000 [2:05:26<2:36:35,  4.24s/it]
 45%|████▍     | 1786/4000 [2:05:30<2:37:37,  4.27s/it]
 45%|████▍     | 1787/4000 [2:05:34<2:36:31,  4.24s/it]
 45%|████▍     | 1788/4000 [2:05:38<2:35:36,  4.22s/it]
 45%|████▍     | 1789/4000 [2:05:43<2:34:55,  4.20s/it]
 45%|████▍     | 1790/4000 [2:05:47<2:34:27,  4.19s/it]
                                                       
{'loss': '0.5288', 'grad_norm': '0.1151', 'learning_rate': '0.000335', 'epoch': '0.4337'}

 45%|████▍     | 1790/4000 [2:05:47<2:34:27,  4.19s/it]
 45%|████▍     | 1791/4000 [2:05:51<2:34:11,  4.19s/it]
 45%|████▍     | 1792/4000 [2:05:55<2:37:52,  4.29s/it]
 45%|████▍     | 1793/4000 [2:06:00<2:36:10,  4.25s/it]
 45%|████▍     | 1794/4000 [2:06:04<2:35:14,  4.22s/it]
 45%|████▍     | 1795/4000 [2:06:08<2:34:30,  4.20s/it]
 45%|████▍     | 1796/4000 [2:06:12<2:34:07,  4.20s/it]
 45%|████▍     | 1797/4000 [2:06:16<2:33:37,  4.18s/it]
 45%|████▍     | 1798/4000 [2:06:20<2:33:11,  4.17s/it]
 45%|████▍     | 1799/4000 [2:06:25<2:36:48,  4.27s/it]
 45%|████▌     | 1800/4000 [2:06:29<2:35:40,  4.25s/it]
                                                       
{'loss': '0.4617', 'grad_norm': '0.1257', 'learning_rate': '0.0003335', 'epoch': '0.4361'}

 45%|████▌     | 1800/4000 [2:06:29<2:35:40,  4.25s/it]
 45%|████▌     | 1801/4000 [2:06:33<2:34:39,  4.22s/it]
 45%|████▌     | 1802/4000 [2:06:37<2:33:57,  4.20s/it]
 45%|████▌     | 1803/4000 [2:06:42<2:33:33,  4.19s/it]
 45%|████▌     | 1804/4000 [2:06:46<2:33:06,  4.18s/it]
 45%|████▌     | 1805/4000 [2:06:50<2:32:48,  4.18s/it]
 45%|████▌     | 1806/4000 [2:06:54<2:36:57,  4.29s/it]
 45%|████▌     | 1807/4000 [2:06:59<2:35:36,  4.26s/it]
 45%|████▌     | 1808/4000 [2:07:03<2:34:28,  4.23s/it]
 45%|████▌     | 1809/4000 [2:07:07<2:33:44,  4.21s/it]
 45%|████▌     | 1810/4000 [2:07:11<2:33:10,  4.20s/it]
                                                       
{'loss': '0.4481', 'grad_norm': '0.1269', 'learning_rate': '0.000332', 'epoch': '0.4386'}

 45%|████▌     | 1810/4000 [2:07:11<2:33:10,  4.20s/it]
 45%|████▌     | 1811/4000 [2:07:15<2:32:58,  4.19s/it]
 45%|████▌     | 1812/4000 [2:07:20<2:35:27,  4.26s/it]
 45%|████▌     | 1813/4000 [2:07:24<2:37:04,  4.31s/it]
 45%|████▌     | 1814/4000 [2:07:28<2:35:20,  4.26s/it]
 45%|████▌     | 1815/4000 [2:07:32<2:34:13,  4.23s/it]
 45%|████▌     | 1816/4000 [2:07:37<2:33:32,  4.22s/it]
 45%|████▌     | 1817/4000 [2:07:41<2:33:09,  4.21s/it]
 45%|████▌     | 1818/4000 [2:07:45<2:32:50,  4.20s/it]
 45%|████▌     | 1819/4000 [2:07:50<2:36:24,  4.30s/it]
 46%|████▌     | 1820/4000 [2:07:54<2:34:54,  4.26s/it]
                                                       
{'loss': '0.5216', 'grad_norm': '0.2422', 'learning_rate': '0.0003305', 'epoch': '0.441'}

 46%|████▌     | 1820/4000 [2:07:54<2:34:54,  4.26s/it]
 46%|████▌     | 1821/4000 [2:07:58<2:33:55,  4.24s/it]
 46%|████▌     | 1822/4000 [2:08:02<2:33:05,  4.22s/it]
 46%|████▌     | 1823/4000 [2:08:06<2:32:44,  4.21s/it]
 46%|████▌     | 1824/4000 [2:08:10<2:32:19,  4.20s/it]
 46%|████▌     | 1825/4000 [2:08:15<2:31:57,  4.19s/it]
 46%|████▌     | 1826/4000 [2:08:19<2:35:52,  4.30s/it]
 46%|████▌     | 1827/4000 [2:08:23<2:34:10,  4.26s/it]
 46%|████▌     | 1828/4000 [2:08:27<2:33:01,  4.23s/it]
 46%|████▌     | 1829/4000 [2:08:32<2:32:10,  4.21s/it]
 46%|████▌     | 1830/4000 [2:08:36<2:31:34,  4.19s/it]
                                                       
{'loss': '0.4807', 'grad_norm': '0.1571', 'learning_rate': '0.0003289', 'epoch': '0.4434'}

 46%|████▌     | 1830/4000 [2:08:36<2:31:34,  4.19s/it]
 46%|████▌     | 1831/4000 [2:08:40<2:31:13,  4.18s/it]
 46%|████▌     | 1832/4000 [2:08:44<2:30:57,  4.18s/it]
 46%|████▌     | 1833/4000 [2:08:49<2:34:34,  4.28s/it]
 46%|████▌     | 1834/4000 [2:08:53<2:33:13,  4.24s/it]
 46%|████▌     | 1835/4000 [2:08:57<2:32:19,  4.22s/it]
 46%|████▌     | 1836/4000 [2:09:01<2:31:43,  4.21s/it]
 46%|████▌     | 1837/4000 [2:09:05<2:31:09,  4.19s/it]
 46%|████▌     | 1838/4000 [2:09:09<2:30:36,  4.18s/it]
 46%|████▌     | 1839/4000 [2:09:14<2:34:00,  4.28s/it]
 46%|████▌     | 1840/4000 [2:09:18<2:32:30,  4.24s/it]
                                                       
{'loss': '0.4902', 'grad_norm': '0.1436', 'learning_rate': '0.0003274', 'epoch': '0.4458'}

 46%|████▌     | 1840/4000 [2:09:18<2:32:30,  4.24s/it]
 46%|████▌     | 1841/4000 [2:09:22<2:31:22,  4.21s/it]
 46%|████▌     | 1842/4000 [2:09:26<2:30:43,  4.19s/it]
 46%|████▌     | 1843/4000 [2:09:31<2:30:12,  4.18s/it]
 46%|████▌     | 1844/4000 [2:09:35<2:29:56,  4.17s/it]
 46%|████▌     | 1845/4000 [2:09:39<2:29:36,  4.17s/it]
 46%|████▌     | 1846/4000 [2:09:43<2:33:11,  4.27s/it]
 46%|████▌     | 1847/4000 [2:09:47<2:31:45,  4.23s/it]
 46%|████▌     | 1848/4000 [2:09:52<2:30:51,  4.21s/it]
 46%|████▌     | 1849/4000 [2:09:56<2:30:29,  4.20s/it]
 46%|████▋     | 1850/4000 [2:10:00<2:30:05,  4.19s/it]
                                                       
{'loss': '0.4978', 'grad_norm': '0.1289', 'learning_rate': '0.0003259', 'epoch': '0.4483'}

 46%|████▋     | 1850/4000 [2:10:00<2:30:05,  4.19s/it]
 46%|████▋     | 1851/4000 [2:10:04<2:29:37,  4.18s/it]
 46%|████▋     | 1852/4000 [2:10:08<2:29:21,  4.17s/it]
 46%|████▋     | 1853/4000 [2:10:13<2:33:21,  4.29s/it]
 46%|████▋     | 1854/4000 [2:10:17<2:32:15,  4.26s/it]
 46%|████▋     | 1855/4000 [2:10:21<2:31:19,  4.23s/it]
 46%|████▋     | 1856/4000 [2:10:25<2:30:43,  4.22s/it]
 46%|████▋     | 1857/4000 [2:10:30<2:30:04,  4.20s/it]
 46%|████▋     | 1858/4000 [2:10:34<2:29:42,  4.19s/it]
 46%|████▋     | 1859/4000 [2:10:38<2:29:22,  4.19s/it]
 46%|████▋     | 1860/4000 [2:10:42<2:33:13,  4.30s/it]
                                                       
{'loss': '0.4719', 'grad_norm': '0.1304', 'learning_rate': '0.0003244', 'epoch': '0.4507'}

 46%|████▋     | 1860/4000 [2:10:42<2:33:13,  4.30s/it]
 47%|████▋     | 1861/4000 [2:10:47<2:31:36,  4.25s/it]
 47%|████▋     | 1862/4000 [2:10:51<2:30:27,  4.22s/it]
 47%|████▋     | 1863/4000 [2:10:55<2:29:31,  4.20s/it]
 47%|████▋     | 1864/4000 [2:10:59<2:29:39,  4.20s/it]
 47%|████▋     | 1865/4000 [2:11:03<2:29:29,  4.20s/it]
 47%|████▋     | 1866/4000 [2:11:08<2:32:51,  4.30s/it]
 47%|████▋     | 1867/4000 [2:11:12<2:31:21,  4.26s/it]
 47%|████▋     | 1868/4000 [2:11:16<2:30:14,  4.23s/it]
 47%|████▋     | 1869/4000 [2:11:20<2:29:24,  4.21s/it]
 47%|████▋     | 1870/4000 [2:11:24<2:28:58,  4.20s/it]
                                                       
{'loss': '0.4726', 'grad_norm': '0.1169', 'learning_rate': '0.0003229', 'epoch': '0.4531'}

 47%|████▋     | 1870/4000 [2:11:24<2:28:58,  4.20s/it]
 47%|████▋     | 1871/4000 [2:11:29<2:28:34,  4.19s/it]
 47%|████▋     | 1872/4000 [2:11:33<2:28:10,  4.18s/it]
 47%|████▋     | 1873/4000 [2:11:37<2:32:10,  4.29s/it]
 47%|████▋     | 1874/4000 [2:11:42<2:30:55,  4.26s/it]
 47%|████▋     | 1875/4000 [2:11:46<2:30:06,  4.24s/it]
 47%|████▋     | 1876/4000 [2:11:50<2:29:40,  4.23s/it]
 47%|████▋     | 1877/4000 [2:11:54<2:28:59,  4.21s/it]
 47%|████▋     | 1878/4000 [2:11:58<2:28:31,  4.20s/it]
 47%|████▋     | 1879/4000 [2:12:02<2:28:00,  4.19s/it]
 47%|████▋     | 1880/4000 [2:12:07<2:31:18,  4.28s/it]
                                                       
{'loss': '0.487', 'grad_norm': '0.1263', 'learning_rate': '0.0003214', 'epoch': '0.4555'}

 47%|████▋     | 1880/4000 [2:12:07<2:31:18,  4.28s/it]
 47%|████▋     | 1881/4000 [2:12:11<2:29:54,  4.24s/it]
 47%|████▋     | 1882/4000 [2:12:15<2:28:57,  4.22s/it]
 47%|████▋     | 1883/4000 [2:12:19<2:28:24,  4.21s/it]
 47%|████▋     | 1884/4000 [2:12:24<2:27:59,  4.20s/it]
 47%|████▋     | 1885/4000 [2:12:28<2:27:46,  4.19s/it]
 47%|████▋     | 1886/4000 [2:12:32<2:29:19,  4.24s/it]
 47%|████▋     | 1887/4000 [2:12:37<2:30:36,  4.28s/it]
 47%|████▋     | 1888/4000 [2:12:41<2:29:18,  4.24s/it]
 47%|████▋     | 1889/4000 [2:12:45<2:28:20,  4.22s/it]
 47%|████▋     | 1890/4000 [2:12:49<2:27:50,  4.20s/it]
                                                       
{'loss': '0.455', 'grad_norm': '0.134', 'learning_rate': '0.0003198', 'epoch': '0.4579'}

 47%|████▋     | 1890/4000 [2:12:49<2:27:50,  4.20s/it]
 47%|████▋     | 1891/4000 [2:12:53<2:27:27,  4.19s/it]
 47%|████▋     | 1892/4000 [2:12:57<2:27:14,  4.19s/it]
 47%|████▋     | 1893/4000 [2:13:02<2:30:55,  4.30s/it]
 47%|████▋     | 1894/4000 [2:13:06<2:29:18,  4.25s/it]
 47%|████▋     | 1895/4000 [2:13:10<2:28:03,  4.22s/it]
 47%|████▋     | 1896/4000 [2:13:14<2:27:11,  4.20s/it]
 47%|████▋     | 1897/4000 [2:13:19<2:26:42,  4.19s/it]
 47%|████▋     | 1898/4000 [2:13:23<2:26:12,  4.17s/it]
 47%|████▋     | 1899/4000 [2:13:27<2:25:52,  4.17s/it]
 48%|████▊     | 1900/4000 [2:13:31<2:29:31,  4.27s/it]
                                                       
{'loss': '0.4551', 'grad_norm': '0.119', 'learning_rate': '0.0003183', 'epoch': '0.4604'}

 48%|████▊     | 1900/4000 [2:13:31<2:29:31,  4.27s/it]
 48%|████▊     | 1901/4000 [2:13:35<2:28:23,  4.24s/it]
 48%|████▊     | 1902/4000 [2:13:40<2:27:21,  4.21s/it]
 48%|████▊     | 1903/4000 [2:13:44<2:26:29,  4.19s/it]
 48%|████▊     | 1904/4000 [2:13:48<2:25:58,  4.18s/it]
 48%|████▊     | 1905/4000 [2:13:52<2:25:38,  4.17s/it]
 48%|████▊     | 1906/4000 [2:13:56<2:25:11,  4.16s/it]
 48%|████▊     | 1907/4000 [2:14:01<2:29:11,  4.28s/it]
 48%|████▊     | 1908/4000 [2:14:05<2:27:59,  4.24s/it]
 48%|████▊     | 1909/4000 [2:14:09<2:26:58,  4.22s/it]
 48%|████▊     | 1910/4000 [2:14:13<2:26:10,  4.20s/it]
                                                       
{'loss': '0.4717', 'grad_norm': '0.1591', 'learning_rate': '0.0003168', 'epoch': '0.4628'}

 48%|████▊     | 1910/4000 [2:14:13<2:26:10,  4.20s/it]
 48%|████▊     | 1911/4000 [2:14:17<2:25:39,  4.18s/it]
 48%|████▊     | 1912/4000 [2:14:22<2:25:19,  4.18s/it]
 48%|████▊     | 1913/4000 [2:14:26<2:27:07,  4.23s/it]
 48%|████▊     | 1914/4000 [2:14:30<2:27:59,  4.26s/it]
 48%|████▊     | 1915/4000 [2:14:34<2:26:57,  4.23s/it]
 48%|████▊     | 1916/4000 [2:14:39<2:26:15,  4.21s/it]
 48%|████▊     | 1917/4000 [2:14:43<2:25:38,  4.20s/it]
 48%|████▊     | 1918/4000 [2:14:47<2:25:20,  4.19s/it]
 48%|████▊     | 1919/4000 [2:14:51<2:24:57,  4.18s/it]
 48%|████▊     | 1920/4000 [2:14:56<2:28:28,  4.28s/it]
                                                       
{'loss': '0.4751', 'grad_norm': '0.163', 'learning_rate': '0.0003153', 'epoch': '0.4652'}

 48%|████▊     | 1920/4000 [2:14:56<2:28:28,  4.28s/it]
 48%|████▊     | 1921/4000 [2:15:00<2:27:08,  4.25s/it]
 48%|████▊     | 1922/4000 [2:15:04<2:26:15,  4.22s/it]
 48%|████▊     | 1923/4000 [2:15:08<2:25:35,  4.21s/it]
 48%|████▊     | 1924/4000 [2:15:12<2:25:04,  4.19s/it]
 48%|████▊     | 1925/4000 [2:15:16<2:24:48,  4.19s/it]
 48%|████▊     | 1926/4000 [2:15:21<2:24:35,  4.18s/it]
 48%|████▊     | 1927/4000 [2:15:25<2:28:08,  4.29s/it]
 48%|████▊     | 1928/4000 [2:15:29<2:27:01,  4.26s/it]
 48%|████▊     | 1929/4000 [2:15:33<2:26:10,  4.24s/it]
 48%|████▊     | 1930/4000 [2:15:38<2:25:20,  4.21s/it]
                                                       
{'loss': '0.4428', 'grad_norm': '0.1237', 'learning_rate': '0.0003138', 'epoch': '0.4676'}

 48%|████▊     | 1930/4000 [2:15:38<2:25:20,  4.21s/it]
 48%|████▊     | 1931/4000 [2:15:42<2:24:51,  4.20s/it]
 48%|████▊     | 1932/4000 [2:15:46<2:24:35,  4.20s/it]
 48%|████▊     | 1933/4000 [2:15:50<2:24:15,  4.19s/it]
 48%|████▊     | 1934/4000 [2:15:55<2:27:44,  4.29s/it]
 48%|████▊     | 1935/4000 [2:15:59<2:26:21,  4.25s/it]
 48%|████▊     | 1936/4000 [2:16:03<2:25:27,  4.23s/it]
 48%|████▊     | 1937/4000 [2:16:07<2:24:44,  4.21s/it]
 48%|████▊     | 1938/4000 [2:16:11<2:24:14,  4.20s/it]
 48%|████▊     | 1939/4000 [2:16:16<2:23:58,  4.19s/it]
 48%|████▊     | 1940/4000 [2:16:20<2:26:01,  4.25s/it]
                                                       
{'loss': '0.4963', 'grad_norm': '0.1213', 'learning_rate': '0.0003123', 'epoch': '0.4701'}

 48%|████▊     | 1940/4000 [2:16:20<2:26:01,  4.25s/it]
 49%|████▊     | 1941/4000 [2:16:24<2:26:57,  4.28s/it]
 49%|████▊     | 1942/4000 [2:16:28<2:25:40,  4.25s/it]
 49%|████▊     | 1943/4000 [2:16:33<2:24:55,  4.23s/it]
 49%|████▊     | 1944/4000 [2:16:37<2:24:06,  4.21s/it]
 49%|████▊     | 1945/4000 [2:16:41<2:23:32,  4.19s/it]
 49%|████▊     | 1946/4000 [2:16:45<2:23:04,  4.18s/it]
 49%|████▊     | 1947/4000 [2:16:50<2:26:05,  4.27s/it]
 49%|████▊     | 1948/4000 [2:16:54<2:24:46,  4.23s/it]
 49%|████▊     | 1949/4000 [2:16:58<2:23:48,  4.21s/it]
 49%|████▉     | 1950/4000 [2:17:02<2:23:19,  4.19s/it]
                                                       
{'loss': '0.4976', 'grad_norm': '0.2503', 'learning_rate': '0.0003108', 'epoch': '0.4725'}

 49%|████▉     | 1950/4000 [2:17:02<2:23:19,  4.19s/it]
 49%|████▉     | 1951/4000 [2:17:06<2:22:55,  4.19s/it]
 49%|████▉     | 1952/4000 [2:17:10<2:22:36,  4.18s/it]
 49%|████▉     | 1953/4000 [2:17:15<2:22:25,  4.17s/it]
 49%|████▉     | 1954/4000 [2:17:19<2:25:57,  4.28s/it]
 49%|████▉     | 1955/4000 [2:17:23<2:24:32,  4.24s/it]
 49%|████▉     | 1956/4000 [2:17:27<2:23:40,  4.22s/it]
 49%|████▉     | 1957/4000 [2:17:32<2:23:03,  4.20s/it]
 49%|████▉     | 1958/4000 [2:17:36<2:22:32,  4.19s/it]
 49%|████▉     | 1959/4000 [2:17:40<2:22:14,  4.18s/it]
 49%|████▉     | 1960/4000 [2:17:44<2:21:52,  4.17s/it]
                                                       
{'loss': '0.4479', 'grad_norm': '0.1186', 'learning_rate': '0.0003092', 'epoch': '0.4749'}

 49%|████▉     | 1960/4000 [2:17:44<2:21:52,  4.17s/it]
 49%|████▉     | 1961/4000 [2:17:49<2:25:28,  4.28s/it]
 49%|████▉     | 1962/4000 [2:17:53<2:24:02,  4.24s/it]
 49%|████▉     | 1963/4000 [2:17:57<2:23:08,  4.22s/it]
 49%|████▉     | 1964/4000 [2:18:01<2:22:20,  4.19s/it]
 49%|████▉     | 1965/4000 [2:18:05<2:21:46,  4.18s/it]
 49%|████▉     | 1966/4000 [2:18:09<2:21:34,  4.18s/it]
 49%|████▉     | 1967/4000 [2:18:14<2:24:47,  4.27s/it]
 49%|████▉     | 1968/4000 [2:18:18<2:23:21,  4.23s/it]
 49%|████▉     | 1969/4000 [2:18:22<2:22:02,  4.20s/it]
 49%|████▉     | 1970/4000 [2:18:26<2:21:23,  4.18s/it]
                                                       
{'loss': '0.4695', 'grad_norm': '0.1176', 'learning_rate': '0.0003077', 'epoch': '0.4773'}

 49%|████▉     | 1970/4000 [2:18:26<2:21:23,  4.18s/it]
 49%|████▉     | 1971/4000 [2:18:30<2:20:32,  4.16s/it]
 49%|████▉     | 1972/4000 [2:18:34<2:19:54,  4.14s/it]
 49%|████▉     | 1973/4000 [2:18:38<2:19:27,  4.13s/it]
 49%|████▉     | 1974/4000 [2:18:43<2:22:44,  4.23s/it]
 49%|████▉     | 1975/4000 [2:18:47<2:21:23,  4.19s/it]
 49%|████▉     | 1976/4000 [2:18:51<2:20:19,  4.16s/it]
 49%|████▉     | 1977/4000 [2:18:55<2:19:39,  4.14s/it]
 49%|████▉     | 1978/4000 [2:18:59<2:19:16,  4.13s/it]
 49%|████▉     | 1979/4000 [2:19:03<2:18:56,  4.13s/it]
 50%|████▉     | 1980/4000 [2:19:08<2:18:42,  4.12s/it]
                                                       
{'loss': '0.4677', 'grad_norm': '0.127', 'learning_rate': '0.0003062', 'epoch': '0.4798'}

 50%|████▉     | 1980/4000 [2:19:08<2:18:42,  4.12s/it]
 50%|████▉     | 1981/4000 [2:19:12<2:21:39,  4.21s/it]
 50%|████▉     | 1982/4000 [2:19:16<2:20:26,  4.18s/it]
 50%|████▉     | 1983/4000 [2:19:20<2:19:40,  4.16s/it]
 50%|████▉     | 1984/4000 [2:19:24<2:19:04,  4.14s/it]
 50%|████▉     | 1985/4000 [2:19:28<2:18:40,  4.13s/it]
 50%|████▉     | 1986/4000 [2:19:33<2:18:19,  4.12s/it]
 50%|████▉     | 1987/4000 [2:19:37<2:18:04,  4.12s/it]
 50%|████▉     | 1988/4000 [2:19:41<2:21:35,  4.22s/it]
 50%|████▉     | 1989/4000 [2:19:45<2:20:17,  4.19s/it]
 50%|████▉     | 1990/4000 [2:19:49<2:19:18,  4.16s/it]
                                                       
{'loss': '0.4632', 'grad_norm': '0.1225', 'learning_rate': '0.0003047', 'epoch': '0.4822'}

 50%|████▉     | 1990/4000 [2:19:49<2:19:18,  4.16s/it]
 50%|████▉     | 1991/4000 [2:19:53<2:18:39,  4.14s/it]
 50%|████▉     | 1992/4000 [2:19:57<2:18:14,  4.13s/it]
 50%|████▉     | 1993/4000 [2:20:02<2:17:57,  4.12s/it]
 50%|████▉     | 1994/4000 [2:20:06<2:21:14,  4.22s/it]
 50%|████▉     | 1995/4000 [2:20:10<2:19:52,  4.19s/it]
 50%|████▉     | 1996/4000 [2:20:14<2:18:54,  4.16s/it]
 50%|████▉     | 1997/4000 [2:20:18<2:19:11,  4.17s/it]
 50%|████▉     | 1998/4000 [2:20:23<2:19:32,  4.18s/it]
 50%|████▉     | 1999/4000 [2:20:27<2:18:38,  4.16s/it]
 50%|█████     | 2000/4000 [2:20:31<2:17:53,  4.14s/it]
                                                       
{'loss': '0.4511', 'grad_norm': '0.1182', 'learning_rate': '0.0003032', 'epoch': '0.4846'}

 50%|█████     | 2000/4000 [2:20:31<2:17:53,  4.14s/it]

  0%|          | 0/2 [00:00<?, ?it/s]

100%|██████████| 2/2 [00:00<00:00,  4.28it/s]
                                                       

                                             
{'eval_loss': '2.541', 'eval_runtime': '0.9973', 'eval_samples_per_second': '22.06', 'eval_steps_per_second': '2.005', 'epoch': '0.4846'}

 50%|█████     | 2000/4000 [2:20:32<2:17:53,  4.14s/it]

100%|██████████| 2/2 [00:00<00:00,  4.28it/s]

                                             [transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
  - If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
  - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
  - If you are not the owner of the model architecture class, please contact the model code owner to update it.


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:00<00:00,  3.64it/s]
Writing model shards: 100%|██████████| 1/1 [00:00<00:00,  3.63it/s]

 50%|█████     | 2001/4000 [2:20:40<3:10:18,  5.71s/it]
 50%|█████     | 2002/4000 [2:20:44<2:54:03,  5.23s/it]
 50%|█████     | 2003/4000 [2:20:48<2:42:39,  4.89s/it]
 50%|█████     | 2004/4000 [2:20:53<2:34:43,  4.65s/it]
 50%|█████     | 2005/4000 [2:20:57<2:29:05,  4.48s/it]
 50%|█████     | 2006/4000 [2:21:01<2:25:06,  4.37s/it]
 50%|█████     | 2007/4000 [2:21:05<2:22:25,  4.29s/it]
 50%|█████     | 2008/4000 [2:21:09<2:23:45,  4.33s/it]
 50%|█████     | 2009/4000 [2:21:13<2:21:25,  4.26s/it]
 50%|█████     | 2010/4000 [2:21:17<2:19:44,  4.21s/it]
                                                       
{'loss': '0.4894', 'grad_norm': '0.1521', 'learning_rate': '0.0003017', 'epoch': '0.487'}

 50%|█████     | 2010/4000 [2:21:17<2:19:44,  4.21s/it]
 50%|█████     | 2011/4000 [2:21:22<2:18:32,  4.18s/it]
 50%|█████     | 2012/4000 [2:21:26<2:17:42,  4.16s/it]
 50%|█████     | 2013/4000 [2:21:30<2:17:01,  4.14s/it]
 50%|█████     | 2014/4000 [2:21:34<2:16:29,  4.12s/it]
 50%|█████     | 2015/4000 [2:21:38<2:19:32,  4.22s/it]
 50%|█████     | 2016/4000 [2:21:42<2:18:23,  4.19s/it]
 50%|█████     | 2017/4000 [2:21:46<2:17:23,  4.16s/it]
 50%|█████     | 2018/4000 [2:21:51<2:16:41,  4.14s/it]
 50%|█████     | 2019/4000 [2:21:55<2:16:13,  4.13s/it]
 50%|█████     | 2020/4000 [2:21:59<2:15:48,  4.12s/it]
                                                       
{'loss': '0.4608', 'grad_norm': '0.1268', 'learning_rate': '0.0003002', 'epoch': '0.4894'}

 50%|█████     | 2020/4000 [2:21:59<2:15:48,  4.12s/it]
 51%|█████     | 2021/4000 [2:22:03<2:18:48,  4.21s/it]
 51%|█████     | 2022/4000 [2:22:07<2:17:42,  4.18s/it]
 51%|█████     | 2023/4000 [2:22:11<2:16:53,  4.15s/it]
 51%|█████     | 2024/4000 [2:22:15<2:16:13,  4.14s/it]
 51%|█████     | 2025/4000 [2:22:20<2:15:48,  4.13s/it]
 51%|█████     | 2026/4000 [2:22:24<2:15:33,  4.12s/it]
 51%|█████     | 2027/4000 [2:22:28<2:15:14,  4.11s/it]
 51%|█████     | 2028/4000 [2:22:32<2:18:18,  4.21s/it]
 51%|█████     | 2029/4000 [2:22:36<2:17:07,  4.17s/it]
 51%|█████     | 2030/4000 [2:22:40<2:16:16,  4.15s/it]
                                                       
{'loss': '0.4211', 'grad_norm': '0.119', 'learning_rate': '0.0002986', 'epoch': '0.4919'}

 51%|█████     | 2030/4000 [2:22:40<2:16:16,  4.15s/it]
 51%|█████     | 2031/4000 [2:22:44<2:15:39,  4.13s/it]
 51%|█████     | 2032/4000 [2:22:49<2:15:20,  4.13s/it]
 51%|█████     | 2033/4000 [2:22:53<2:15:00,  4.12s/it]
 51%|█████     | 2034/4000 [2:22:57<2:14:50,  4.12s/it]
 51%|█████     | 2035/4000 [2:23:01<2:18:11,  4.22s/it]
 51%|█████     | 2036/4000 [2:23:05<2:16:55,  4.18s/it]
 51%|█████     | 2037/4000 [2:23:09<2:15:58,  4.16s/it]
 51%|█████     | 2038/4000 [2:23:14<2:15:19,  4.14s/it]
 51%|█████     | 2039/4000 [2:23:18<2:14:53,  4.13s/it]
 51%|█████     | 2040/4000 [2:23:22<2:14:31,  4.12s/it]
                                                       
{'loss': '0.4982', 'grad_norm': '0.1192', 'learning_rate': '0.0002971', 'epoch': '0.4943'}

 51%|█████     | 2040/4000 [2:23:22<2:14:31,  4.12s/it]
 51%|█████     | 2041/4000 [2:23:26<2:15:41,  4.16s/it]
 51%|█████     | 2042/4000 [2:23:30<2:16:26,  4.18s/it]
 51%|█████     | 2043/4000 [2:23:34<2:15:33,  4.16s/it]
 51%|█████     | 2044/4000 [2:23:38<2:14:56,  4.14s/it]
 51%|█████     | 2045/4000 [2:23:43<2:14:34,  4.13s/it]
 51%|█████     | 2046/4000 [2:23:47<2:14:13,  4.12s/it]
 51%|█████     | 2047/4000 [2:23:51<2:13:52,  4.11s/it]
 51%|█████     | 2048/4000 [2:23:55<2:16:58,  4.21s/it]
 51%|█████     | 2049/4000 [2:23:59<2:15:51,  4.18s/it]
 51%|█████▏    | 2050/4000 [2:24:03<2:14:56,  4.15s/it]
                                                       
{'loss': '0.4563', 'grad_norm': '0.1189', 'learning_rate': '0.0002956', 'epoch': '0.4967'}

 51%|█████▏    | 2050/4000 [2:24:03<2:14:56,  4.15s/it]
 51%|█████▏    | 2051/4000 [2:24:07<2:14:24,  4.14s/it]
 51%|█████▏    | 2052/4000 [2:24:12<2:14:01,  4.13s/it]
 51%|█████▏    | 2053/4000 [2:24:16<2:13:39,  4.12s/it]
 51%|█████▏    | 2054/4000 [2:24:20<2:13:24,  4.11s/it]
 51%|█████▏    | 2055/4000 [2:24:24<2:16:40,  4.22s/it]
 51%|█████▏    | 2056/4000 [2:24:28<2:15:30,  4.18s/it]
 51%|█████▏    | 2057/4000 [2:24:32<2:14:39,  4.16s/it]
 51%|█████▏    | 2058/4000 [2:24:37<2:13:54,  4.14s/it]
 51%|█████▏    | 2059/4000 [2:24:41<2:13:26,  4.13s/it]
 52%|█████▏    | 2060/4000 [2:24:45<2:13:06,  4.12s/it]
                                                       
{'loss': '0.4767', 'grad_norm': '0.1314', 'learning_rate': '0.0002941', 'epoch': '0.4991'}

 52%|█████▏    | 2060/4000 [2:24:45<2:13:06,  4.12s/it]
 52%|█████▏    | 2061/4000 [2:24:49<2:12:54,  4.11s/it]
 52%|█████▏    | 2062/4000 [2:24:53<2:16:11,  4.22s/it]
 52%|█████▏    | 2063/4000 [2:24:57<2:15:02,  4.18s/it]
 52%|█████▏    | 2064/4000 [2:25:01<2:14:07,  4.16s/it]
 52%|█████▏    | 2065/4000 [2:25:06<2:13:31,  4.14s/it]
 52%|█████▏    | 2066/4000 [2:25:10<2:13:12,  4.13s/it]
 52%|█████▏    | 2067/4000 [2:25:14<2:12:54,  4.13s/it]
 52%|█████▏    | 2068/4000 [2:25:18<2:14:18,  4.17s/it]
 52%|█████▏    | 2069/4000 [2:25:22<2:15:11,  4.20s/it]
 52%|█████▏    | 2070/4000 [2:25:26<2:14:15,  4.17s/it]
                                                       
{'loss': '0.465', 'grad_norm': '0.1344', 'learning_rate': '0.0002926', 'epoch': '0.5016'}

 52%|█████▏    | 2070/4000 [2:25:26<2:14:15,  4.17s/it]
 52%|█████▏    | 2071/4000 [2:25:31<2:13:33,  4.15s/it]
 52%|█████▏    | 2072/4000 [2:25:35<2:14:04,  4.17s/it]
 52%|█████▏    | 2073/4000 [2:25:39<2:14:45,  4.20s/it]
 52%|█████▏    | 2074/4000 [2:25:43<2:13:56,  4.17s/it]
 52%|█████▏    | 2075/4000 [2:25:48<2:16:25,  4.25s/it]
 52%|█████▏    | 2076/4000 [2:25:52<2:15:09,  4.21s/it]
 52%|█████▏    | 2077/4000 [2:25:56<2:14:09,  4.19s/it]
 52%|█████▏    | 2078/4000 [2:26:00<2:13:26,  4.17s/it]
 52%|█████▏    | 2079/4000 [2:26:04<2:12:50,  4.15s/it]
 52%|█████▏    | 2080/4000 [2:26:08<2:12:21,  4.14s/it]
                                                       
{'loss': '0.4363', 'grad_norm': '0.1398', 'learning_rate': '0.0002911', 'epoch': '0.504'}

 52%|█████▏    | 2080/4000 [2:26:08<2:12:21,  4.14s/it]
 52%|█████▏    | 2081/4000 [2:26:12<2:12:03,  4.13s/it]
 52%|█████▏    | 2082/4000 [2:26:17<2:15:02,  4.22s/it]
 52%|█████▏    | 2083/4000 [2:26:21<2:13:42,  4.19s/it]
 52%|█████▏    | 2084/4000 [2:26:25<2:13:13,  4.17s/it]
 52%|█████▏    | 2085/4000 [2:26:29<2:12:37,  4.16s/it]
 52%|█████▏    | 2086/4000 [2:26:33<2:12:04,  4.14s/it]
 52%|█████▏    | 2087/4000 [2:26:37<2:11:43,  4.13s/it]
 52%|█████▏    | 2088/4000 [2:26:41<2:11:30,  4.13s/it]
 52%|█████▏    | 2089/4000 [2:26:46<2:14:28,  4.22s/it]
 52%|█████▏    | 2090/4000 [2:26:50<2:13:24,  4.19s/it]
                                                       
{'loss': '0.4345', 'grad_norm': '0.131', 'learning_rate': '0.0002895', 'epoch': '0.5064'}

 52%|█████▏    | 2090/4000 [2:26:50<2:13:24,  4.19s/it]
 52%|█████▏    | 2091/4000 [2:26:54<2:12:36,  4.17s/it]
 52%|█████▏    | 2092/4000 [2:26:58<2:12:11,  4.16s/it]
 52%|█████▏    | 2093/4000 [2:27:02<2:11:47,  4.15s/it]
 52%|█████▏    | 2094/4000 [2:27:06<2:11:30,  4.14s/it]
 52%|█████▏    | 2095/4000 [2:27:11<2:12:59,  4.19s/it]
 52%|█████▏    | 2096/4000 [2:27:15<2:13:58,  4.22s/it]
 52%|█████▏    | 2097/4000 [2:27:19<2:12:50,  4.19s/it]
 52%|█████▏    | 2098/4000 [2:27:23<2:12:04,  4.17s/it]
 52%|█████▏    | 2099/4000 [2:27:27<2:11:34,  4.15s/it]
 52%|█████▎    | 2100/4000 [2:27:32<2:11:26,  4.15s/it]
                                                       
{'loss': '0.4314', 'grad_norm': '0.1013', 'learning_rate': '0.000288', 'epoch': '0.5088'}

 52%|█████▎    | 2100/4000 [2:27:32<2:11:26,  4.15s/it]
 53%|█████▎    | 2101/4000 [2:27:36<2:11:06,  4.14s/it]
 53%|█████▎    | 2102/4000 [2:27:40<2:14:07,  4.24s/it]
 53%|█████▎    | 2103/4000 [2:27:44<2:12:49,  4.20s/it]
 53%|█████▎    | 2104/4000 [2:27:48<2:11:48,  4.17s/it]
 53%|█████▎    | 2105/4000 [2:27:52<2:11:01,  4.15s/it]
 53%|█████▎    | 2106/4000 [2:27:57<2:10:28,  4.13s/it]
 53%|█████▎    | 2107/4000 [2:28:01<2:10:05,  4.12s/it]
 53%|█████▎    | 2108/4000 [2:28:05<2:09:51,  4.12s/it]
 53%|█████▎    | 2109/4000 [2:28:09<2:12:32,  4.21s/it]
 53%|█████▎    | 2110/4000 [2:28:13<2:11:33,  4.18s/it]
                                                       
{'loss': '0.4538', 'grad_norm': '0.1193', 'learning_rate': '0.0002865', 'epoch': '0.5113'}

 53%|█████▎    | 2110/4000 [2:28:13<2:11:33,  4.18s/it]
 53%|█████▎    | 2111/4000 [2:28:17<2:10:45,  4.15s/it]
 53%|█████▎    | 2112/4000 [2:28:21<2:10:14,  4.14s/it]
 53%|█████▎    | 2113/4000 [2:28:26<2:09:50,  4.13s/it]
 53%|█████▎    | 2114/4000 [2:28:30<2:09:32,  4.12s/it]
 53%|█████▎    | 2115/4000 [2:28:34<2:09:21,  4.12s/it]
 53%|█████▎    | 2116/4000 [2:28:38<2:12:42,  4.23s/it]
 53%|█████▎    | 2117/4000 [2:28:42<2:11:28,  4.19s/it]
 53%|█████▎    | 2118/4000 [2:28:47<2:10:39,  4.17s/it]
 53%|█████▎    | 2119/4000 [2:28:51<2:10:21,  4.16s/it]
 53%|█████▎    | 2120/4000 [2:28:55<2:09:49,  4.14s/it]
                                                       
{'loss': '0.447', 'grad_norm': '0.1392', 'learning_rate': '0.000285', 'epoch': '0.5137'}

 53%|█████▎    | 2120/4000 [2:28:55<2:09:49,  4.14s/it]
 53%|█████▎    | 2121/4000 [2:28:59<2:09:23,  4.13s/it]
 53%|█████▎    | 2122/4000 [2:29:03<2:12:34,  4.24s/it]
 53%|█████▎    | 2123/4000 [2:29:07<2:11:17,  4.20s/it]
 53%|█████▎    | 2124/4000 [2:29:12<2:10:22,  4.17s/it]
 53%|█████▎    | 2125/4000 [2:29:16<2:09:45,  4.15s/it]
 53%|█████▎    | 2126/4000 [2:29:20<2:09:17,  4.14s/it]
 53%|█████▎    | 2127/4000 [2:29:24<2:08:59,  4.13s/it]
 53%|█████▎    | 2128/4000 [2:29:28<2:08:42,  4.13s/it]
 53%|█████▎    | 2129/4000 [2:29:32<2:11:42,  4.22s/it]
 53%|█████▎    | 2130/4000 [2:29:37<2:10:41,  4.19s/it]
                                                       
{'loss': '0.4419', 'grad_norm': '0.2505', 'learning_rate': '0.0002835', 'epoch': '0.5161'}

 53%|█████▎    | 2130/4000 [2:29:37<2:10:41,  4.19s/it]
 53%|█████▎    | 2131/4000 [2:29:41<2:09:53,  4.17s/it]
 53%|█████▎    | 2132/4000 [2:29:45<2:09:17,  4.15s/it]
 53%|█████▎    | 2133/4000 [2:29:49<2:08:50,  4.14s/it]
 53%|█████▎    | 2134/4000 [2:29:53<2:08:31,  4.13s/it]
 53%|█████▎    | 2135/4000 [2:29:57<2:08:17,  4.13s/it]
 53%|█████▎    | 2136/4000 [2:30:02<2:11:14,  4.22s/it]
 53%|█████▎    | 2137/4000 [2:30:06<2:10:10,  4.19s/it]
 53%|█████▎    | 2138/4000 [2:30:10<2:09:21,  4.17s/it]
 53%|█████▎    | 2139/4000 [2:30:14<2:08:51,  4.15s/it]
 54%|█████▎    | 2140/4000 [2:30:18<2:08:25,  4.14s/it]
                                                       
{'loss': '0.465', 'grad_norm': '0.1463', 'learning_rate': '0.000282', 'epoch': '0.5185'}

 54%|█████▎    | 2140/4000 [2:30:18<2:08:25,  4.14s/it]
 54%|█████▎    | 2141/4000 [2:30:22<2:08:01,  4.13s/it]
 54%|█████▎    | 2142/4000 [2:30:26<2:07:45,  4.13s/it]
 54%|█████▎    | 2143/4000 [2:30:31<2:10:59,  4.23s/it]
 54%|█████▎    | 2144/4000 [2:30:35<2:09:50,  4.20s/it]
 54%|█████▎    | 2145/4000 [2:30:39<2:08:58,  4.17s/it]
 54%|█████▎    | 2146/4000 [2:30:43<2:08:26,  4.16s/it]
 54%|█████▎    | 2147/4000 [2:30:47<2:08:03,  4.15s/it]
 54%|█████▎    | 2148/4000 [2:30:51<2:07:40,  4.14s/it]
 54%|█████▎    | 2149/4000 [2:30:56<2:09:08,  4.19s/it]
 54%|█████▍    | 2150/4000 [2:31:00<2:09:56,  4.21s/it]
                                                       
{'loss': '0.4351', 'grad_norm': '0.1265', 'learning_rate': '0.0002805', 'epoch': '0.5209'}

 54%|█████▍    | 2150/4000 [2:31:00<2:09:56,  4.21s/it]
 54%|█████▍    | 2151/4000 [2:31:04<2:08:55,  4.18s/it]
 54%|█████▍    | 2152/4000 [2:31:08<2:08:15,  4.16s/it]
 54%|█████▍    | 2153/4000 [2:31:12<2:07:38,  4.15s/it]
 54%|█████▍    | 2154/4000 [2:31:16<2:07:15,  4.14s/it]
 54%|█████▍    | 2155/4000 [2:31:20<2:06:57,  4.13s/it]
 54%|█████▍    | 2156/4000 [2:31:25<2:09:52,  4.23s/it]
 54%|█████▍    | 2157/4000 [2:31:29<2:08:50,  4.19s/it]
 54%|█████▍    | 2158/4000 [2:31:33<2:08:04,  4.17s/it]
 54%|█████▍    | 2159/4000 [2:31:37<2:07:31,  4.16s/it]
 54%|█████▍    | 2160/4000 [2:31:41<2:07:17,  4.15s/it]
                                                       
{'loss': '0.45', 'grad_norm': '0.1343', 'learning_rate': '0.0002789', 'epoch': '0.5234'}

 54%|█████▍    | 2160/4000 [2:31:41<2:07:17,  4.15s/it]
 54%|█████▍    | 2161/4000 [2:31:46<2:06:58,  4.14s/it]
 54%|█████▍    | 2162/4000 [2:31:50<2:06:35,  4.13s/it]
 54%|█████▍    | 2163/4000 [2:31:54<2:09:52,  4.24s/it]
 54%|█████▍    | 2164/4000 [2:31:58<2:08:40,  4.21s/it]
 54%|█████▍    | 2165/4000 [2:32:02<2:07:44,  4.18s/it]
 54%|█████▍    | 2166/4000 [2:32:07<2:07:13,  4.16s/it]
 54%|█████▍    | 2167/4000 [2:32:11<2:06:55,  4.15s/it]
 54%|█████▍    | 2168/4000 [2:32:15<2:06:27,  4.14s/it]
 54%|█████▍    | 2169/4000 [2:32:19<2:07:51,  4.19s/it]
 54%|█████▍    | 2170/4000 [2:32:23<2:09:12,  4.24s/it]
                                                       
{'loss': '0.4569', 'grad_norm': '0.1064', 'learning_rate': '0.0002774', 'epoch': '0.5258'}

 54%|█████▍    | 2170/4000 [2:32:23<2:09:12,  4.24s/it]
 54%|█████▍    | 2171/4000 [2:32:28<2:08:02,  4.20s/it]
 54%|█████▍    | 2172/4000 [2:32:32<2:07:05,  4.17s/it]
 54%|█████▍    | 2173/4000 [2:32:36<2:06:23,  4.15s/it]
 54%|█████▍    | 2174/4000 [2:32:40<2:06:08,  4.14s/it]
 54%|█████▍    | 2175/4000 [2:32:44<2:05:44,  4.13s/it]
 54%|█████▍    | 2176/4000 [2:32:48<2:07:51,  4.21s/it]
 54%|█████▍    | 2177/4000 [2:32:52<2:06:50,  4.17s/it]
 54%|█████▍    | 2178/4000 [2:32:57<2:06:05,  4.15s/it]
 54%|█████▍    | 2179/4000 [2:33:01<2:05:34,  4.14s/it]
 55%|█████▍    | 2180/4000 [2:33:05<2:05:29,  4.14s/it]
                                                       
{'loss': '0.4537', 'grad_norm': '0.1231', 'learning_rate': '0.0002759', 'epoch': '0.5282'}

 55%|█████▍    | 2180/4000 [2:33:05<2:05:29,  4.14s/it]
 55%|█████▍    | 2181/4000 [2:33:09<2:06:05,  4.16s/it]
 55%|█████▍    | 2182/4000 [2:33:13<2:05:43,  4.15s/it]
 55%|█████▍    | 2183/4000 [2:33:18<2:08:49,  4.25s/it]
 55%|█████▍    | 2184/4000 [2:33:22<2:07:31,  4.21s/it]
 55%|█████▍    | 2185/4000 [2:33:26<2:06:36,  4.19s/it]
 55%|█████▍    | 2186/4000 [2:33:30<2:05:57,  4.17s/it]
 55%|█████▍    | 2187/4000 [2:33:34<2:05:30,  4.15s/it]
 55%|█████▍    | 2188/4000 [2:33:38<2:05:05,  4.14s/it]
 55%|█████▍    | 2189/4000 [2:33:42<2:04:49,  4.14s/it]
 55%|█████▍    | 2190/4000 [2:33:47<2:07:54,  4.24s/it]
                                                       
{'loss': '0.4676', 'grad_norm': '0.122', 'learning_rate': '0.0002744', 'epoch': '0.5306'}

 55%|█████▍    | 2190/4000 [2:33:47<2:07:54,  4.24s/it]
 55%|█████▍    | 2191/4000 [2:33:51<2:06:44,  4.20s/it]
 55%|█████▍    | 2192/4000 [2:33:55<2:05:58,  4.18s/it]
 55%|█████▍    | 2193/4000 [2:33:59<2:05:21,  4.16s/it]
 55%|█████▍    | 2194/4000 [2:34:03<2:04:53,  4.15s/it]
 55%|█████▍    | 2195/4000 [2:34:07<2:04:34,  4.14s/it]
 55%|█████▍    | 2196/4000 [2:34:12<2:05:59,  4.19s/it]
 55%|█████▍    | 2197/4000 [2:34:16<2:06:42,  4.22s/it]
 55%|█████▍    | 2198/4000 [2:34:20<2:07:15,  4.24s/it]
 55%|█████▍    | 2199/4000 [2:34:25<2:07:32,  4.25s/it]
 55%|█████▌    | 2200/4000 [2:34:29<2:07:43,  4.26s/it]
                                                       
{'loss': '0.4642', 'grad_norm': '0.1354', 'learning_rate': '0.0002729', 'epoch': '0.5331'}

 55%|█████▌    | 2200/4000 [2:34:29<2:07:43,  4.26s/it]
 55%|█████▌    | 2201/4000 [2:34:33<2:07:44,  4.26s/it]
 55%|█████▌    | 2202/4000 [2:34:37<2:07:49,  4.27s/it]
 55%|█████▌    | 2203/4000 [2:34:42<2:11:41,  4.40s/it]
 55%|█████▌    | 2204/4000 [2:34:46<2:10:26,  4.36s/it]
 55%|█████▌    | 2205/4000 [2:34:51<2:08:24,  4.29s/it]
 55%|█████▌    | 2206/4000 [2:34:55<2:06:47,  4.24s/it]
 55%|█████▌    | 2207/4000 [2:34:59<2:05:43,  4.21s/it]
 55%|█████▌    | 2208/4000 [2:35:03<2:04:59,  4.19s/it]
 55%|█████▌    | 2209/4000 [2:35:07<2:04:30,  4.17s/it]
 55%|█████▌    | 2210/4000 [2:35:12<2:07:09,  4.26s/it]
                                                       
{'loss': '0.4432', 'grad_norm': '0.1118', 'learning_rate': '0.0002714', 'epoch': '0.5355'}

 55%|█████▌    | 2210/4000 [2:35:12<2:07:09,  4.26s/it]
 55%|█████▌    | 2211/4000 [2:35:16<2:05:51,  4.22s/it]
 55%|█████▌    | 2212/4000 [2:35:20<2:04:55,  4.19s/it]
 55%|█████▌    | 2213/4000 [2:35:24<2:04:16,  4.17s/it]
 55%|█████▌    | 2214/4000 [2:35:28<2:03:46,  4.16s/it]
 55%|█████▌    | 2215/4000 [2:35:32<2:03:26,  4.15s/it]
 55%|█████▌    | 2216/4000 [2:35:36<2:03:08,  4.14s/it]
 55%|█████▌    | 2217/4000 [2:35:41<2:06:14,  4.25s/it]
 55%|█████▌    | 2218/4000 [2:35:45<2:05:13,  4.22s/it]
 55%|█████▌    | 2219/4000 [2:35:49<2:04:28,  4.19s/it]
 56%|█████▌    | 2220/4000 [2:35:53<2:03:51,  4.17s/it]
                                                       
{'loss': '0.4497', 'grad_norm': '0.13', 'learning_rate': '0.0002698', 'epoch': '0.5379'}

 56%|█████▌    | 2220/4000 [2:35:53<2:03:51,  4.17s/it]
 56%|█████▌    | 2221/4000 [2:35:58<2:05:38,  4.24s/it]
 56%|█████▌    | 2222/4000 [2:36:02<2:06:07,  4.26s/it]
 56%|█████▌    | 2223/4000 [2:36:06<2:08:18,  4.33s/it]
 56%|█████▌    | 2224/4000 [2:36:11<2:09:42,  4.38s/it]
 56%|█████▌    | 2225/4000 [2:36:15<2:08:49,  4.35s/it]
 56%|█████▌    | 2226/4000 [2:36:19<2:08:05,  4.33s/it]
 56%|█████▌    | 2227/4000 [2:36:24<2:06:21,  4.28s/it]
 56%|█████▌    | 2228/4000 [2:36:28<2:05:07,  4.24s/it]
 56%|█████▌    | 2229/4000 [2:36:32<2:04:08,  4.21s/it]
 56%|█████▌    | 2230/4000 [2:36:36<2:07:08,  4.31s/it]
                                                       
{'loss': '0.4474', 'grad_norm': '0.121', 'learning_rate': '0.0002683', 'epoch': '0.5403'}

 56%|█████▌    | 2230/4000 [2:36:36<2:07:08,  4.31s/it]
 56%|█████▌    | 2231/4000 [2:36:41<2:05:27,  4.26s/it]
 56%|█████▌    | 2232/4000 [2:36:45<2:04:24,  4.22s/it]
 56%|█████▌    | 2233/4000 [2:36:49<2:03:36,  4.20s/it]
 56%|█████▌    | 2234/4000 [2:36:53<2:03:08,  4.18s/it]
 56%|█████▌    | 2235/4000 [2:36:57<2:02:42,  4.17s/it]
 56%|█████▌    | 2236/4000 [2:37:01<2:02:18,  4.16s/it]
 56%|█████▌    | 2237/4000 [2:37:06<2:05:42,  4.28s/it]
 56%|█████▌    | 2238/4000 [2:37:10<2:04:27,  4.24s/it]
 56%|█████▌    | 2239/4000 [2:37:14<2:04:04,  4.23s/it]
 56%|█████▌    | 2240/4000 [2:37:18<2:04:03,  4.23s/it]
                                                       
{'loss': '0.4517', 'grad_norm': '0.122', 'learning_rate': '0.0002668', 'epoch': '0.5428'}

 56%|█████▌    | 2240/4000 [2:37:18<2:04:03,  4.23s/it]
 56%|█████▌    | 2241/4000 [2:37:23<2:03:09,  4.20s/it]
 56%|█████▌    | 2242/4000 [2:37:27<2:05:13,  4.27s/it]
 56%|█████▌    | 2243/4000 [2:37:31<2:05:35,  4.29s/it]
 56%|█████▌    | 2244/4000 [2:37:36<2:08:48,  4.40s/it]
 56%|█████▌    | 2245/4000 [2:37:40<2:07:41,  4.37s/it]
 56%|█████▌    | 2246/4000 [2:37:44<2:06:02,  4.31s/it]
 56%|█████▌    | 2247/4000 [2:37:49<2:04:24,  4.26s/it]
 56%|█████▌    | 2248/4000 [2:37:53<2:03:14,  4.22s/it]
 56%|█████▌    | 2249/4000 [2:37:57<2:02:25,  4.20s/it]
 56%|█████▋    | 2250/4000 [2:38:01<2:03:37,  4.24s/it]
                                                       
{'loss': '0.4452', 'grad_norm': '0.1265', 'learning_rate': '0.0002653', 'epoch': '0.5452'}

 56%|█████▋    | 2250/4000 [2:38:01<2:03:37,  4.24s/it]
 56%|█████▋    | 2251/4000 [2:38:05<2:04:18,  4.26s/it]
 56%|█████▋    | 2252/4000 [2:38:10<2:03:03,  4.22s/it]
 56%|█████▋    | 2253/4000 [2:38:14<2:02:14,  4.20s/it]
 56%|█████▋    | 2254/4000 [2:38:18<2:01:33,  4.18s/it]
 56%|█████▋    | 2255/4000 [2:38:22<2:01:08,  4.17s/it]
 56%|█████▋    | 2256/4000 [2:38:26<2:03:03,  4.23s/it]
 56%|█████▋    | 2257/4000 [2:38:31<2:05:38,  4.33s/it]
 56%|█████▋    | 2258/4000 [2:38:35<2:04:13,  4.28s/it]
 56%|█████▋    | 2259/4000 [2:38:39<2:02:57,  4.24s/it]
 56%|█████▋    | 2260/4000 [2:38:43<2:02:00,  4.21s/it]
                                                       
{'loss': '0.4498', 'grad_norm': '0.09977', 'learning_rate': '0.0002638', 'epoch': '0.5476'}

 56%|█████▋    | 2260/4000 [2:38:43<2:02:00,  4.21s/it]
 57%|█████▋    | 2261/4000 [2:38:48<2:02:10,  4.22s/it]
 57%|█████▋    | 2262/4000 [2:38:52<2:02:06,  4.22s/it]
 57%|█████▋    | 2263/4000 [2:38:56<2:02:28,  4.23s/it]
 57%|█████▋    | 2264/4000 [2:39:01<2:06:50,  4.38s/it]
 57%|█████▋    | 2265/4000 [2:39:05<2:04:48,  4.32s/it]
 57%|█████▋    | 2266/4000 [2:39:09<2:03:07,  4.26s/it]
 57%|█████▋    | 2267/4000 [2:39:13<2:01:59,  4.22s/it]
 57%|█████▋    | 2268/4000 [2:39:17<2:01:08,  4.20s/it]
 57%|█████▋    | 2269/4000 [2:39:22<2:00:29,  4.18s/it]
 57%|█████▋    | 2270/4000 [2:39:26<1:59:53,  4.16s/it]
                                                       
{'loss': '0.4221', 'grad_norm': '0.1154', 'learning_rate': '0.0002623', 'epoch': '0.55'}

 57%|█████▋    | 2270/4000 [2:39:26<1:59:53,  4.16s/it]
 57%|█████▋    | 2271/4000 [2:39:30<2:02:57,  4.27s/it]
 57%|█████▋    | 2272/4000 [2:39:34<2:01:41,  4.23s/it]
 57%|█████▋    | 2273/4000 [2:39:39<2:01:51,  4.23s/it]
 57%|█████▋    | 2274/4000 [2:39:43<2:00:57,  4.20s/it]
 57%|█████▋    | 2275/4000 [2:39:47<2:00:53,  4.21s/it]
 57%|█████▋    | 2276/4000 [2:39:51<2:00:09,  4.18s/it]
 57%|█████▋    | 2277/4000 [2:39:55<2:02:27,  4.26s/it]
 57%|█████▋    | 2278/4000 [2:40:00<2:02:20,  4.26s/it]
 57%|█████▋    | 2279/4000 [2:40:04<2:01:22,  4.23s/it]
 57%|█████▋    | 2280/4000 [2:40:08<2:00:30,  4.20s/it]
                                                       
{'loss': '0.4633', 'grad_norm': '0.1137', 'learning_rate': '0.0002608', 'epoch': '0.5524'}

 57%|█████▋    | 2280/4000 [2:40:08<2:00:30,  4.20s/it]
 57%|█████▋    | 2281/4000 [2:40:12<2:00:46,  4.22s/it]
 57%|█████▋    | 2282/4000 [2:40:17<2:01:20,  4.24s/it]
 57%|█████▋    | 2283/4000 [2:40:21<2:01:50,  4.26s/it]
 57%|█████▋    | 2284/4000 [2:40:25<2:04:02,  4.34s/it]
 57%|█████▋    | 2285/4000 [2:40:30<2:02:03,  4.27s/it]
 57%|█████▋    | 2286/4000 [2:40:34<2:00:51,  4.23s/it]
 57%|█████▋    | 2287/4000 [2:40:38<1:59:55,  4.20s/it]
 57%|█████▋    | 2288/4000 [2:40:42<1:59:19,  4.18s/it]
 57%|█████▋    | 2289/4000 [2:40:46<1:58:47,  4.17s/it]
 57%|█████▋    | 2290/4000 [2:40:50<1:59:17,  4.19s/it]
                                                       
{'loss': '0.4553', 'grad_norm': '0.1379', 'learning_rate': '0.0002592', 'epoch': '0.5549'}

 57%|█████▋    | 2290/4000 [2:40:50<1:59:17,  4.19s/it]
 57%|█████▋    | 2291/4000 [2:40:55<2:04:00,  4.35s/it]
 57%|█████▋    | 2292/4000 [2:40:59<2:02:23,  4.30s/it]
 57%|█████▋    | 2293/4000 [2:41:03<2:00:57,  4.25s/it]
 57%|█████▋    | 2294/4000 [2:41:08<2:00:01,  4.22s/it]
 57%|█████▋    | 2295/4000 [2:41:12<1:59:28,  4.20s/it]
 57%|█████▋    | 2296/4000 [2:41:16<2:00:48,  4.25s/it]
 57%|█████▋    | 2297/4000 [2:41:20<2:01:37,  4.28s/it]
 57%|█████▋    | 2298/4000 [2:41:25<2:02:06,  4.30s/it]
 57%|█████▋    | 2299/4000 [2:41:29<2:00:39,  4.26s/it]
 57%|█████▊    | 2300/4000 [2:41:33<2:00:28,  4.25s/it]
                                                       
{'loss': '0.4624', 'grad_norm': '0.1325', 'learning_rate': '0.0002577', 'epoch': '0.5573'}

 57%|█████▊    | 2300/4000 [2:41:33<2:00:28,  4.25s/it]
 58%|█████▊    | 2301/4000 [2:41:37<2:00:56,  4.27s/it]
 58%|█████▊    | 2302/4000 [2:41:42<2:00:21,  4.25s/it]
 58%|█████▊    | 2303/4000 [2:41:46<1:59:26,  4.22s/it]
 58%|█████▊    | 2304/4000 [2:41:50<2:01:47,  4.31s/it]
 58%|█████▊    | 2305/4000 [2:41:54<2:00:21,  4.26s/it]
 58%|█████▊    | 2306/4000 [2:41:59<1:59:20,  4.23s/it]
 58%|█████▊    | 2307/4000 [2:42:03<2:01:00,  4.29s/it]
 58%|█████▊    | 2308/4000 [2:42:07<2:02:12,  4.33s/it]
 58%|█████▊    | 2309/4000 [2:42:12<2:00:36,  4.28s/it]
 58%|█████▊    | 2310/4000 [2:42:16<1:59:26,  4.24s/it]
                                                       
{'loss': '0.4456', 'grad_norm': '0.1388', 'learning_rate': '0.0002562', 'epoch': '0.5597'}

 58%|█████▊    | 2310/4000 [2:42:16<1:59:26,  4.24s/it]
 58%|█████▊    | 2311/4000 [2:42:20<2:01:33,  4.32s/it]
 58%|█████▊    | 2312/4000 [2:42:24<2:00:18,  4.28s/it]
 58%|█████▊    | 2313/4000 [2:42:29<2:00:26,  4.28s/it]
 58%|█████▊    | 2314/4000 [2:42:33<1:59:20,  4.25s/it]
 58%|█████▊    | 2315/4000 [2:42:37<1:58:25,  4.22s/it]
 58%|█████▊    | 2316/4000 [2:42:41<1:57:52,  4.20s/it]
 58%|█████▊    | 2317/4000 [2:42:45<1:57:21,  4.18s/it]
 58%|█████▊    | 2318/4000 [2:42:50<2:00:28,  4.30s/it]
 58%|█████▊    | 2319/4000 [2:42:54<2:00:14,  4.29s/it]
 58%|█████▊    | 2320/4000 [2:42:59<2:00:22,  4.30s/it]
                                                       
{'loss': '0.4758', 'grad_norm': '0.1284', 'learning_rate': '0.0002547', 'epoch': '0.5621'}

 58%|█████▊    | 2320/4000 [2:42:59<2:00:22,  4.30s/it]
 58%|█████▊    | 2321/4000 [2:43:03<1:59:24,  4.27s/it]
 58%|█████▊    | 2322/4000 [2:43:07<1:58:19,  4.23s/it]
 58%|█████▊    | 2323/4000 [2:43:11<1:58:17,  4.23s/it]
 58%|█████▊    | 2324/4000 [2:43:15<1:59:27,  4.28s/it]
 58%|█████▊    | 2325/4000 [2:43:20<2:01:58,  4.37s/it]
 58%|█████▊    | 2326/4000 [2:43:24<2:00:06,  4.30s/it]
 58%|█████▊    | 2327/4000 [2:43:28<1:58:44,  4.26s/it]
 58%|█████▊    | 2328/4000 [2:43:33<1:57:42,  4.22s/it]
 58%|█████▊    | 2329/4000 [2:43:37<1:56:54,  4.20s/it]
 58%|█████▊    | 2330/4000 [2:43:41<1:58:19,  4.25s/it]
                                                       
{'loss': '0.4533', 'grad_norm': '0.1235', 'learning_rate': '0.0002532', 'epoch': '0.5646'}

 58%|█████▊    | 2330/4000 [2:43:41<1:58:19,  4.25s/it]
 58%|█████▊    | 2331/4000 [2:43:46<2:00:57,  4.35s/it]
 58%|█████▊    | 2332/4000 [2:43:50<1:59:11,  4.29s/it]
 58%|█████▊    | 2333/4000 [2:43:54<1:58:02,  4.25s/it]
 58%|█████▊    | 2334/4000 [2:43:58<1:57:06,  4.22s/it]
 58%|█████▊    | 2335/4000 [2:44:02<1:56:24,  4.19s/it]
 58%|█████▊    | 2336/4000 [2:44:06<1:56:00,  4.18s/it]
 58%|█████▊    | 2337/4000 [2:44:11<1:56:49,  4.21s/it]
 58%|█████▊    | 2338/4000 [2:44:15<2:01:03,  4.37s/it]
 58%|█████▊    | 2339/4000 [2:44:20<2:00:27,  4.35s/it]
 58%|█████▊    | 2340/4000 [2:44:24<1:59:03,  4.30s/it]
                                                       
{'loss': '0.4112', 'grad_norm': '0.1257', 'learning_rate': '0.0002517', 'epoch': '0.567'}

 58%|█████▊    | 2340/4000 [2:44:24<1:59:03,  4.30s/it]
 59%|█████▊    | 2341/4000 [2:44:28<1:57:51,  4.26s/it]
 59%|█████▊    | 2342/4000 [2:44:32<1:58:49,  4.30s/it]
 59%|█████▊    | 2343/4000 [2:44:37<1:57:59,  4.27s/it]
 59%|█████▊    | 2344/4000 [2:44:41<1:56:50,  4.23s/it]
 59%|█████▊    | 2345/4000 [2:44:45<1:58:50,  4.31s/it]
 59%|█████▊    | 2346/4000 [2:44:49<1:57:22,  4.26s/it]
 59%|█████▊    | 2347/4000 [2:44:54<1:57:16,  4.26s/it]
 59%|█████▊    | 2348/4000 [2:44:58<1:56:16,  4.22s/it]
 59%|█████▊    | 2349/4000 [2:45:02<1:55:28,  4.20s/it]
 59%|█████▉    | 2350/4000 [2:45:06<1:54:50,  4.18s/it]
                                                       
{'loss': '0.423', 'grad_norm': '0.1292', 'learning_rate': '0.0002502', 'epoch': '0.5694'}

 59%|█████▉    | 2350/4000 [2:45:06<1:54:50,  4.18s/it]
 59%|█████▉    | 2351/4000 [2:45:10<1:56:10,  4.23s/it]
 59%|█████▉    | 2352/4000 [2:45:15<1:56:42,  4.25s/it]
 59%|█████▉    | 2353/4000 [2:45:19<1:55:44,  4.22s/it]
 59%|█████▉    | 2354/4000 [2:45:23<1:55:01,  4.19s/it]
 59%|█████▉    | 2355/4000 [2:45:27<1:55:57,  4.23s/it]
 59%|█████▉    | 2356/4000 [2:45:32<1:57:34,  4.29s/it]
 59%|█████▉    | 2357/4000 [2:45:36<1:57:30,  4.29s/it]
 59%|█████▉    | 2358/4000 [2:45:41<1:59:24,  4.36s/it]
 59%|█████▉    | 2359/4000 [2:45:45<2:00:18,  4.40s/it]
 59%|█████▉    | 2360/4000 [2:45:49<1:58:09,  4.32s/it]
                                                       
{'loss': '0.4463', 'grad_norm': '0.1273', 'learning_rate': '0.0002486', 'epoch': '0.5718'}

 59%|█████▉    | 2360/4000 [2:45:49<1:58:09,  4.32s/it]
 59%|█████▉    | 2361/4000 [2:45:53<1:56:35,  4.27s/it]
 59%|█████▉    | 2362/4000 [2:45:57<1:55:34,  4.23s/it]
 59%|█████▉    | 2363/4000 [2:46:02<1:55:32,  4.23s/it]
 59%|█████▉    | 2364/4000 [2:46:06<1:56:15,  4.26s/it]
 59%|█████▉    | 2365/4000 [2:46:11<1:58:41,  4.36s/it]
 59%|█████▉    | 2366/4000 [2:46:15<1:56:58,  4.30s/it]
 59%|█████▉    | 2367/4000 [2:46:19<1:55:36,  4.25s/it]
 59%|█████▉    | 2368/4000 [2:46:23<1:54:42,  4.22s/it]
 59%|█████▉    | 2369/4000 [2:46:27<1:54:02,  4.20s/it]
 59%|█████▉    | 2370/4000 [2:46:31<1:53:30,  4.18s/it]
                                                       
{'loss': '0.4244', 'grad_norm': '0.1066', 'learning_rate': '0.0002471', 'epoch': '0.5742'}

 59%|█████▉    | 2370/4000 [2:46:31<1:53:30,  4.18s/it]
 59%|█████▉    | 2371/4000 [2:46:36<1:54:09,  4.20s/it]
 59%|█████▉    | 2372/4000 [2:46:40<1:56:54,  4.31s/it]
 59%|█████▉    | 2373/4000 [2:46:44<1:56:13,  4.29s/it]
 59%|█████▉    | 2374/4000 [2:46:49<1:56:39,  4.31s/it]
 59%|█████▉    | 2375/4000 [2:46:53<1:56:48,  4.31s/it]
 59%|█████▉    | 2376/4000 [2:46:57<1:56:34,  4.31s/it]
 59%|█████▉    | 2377/4000 [2:47:02<1:55:09,  4.26s/it]
 59%|█████▉    | 2378/4000 [2:47:06<1:55:37,  4.28s/it]
 59%|█████▉    | 2379/4000 [2:47:10<1:55:54,  4.29s/it]
 60%|█████▉    | 2380/4000 [2:47:14<1:56:10,  4.30s/it]
                                                       
{'loss': '0.4588', 'grad_norm': '0.1298', 'learning_rate': '0.0002456', 'epoch': '0.5767'}

 60%|█████▉    | 2380/4000 [2:47:14<1:56:10,  4.30s/it]
 60%|█████▉    | 2381/4000 [2:47:19<1:54:52,  4.26s/it]
 60%|█████▉    | 2382/4000 [2:47:23<1:53:47,  4.22s/it]
 60%|█████▉    | 2383/4000 [2:47:27<1:52:57,  4.19s/it]
 60%|█████▉    | 2384/4000 [2:47:31<1:52:29,  4.18s/it]
 60%|█████▉    | 2385/4000 [2:47:36<1:55:16,  4.28s/it]
 60%|█████▉    | 2386/4000 [2:47:40<1:54:02,  4.24s/it]
 60%|█████▉    | 2387/4000 [2:47:44<1:53:19,  4.22s/it]
 60%|█████▉    | 2388/4000 [2:47:48<1:54:41,  4.27s/it]
 60%|█████▉    | 2389/4000 [2:47:52<1:53:32,  4.23s/it]
 60%|█████▉    | 2390/4000 [2:47:57<1:52:47,  4.20s/it]
                                                       
{'loss': '0.422', 'grad_norm': '0.1311', 'learning_rate': '0.0002441', 'epoch': '0.5791'}

 60%|█████▉    | 2390/4000 [2:47:57<1:52:47,  4.20s/it]
 60%|█████▉    | 2391/4000 [2:48:01<1:54:43,  4.28s/it]
 60%|█████▉    | 2392/4000 [2:48:06<1:58:38,  4.43s/it]
 60%|█████▉    | 2393/4000 [2:48:10<1:58:17,  4.42s/it]
 60%|█████▉    | 2394/4000 [2:48:14<1:55:59,  4.33s/it]
 60%|█████▉    | 2395/4000 [2:48:18<1:54:16,  4.27s/it]
 60%|█████▉    | 2396/4000 [2:48:23<1:53:07,  4.23s/it]
 60%|█████▉    | 2397/4000 [2:48:27<1:53:45,  4.26s/it]
 60%|█████▉    | 2398/4000 [2:48:31<1:52:43,  4.22s/it]
 60%|█████▉    | 2399/4000 [2:48:36<1:54:56,  4.31s/it]
 60%|██████    | 2400/4000 [2:48:40<1:53:28,  4.26s/it]
                                                       
{'loss': '0.4471', 'grad_norm': '0.1121', 'learning_rate': '0.0002426', 'epoch': '0.5815'}

 60%|██████    | 2400/4000 [2:48:40<1:53:28,  4.26s/it]
 60%|██████    | 2401/4000 [2:48:44<1:52:23,  4.22s/it]
 60%|██████    | 2402/4000 [2:48:48<1:51:42,  4.19s/it]
 60%|██████    | 2403/4000 [2:48:52<1:51:15,  4.18s/it]
 60%|██████    | 2404/4000 [2:48:57<1:53:17,  4.26s/it]
 60%|██████    | 2405/4000 [2:49:01<1:56:21,  4.38s/it]
 60%|██████    | 2406/4000 [2:49:05<1:54:19,  4.30s/it]
 60%|██████    | 2407/4000 [2:49:09<1:53:17,  4.27s/it]
 60%|██████    | 2408/4000 [2:49:14<1:53:14,  4.27s/it]
 60%|██████    | 2409/4000 [2:49:18<1:52:50,  4.26s/it]
 60%|██████    | 2410/4000 [2:49:22<1:53:57,  4.30s/it]
                                                       
{'loss': '0.4311', 'grad_norm': '0.113', 'learning_rate': '0.0002411', 'epoch': '0.5839'}

 60%|██████    | 2410/4000 [2:49:22<1:53:57,  4.30s/it]
 60%|██████    | 2411/4000 [2:49:27<1:52:47,  4.26s/it]
 60%|██████    | 2412/4000 [2:49:31<1:54:37,  4.33s/it]
 60%|██████    | 2413/4000 [2:49:35<1:54:35,  4.33s/it]
 60%|██████    | 2414/4000 [2:49:40<1:53:27,  4.29s/it]
 60%|██████    | 2415/4000 [2:49:44<1:52:06,  4.24s/it]
 60%|██████    | 2416/4000 [2:49:48<1:51:10,  4.21s/it]
 60%|██████    | 2417/4000 [2:49:52<1:50:25,  4.19s/it]
 60%|██████    | 2418/4000 [2:49:56<1:50:00,  4.17s/it]
 60%|██████    | 2419/4000 [2:50:01<1:52:31,  4.27s/it]
 60%|██████    | 2420/4000 [2:50:05<1:51:25,  4.23s/it]
                                                       
{'loss': '0.4425', 'grad_norm': '0.1242', 'learning_rate': '0.0002395', 'epoch': '0.5864'}

 60%|██████    | 2420/4000 [2:50:05<1:51:25,  4.23s/it]
 61%|██████    | 2421/4000 [2:50:09<1:53:25,  4.31s/it]
 61%|██████    | 2422/4000 [2:50:13<1:52:06,  4.26s/it]
 61%|██████    | 2423/4000 [2:50:18<1:51:00,  4.22s/it]
 61%|██████    | 2424/4000 [2:50:22<1:50:52,  4.22s/it]
 61%|██████    | 2425/4000 [2:50:26<1:52:16,  4.28s/it]
 61%|██████    | 2426/4000 [2:50:30<1:52:29,  4.29s/it]
 61%|██████    | 2427/4000 [2:50:35<1:53:50,  4.34s/it]
 61%|██████    | 2428/4000 [2:50:39<1:53:05,  4.32s/it]
 61%|██████    | 2429/4000 [2:50:43<1:51:28,  4.26s/it]
 61%|██████    | 2430/4000 [2:50:48<1:52:24,  4.30s/it]
                                                       
{'loss': '0.4242', 'grad_norm': '0.1184', 'learning_rate': '0.000238', 'epoch': '0.5888'}

 61%|██████    | 2430/4000 [2:50:48<1:52:24,  4.30s/it]
 61%|██████    | 2431/4000 [2:50:52<1:51:03,  4.25s/it]
 61%|██████    | 2432/4000 [2:50:56<1:53:09,  4.33s/it]
 61%|██████    | 2433/4000 [2:51:00<1:51:29,  4.27s/it]
 61%|██████    | 2434/4000 [2:51:05<1:50:18,  4.23s/it]
 61%|██████    | 2435/4000 [2:51:09<1:49:30,  4.20s/it]
 61%|██████    | 2436/4000 [2:51:13<1:48:50,  4.18s/it]
 61%|██████    | 2437/4000 [2:51:17<1:48:20,  4.16s/it]
 61%|██████    | 2438/4000 [2:51:21<1:50:03,  4.23s/it]
 61%|██████    | 2439/4000 [2:51:26<1:52:16,  4.32s/it]
 61%|██████    | 2440/4000 [2:51:30<1:50:47,  4.26s/it]
                                                       
{'loss': '0.3936', 'grad_norm': '0.1247', 'learning_rate': '0.0002365', 'epoch': '0.5912'}

 61%|██████    | 2440/4000 [2:51:30<1:50:47,  4.26s/it]
 61%|██████    | 2441/4000 [2:51:34<1:50:02,  4.24s/it]
 61%|██████    | 2442/4000 [2:51:38<1:49:45,  4.23s/it]
 61%|██████    | 2443/4000 [2:51:43<1:49:25,  4.22s/it]
 61%|██████    | 2444/4000 [2:51:47<1:49:40,  4.23s/it]
 61%|██████    | 2445/4000 [2:51:51<1:50:47,  4.28s/it]
 61%|██████    | 2446/4000 [2:51:56<1:52:22,  4.34s/it]
 61%|██████    | 2447/4000 [2:52:00<1:52:48,  4.36s/it]
 61%|██████    | 2448/4000 [2:52:04<1:51:06,  4.30s/it]
 61%|██████    | 2449/4000 [2:52:08<1:49:49,  4.25s/it]
 61%|██████▏   | 2450/4000 [2:52:13<1:48:49,  4.21s/it]
                                                       
{'loss': '0.4206', 'grad_norm': '0.117', 'learning_rate': '0.000235', 'epoch': '0.5936'}

 61%|██████▏   | 2450/4000 [2:52:13<1:48:49,  4.21s/it]
 61%|██████▏   | 2451/4000 [2:52:17<1:48:07,  4.19s/it]
 61%|██████▏   | 2452/4000 [2:52:21<1:47:41,  4.17s/it]
 61%|██████▏   | 2453/4000 [2:52:25<1:50:24,  4.28s/it]
 61%|██████▏   | 2454/4000 [2:52:30<1:50:43,  4.30s/it]
 61%|██████▏   | 2455/4000 [2:52:34<1:49:38,  4.26s/it]
 61%|██████▏   | 2456/4000 [2:52:38<1:48:37,  4.22s/it]
 61%|██████▏   | 2457/4000 [2:52:42<1:47:55,  4.20s/it]
 61%|██████▏   | 2458/4000 [2:52:46<1:48:26,  4.22s/it]
 61%|██████▏   | 2459/4000 [2:52:51<1:51:03,  4.32s/it]
 62%|██████▏   | 2460/4000 [2:52:55<1:49:57,  4.28s/it]
                                                       
{'loss': '0.4193', 'grad_norm': '0.1188', 'learning_rate': '0.0002335', 'epoch': '0.5961'}

 62%|██████▏   | 2460/4000 [2:52:55<1:49:57,  4.28s/it]
 62%|██████▏   | 2461/4000 [2:52:59<1:48:43,  4.24s/it]
 62%|██████▏   | 2462/4000 [2:53:04<1:51:30,  4.35s/it]
 62%|██████▏   | 2463/4000 [2:53:08<1:49:59,  4.29s/it]
 62%|██████▏   | 2464/4000 [2:53:13<1:52:08,  4.38s/it]
 62%|██████▏   | 2465/4000 [2:53:17<1:50:10,  4.31s/it]
 62%|██████▏   | 2466/4000 [2:53:21<1:51:44,  4.37s/it]
 62%|██████▏   | 2467/4000 [2:53:25<1:49:51,  4.30s/it]
 62%|██████▏   | 2468/4000 [2:53:30<1:48:30,  4.25s/it]
 62%|██████▏   | 2469/4000 [2:53:34<1:47:28,  4.21s/it]
 62%|██████▏   | 2470/4000 [2:53:38<1:46:48,  4.19s/it]
                                                       
{'loss': '0.4233', 'grad_norm': '0.1265', 'learning_rate': '0.000232', 'epoch': '0.5985'}

 62%|██████▏   | 2470/4000 [2:53:38<1:46:48,  4.19s/it]
 62%|██████▏   | 2471/4000 [2:53:42<1:47:20,  4.21s/it]
 62%|██████▏   | 2472/4000 [2:53:46<1:46:41,  4.19s/it]
 62%|██████▏   | 2473/4000 [2:53:51<1:49:12,  4.29s/it]
 62%|██████▏   | 2474/4000 [2:53:55<1:47:53,  4.24s/it]
 62%|██████▏   | 2475/4000 [2:53:59<1:48:20,  4.26s/it]
 62%|██████▏   | 2476/4000 [2:54:03<1:48:01,  4.25s/it]
 62%|██████▏   | 2477/4000 [2:54:08<1:47:24,  4.23s/it]
 62%|██████▏   | 2478/4000 [2:54:12<1:46:30,  4.20s/it]
 62%|██████▏   | 2479/4000 [2:54:17<1:51:39,  4.40s/it]
 62%|██████▏   | 2480/4000 [2:54:21<1:52:25,  4.44s/it]
                                                       
{'loss': '0.429', 'grad_norm': '0.1131', 'learning_rate': '0.0002305', 'epoch': '0.6009'}

 62%|██████▏   | 2480/4000 [2:54:21<1:52:25,  4.44s/it]
 62%|██████▏   | 2481/4000 [2:54:25<1:50:26,  4.36s/it]
 62%|██████▏   | 2482/4000 [2:54:29<1:48:41,  4.30s/it]
 62%|██████▏   | 2483/4000 [2:54:34<1:47:24,  4.25s/it]
 62%|██████▏   | 2484/4000 [2:54:38<1:46:31,  4.22s/it]
 62%|██████▏   | 2485/4000 [2:54:42<1:45:52,  4.19s/it]
 62%|██████▏   | 2486/4000 [2:54:46<1:48:26,  4.30s/it]
 62%|██████▏   | 2487/4000 [2:54:51<1:47:06,  4.25s/it]
 62%|██████▏   | 2488/4000 [2:54:55<1:46:58,  4.25s/it]
 62%|██████▏   | 2489/4000 [2:54:59<1:46:07,  4.21s/it]
 62%|██████▏   | 2490/4000 [2:55:03<1:45:30,  4.19s/it]
                                                       
{'loss': '0.4325', 'grad_norm': '0.1439', 'learning_rate': '0.0002289', 'epoch': '0.6033'}

 62%|██████▏   | 2490/4000 [2:55:03<1:45:30,  4.19s/it]
 62%|██████▏   | 2491/4000 [2:55:07<1:44:59,  4.17s/it]
 62%|██████▏   | 2492/4000 [2:55:11<1:45:22,  4.19s/it]
 62%|██████▏   | 2493/4000 [2:55:16<1:49:33,  4.36s/it]
 62%|██████▏   | 2494/4000 [2:55:20<1:47:50,  4.30s/it]
 62%|██████▏   | 2495/4000 [2:55:24<1:46:36,  4.25s/it]
 62%|██████▏   | 2496/4000 [2:55:29<1:46:47,  4.26s/it]
 62%|██████▏   | 2497/4000 [2:55:33<1:48:39,  4.34s/it]
 62%|██████▏   | 2498/4000 [2:55:37<1:47:03,  4.28s/it]
 62%|██████▏   | 2499/4000 [2:55:42<1:45:47,  4.23s/it]
 62%|██████▎   | 2500/4000 [2:55:46<1:47:57,  4.32s/it]
                                                       
{'loss': '0.4406', 'grad_norm': '0.1617', 'learning_rate': '0.0002274', 'epoch': '0.6057'}

 62%|██████▎   | 2500/4000 [2:55:46<1:47:57,  4.32s/it]
 63%|██████▎   | 2501/4000 [2:55:50<1:46:30,  4.26s/it]
 63%|██████▎   | 2502/4000 [2:55:54<1:45:27,  4.22s/it]
 63%|██████▎   | 2503/4000 [2:55:58<1:44:43,  4.20s/it]
 63%|██████▎   | 2504/4000 [2:56:03<1:44:23,  4.19s/it]
 63%|██████▎   | 2505/4000 [2:56:07<1:44:50,  4.21s/it]
 63%|██████▎   | 2506/4000 [2:56:11<1:45:29,  4.24s/it]
 63%|██████▎   | 2507/4000 [2:56:16<1:46:05,  4.26s/it]
 63%|██████▎   | 2508/4000 [2:56:20<1:45:10,  4.23s/it]
 63%|██████▎   | 2509/4000 [2:56:24<1:45:34,  4.25s/it]
 63%|██████▎   | 2510/4000 [2:56:28<1:45:25,  4.25s/it]
                                                       
{'loss': '0.4191', 'grad_norm': '0.1201', 'learning_rate': '0.0002259', 'epoch': '0.6082'}

 63%|██████▎   | 2510/4000 [2:56:28<1:45:25,  4.25s/it]
 63%|██████▎   | 2511/4000 [2:56:32<1:44:47,  4.22s/it]
 63%|██████▎   | 2512/4000 [2:56:37<1:44:07,  4.20s/it]
 63%|██████▎   | 2513/4000 [2:56:41<1:47:24,  4.33s/it]
 63%|██████▎   | 2514/4000 [2:56:45<1:46:45,  4.31s/it]
 63%|██████▎   | 2515/4000 [2:56:50<1:46:30,  4.30s/it]
 63%|██████▎   | 2516/4000 [2:56:54<1:45:13,  4.25s/it]
 63%|██████▎   | 2517/4000 [2:56:58<1:44:18,  4.22s/it]
 63%|██████▎   | 2518/4000 [2:57:02<1:43:40,  4.20s/it]
 63%|██████▎   | 2519/4000 [2:57:06<1:43:17,  4.18s/it]
 63%|██████▎   | 2520/4000 [2:57:11<1:46:15,  4.31s/it]
                                                       
{'loss': '0.4117', 'grad_norm': '0.1111', 'learning_rate': '0.0002244', 'epoch': '0.6106'}

 63%|██████▎   | 2520/4000 [2:57:11<1:46:15,  4.31s/it]
 63%|██████▎   | 2521/4000 [2:57:15<1:45:54,  4.30s/it]
 63%|██████▎   | 2522/4000 [2:57:19<1:44:48,  4.25s/it]
 63%|██████▎   | 2523/4000 [2:57:23<1:43:56,  4.22s/it]
 63%|██████▎   | 2524/4000 [2:57:28<1:43:16,  4.20s/it]
 63%|██████▎   | 2525/4000 [2:57:32<1:42:46,  4.18s/it]
 63%|██████▎   | 2526/4000 [2:57:36<1:43:57,  4.23s/it]
 63%|██████▎   | 2527/4000 [2:57:41<1:46:40,  4.35s/it]
 63%|██████▎   | 2528/4000 [2:57:45<1:45:12,  4.29s/it]
 63%|██████▎   | 2529/4000 [2:57:49<1:44:06,  4.25s/it]
 63%|██████▎   | 2530/4000 [2:57:53<1:44:24,  4.26s/it]
                                                       
{'loss': '0.4041', 'grad_norm': '0.1393', 'learning_rate': '0.0002229', 'epoch': '0.613'}

 63%|██████▎   | 2530/4000 [2:57:53<1:44:24,  4.26s/it]
 63%|██████▎   | 2531/4000 [2:57:58<1:44:38,  4.27s/it]
 63%|██████▎   | 2532/4000 [2:58:02<1:46:55,  4.37s/it]
 63%|██████▎   | 2533/4000 [2:58:07<1:50:52,  4.53s/it]
 63%|██████▎   | 2534/4000 [2:58:11<1:48:02,  4.42s/it]
 63%|██████▎   | 2535/4000 [2:58:15<1:45:53,  4.34s/it]
 63%|██████▎   | 2536/4000 [2:58:20<1:44:27,  4.28s/it]
 63%|██████▎   | 2537/4000 [2:58:24<1:43:19,  4.24s/it]
 63%|██████▎   | 2538/4000 [2:58:28<1:43:57,  4.27s/it]
 63%|██████▎   | 2539/4000 [2:58:32<1:42:58,  4.23s/it]
 64%|██████▎   | 2540/4000 [2:58:37<1:45:09,  4.32s/it]
                                                       
{'loss': '0.4366', 'grad_norm': '0.1341', 'learning_rate': '0.0002214', 'epoch': '0.6154'}

 64%|██████▎   | 2540/4000 [2:58:37<1:45:09,  4.32s/it]
 64%|██████▎   | 2541/4000 [2:58:41<1:43:42,  4.26s/it]
 64%|██████▎   | 2542/4000 [2:58:45<1:43:55,  4.28s/it]
 64%|██████▎   | 2543/4000 [2:58:49<1:43:19,  4.25s/it]
 64%|██████▎   | 2544/4000 [2:58:54<1:44:31,  4.31s/it]
 64%|██████▎   | 2545/4000 [2:58:58<1:43:27,  4.27s/it]
 64%|██████▎   | 2546/4000 [2:59:02<1:42:26,  4.23s/it]
 64%|██████▎   | 2547/4000 [2:59:07<1:45:54,  4.37s/it]
 64%|██████▎   | 2548/4000 [2:59:11<1:45:45,  4.37s/it]
 64%|██████▎   | 2549/4000 [2:59:15<1:45:03,  4.34s/it]
 64%|██████▍   | 2550/4000 [2:59:20<1:43:30,  4.28s/it]
                                                       
{'loss': '0.4377', 'grad_norm': '0.1391', 'learning_rate': '0.0002198', 'epoch': '0.6179'}

 64%|██████▍   | 2550/4000 [2:59:20<1:43:30,  4.28s/it]
 64%|██████▍   | 2551/4000 [2:59:24<1:42:21,  4.24s/it]
 64%|██████▍   | 2552/4000 [2:59:28<1:41:28,  4.20s/it]
 64%|██████▍   | 2553/4000 [2:59:32<1:40:55,  4.18s/it]
 64%|██████▍   | 2554/4000 [2:59:37<1:43:50,  4.31s/it]
 64%|██████▍   | 2555/4000 [2:59:41<1:43:09,  4.28s/it]
 64%|██████▍   | 2556/4000 [2:59:45<1:42:00,  4.24s/it]
 64%|██████▍   | 2557/4000 [2:59:49<1:41:07,  4.20s/it]
 64%|██████▍   | 2558/4000 [2:59:53<1:40:31,  4.18s/it]
 64%|██████▍   | 2559/4000 [2:59:57<1:40:42,  4.19s/it]
 64%|██████▍   | 2560/4000 [3:00:02<1:42:58,  4.29s/it]
                                                       
{'loss': '0.3938', 'grad_norm': '0.1115', 'learning_rate': '0.0002183', 'epoch': '0.6203'}

 64%|██████▍   | 2560/4000 [3:00:02<1:42:58,  4.29s/it]
 64%|██████▍   | 2561/4000 [3:00:06<1:43:31,  4.32s/it]
 64%|██████▍   | 2562/4000 [3:00:10<1:42:09,  4.26s/it]
 64%|██████▍   | 2563/4000 [3:00:15<1:41:08,  4.22s/it]
 64%|██████▍   | 2564/4000 [3:00:19<1:42:10,  4.27s/it]
 64%|██████▍   | 2565/4000 [3:00:23<1:42:16,  4.28s/it]
 64%|██████▍   | 2566/4000 [3:00:27<1:41:24,  4.24s/it]
 64%|██████▍   | 2567/4000 [3:00:32<1:45:10,  4.40s/it]
 64%|██████▍   | 2568/4000 [3:00:36<1:43:39,  4.34s/it]
 64%|██████▍   | 2569/4000 [3:00:41<1:42:01,  4.28s/it]
 64%|██████▍   | 2570/4000 [3:00:45<1:40:53,  4.23s/it]
                                                       
{'loss': '0.3954', 'grad_norm': '0.1285', 'learning_rate': '0.0002168', 'epoch': '0.6227'}

 64%|██████▍   | 2570/4000 [3:00:45<1:40:53,  4.23s/it]
 64%|██████▍   | 2571/4000 [3:00:49<1:41:10,  4.25s/it]
 64%|██████▍   | 2572/4000 [3:00:53<1:40:17,  4.21s/it]
 64%|██████▍   | 2573/4000 [3:00:57<1:39:36,  4.19s/it]
 64%|██████▍   | 2574/4000 [3:01:02<1:41:48,  4.28s/it]
 64%|██████▍   | 2575/4000 [3:01:06<1:40:34,  4.23s/it]
 64%|██████▍   | 2576/4000 [3:01:10<1:41:39,  4.28s/it]
 64%|██████▍   | 2577/4000 [3:01:14<1:40:43,  4.25s/it]
 64%|██████▍   | 2578/4000 [3:01:19<1:40:31,  4.24s/it]
 64%|██████▍   | 2579/4000 [3:01:23<1:39:46,  4.21s/it]
 64%|██████▍   | 2580/4000 [3:01:27<1:40:28,  4.25s/it]
                                                       
{'loss': '0.4072', 'grad_norm': '0.1221', 'learning_rate': '0.0002153', 'epoch': '0.6251'}

 64%|██████▍   | 2580/4000 [3:01:27<1:40:28,  4.25s/it]
 65%|██████▍   | 2581/4000 [3:01:32<1:42:35,  4.34s/it]
 65%|██████▍   | 2582/4000 [3:01:36<1:41:55,  4.31s/it]
 65%|██████▍   | 2583/4000 [3:01:40<1:40:40,  4.26s/it]
 65%|██████▍   | 2584/4000 [3:01:44<1:40:22,  4.25s/it]
 65%|██████▍   | 2585/4000 [3:01:48<1:39:34,  4.22s/it]
 65%|██████▍   | 2586/4000 [3:01:53<1:38:49,  4.19s/it]
 65%|██████▍   | 2587/4000 [3:01:57<1:40:58,  4.29s/it]
 65%|██████▍   | 2588/4000 [3:02:01<1:41:18,  4.31s/it]
 65%|██████▍   | 2589/4000 [3:02:06<1:40:11,  4.26s/it]
 65%|██████▍   | 2590/4000 [3:02:10<1:39:23,  4.23s/it]
                                                       
{'loss': '0.3849', 'grad_norm': '0.1156', 'learning_rate': '0.0002138', 'epoch': '0.6276'}

 65%|██████▍   | 2590/4000 [3:02:10<1:39:23,  4.23s/it]
 65%|██████▍   | 2591/4000 [3:02:14<1:38:40,  4.20s/it]
 65%|██████▍   | 2592/4000 [3:02:18<1:38:13,  4.19s/it]
 65%|██████▍   | 2593/4000 [3:02:22<1:38:26,  4.20s/it]
 65%|██████▍   | 2594/4000 [3:02:27<1:40:37,  4.29s/it]
 65%|██████▍   | 2595/4000 [3:02:31<1:41:11,  4.32s/it]
 65%|██████▍   | 2596/4000 [3:02:35<1:39:52,  4.27s/it]
 65%|██████▍   | 2597/4000 [3:02:39<1:38:56,  4.23s/it]
 65%|██████▍   | 2598/4000 [3:02:44<1:39:18,  4.25s/it]
 65%|██████▍   | 2599/4000 [3:02:48<1:39:57,  4.28s/it]
 65%|██████▌   | 2600/4000 [3:02:52<1:38:56,  4.24s/it]
                                                       
{'loss': '0.4373', 'grad_norm': '0.1099', 'learning_rate': '0.0002123', 'epoch': '0.63'}

 65%|██████▌   | 2600/4000 [3:02:52<1:38:56,  4.24s/it]
 65%|██████▌   | 2601/4000 [3:02:57<1:42:09,  4.38s/it]
 65%|██████▌   | 2602/4000 [3:03:01<1:40:28,  4.31s/it]
 65%|██████▌   | 2603/4000 [3:03:05<1:39:10,  4.26s/it]
 65%|██████▌   | 2604/4000 [3:03:09<1:39:17,  4.27s/it]
 65%|██████▌   | 2605/4000 [3:03:14<1:38:25,  4.23s/it]
 65%|██████▌   | 2606/4000 [3:03:18<1:37:39,  4.20s/it]
 65%|██████▌   | 2607/4000 [3:03:22<1:38:30,  4.24s/it]
 65%|██████▌   | 2608/4000 [3:03:26<1:38:43,  4.26s/it]
 65%|██████▌   | 2609/4000 [3:03:31<1:38:11,  4.24s/it]
 65%|██████▌   | 2610/4000 [3:03:35<1:37:49,  4.22s/it]
                                                       
{'loss': '0.4537', 'grad_norm': '0.1235', 'learning_rate': '0.0002108', 'epoch': '0.6324'}

 65%|██████▌   | 2610/4000 [3:03:35<1:37:49,  4.22s/it]
 65%|██████▌   | 2611/4000 [3:03:39<1:37:15,  4.20s/it]
 65%|██████▌   | 2612/4000 [3:03:43<1:37:30,  4.22s/it]
 65%|██████▌   | 2613/4000 [3:03:47<1:36:57,  4.19s/it]
 65%|██████▌   | 2614/4000 [3:03:52<1:38:25,  4.26s/it]
 65%|██████▌   | 2615/4000 [3:03:56<1:39:04,  4.29s/it]
 65%|██████▌   | 2616/4000 [3:04:00<1:38:51,  4.29s/it]
 65%|██████▌   | 2617/4000 [3:04:05<1:37:55,  4.25s/it]
 65%|██████▌   | 2618/4000 [3:04:09<1:37:21,  4.23s/it]
 65%|██████▌   | 2619/4000 [3:04:13<1:38:52,  4.30s/it]
 66%|██████▌   | 2620/4000 [3:04:17<1:37:45,  4.25s/it]
                                                       
{'loss': '0.4175', 'grad_norm': '0.112', 'learning_rate': '0.0002092', 'epoch': '0.6348'}

 66%|██████▌   | 2620/4000 [3:04:17<1:37:45,  4.25s/it]
 66%|██████▌   | 2621/4000 [3:04:22<1:40:50,  4.39s/it]
 66%|██████▌   | 2622/4000 [3:04:26<1:39:04,  4.31s/it]
 66%|██████▌   | 2623/4000 [3:04:30<1:37:48,  4.26s/it]
 66%|██████▌   | 2624/4000 [3:04:34<1:36:53,  4.22s/it]
 66%|██████▌   | 2625/4000 [3:04:39<1:36:14,  4.20s/it]
 66%|██████▌   | 2626/4000 [3:04:43<1:36:15,  4.20s/it]
 66%|██████▌   | 2627/4000 [3:04:47<1:36:05,  4.20s/it]
 66%|██████▌   | 2628/4000 [3:04:52<1:39:13,  4.34s/it]
 66%|██████▌   | 2629/4000 [3:04:56<1:37:48,  4.28s/it]
 66%|██████▌   | 2630/4000 [3:05:00<1:36:46,  4.24s/it]
                                                       
{'loss': '0.4294', 'grad_norm': '0.1219', 'learning_rate': '0.0002077', 'epoch': '0.6372'}

 66%|██████▌   | 2630/4000 [3:05:00<1:36:46,  4.24s/it]
 66%|██████▌   | 2631/4000 [3:05:04<1:36:05,  4.21s/it]
 66%|██████▌   | 2632/4000 [3:05:08<1:37:02,  4.26s/it]
 66%|██████▌   | 2633/4000 [3:05:13<1:36:53,  4.25s/it]
 66%|██████▌   | 2634/4000 [3:05:17<1:37:30,  4.28s/it]
 66%|██████▌   | 2635/4000 [3:05:21<1:37:42,  4.29s/it]
 66%|██████▌   | 2636/4000 [3:05:26<1:38:52,  4.35s/it]
 66%|██████▌   | 2637/4000 [3:05:30<1:37:28,  4.29s/it]
 66%|██████▌   | 2638/4000 [3:05:34<1:37:24,  4.29s/it]
 66%|██████▌   | 2639/4000 [3:05:38<1:36:15,  4.24s/it]
 66%|██████▌   | 2640/4000 [3:05:43<1:35:22,  4.21s/it]
                                                       
{'loss': '0.4103', 'grad_norm': '0.1417', 'learning_rate': '0.0002062', 'epoch': '0.6397'}

 66%|██████▌   | 2640/4000 [3:05:43<1:35:22,  4.21s/it]
 66%|██████▌   | 2641/4000 [3:05:47<1:37:07,  4.29s/it]
 66%|██████▌   | 2642/4000 [3:05:51<1:35:59,  4.24s/it]
 66%|██████▌   | 2643/4000 [3:05:56<1:37:02,  4.29s/it]
 66%|██████▌   | 2644/4000 [3:06:00<1:36:10,  4.26s/it]
 66%|██████▌   | 2645/4000 [3:06:04<1:36:26,  4.27s/it]
 66%|██████▌   | 2646/4000 [3:06:08<1:35:29,  4.23s/it]
 66%|██████▌   | 2647/4000 [3:06:12<1:34:47,  4.20s/it]
 66%|██████▌   | 2648/4000 [3:06:17<1:36:25,  4.28s/it]
 66%|██████▌   | 2649/4000 [3:06:21<1:36:07,  4.27s/it]
 66%|██████▋   | 2650/4000 [3:06:25<1:36:27,  4.29s/it]
                                                       
{'loss': '0.4134', 'grad_norm': '0.1227', 'learning_rate': '0.0002047', 'epoch': '0.6421'}

 66%|██████▋   | 2650/4000 [3:06:25<1:36:27,  4.29s/it]
 66%|██████▋   | 2651/4000 [3:06:29<1:35:25,  4.24s/it]
 66%|██████▋   | 2652/4000 [3:06:34<1:34:38,  4.21s/it]
 66%|██████▋   | 2653/4000 [3:06:38<1:36:21,  4.29s/it]
 66%|██████▋   | 2654/4000 [3:06:42<1:35:23,  4.25s/it]
 66%|██████▋   | 2655/4000 [3:06:47<1:38:17,  4.39s/it]
 66%|██████▋   | 2656/4000 [3:06:51<1:36:32,  4.31s/it]
 66%|██████▋   | 2657/4000 [3:06:55<1:35:17,  4.26s/it]
 66%|██████▋   | 2658/4000 [3:06:59<1:34:20,  4.22s/it]
 66%|██████▋   | 2659/4000 [3:07:03<1:33:42,  4.19s/it]
 66%|██████▋   | 2660/4000 [3:07:08<1:34:13,  4.22s/it]
                                                       
{'loss': '0.4317', 'grad_norm': '0.1114', 'learning_rate': '0.0002032', 'epoch': '0.6445'}

 66%|██████▋   | 2660/4000 [3:07:08<1:34:13,  4.22s/it]
 67%|██████▋   | 2661/4000 [3:07:12<1:37:05,  4.35s/it]
 67%|██████▋   | 2662/4000 [3:07:17<1:35:47,  4.30s/it]
 67%|██████▋   | 2663/4000 [3:07:21<1:34:36,  4.25s/it]
 67%|██████▋   | 2664/4000 [3:07:25<1:33:44,  4.21s/it]
 67%|██████▋   | 2665/4000 [3:07:29<1:33:10,  4.19s/it]
 67%|██████▋   | 2666/4000 [3:07:33<1:32:48,  4.17s/it]
 67%|██████▋   | 2667/4000 [3:07:37<1:33:35,  4.21s/it]
 67%|██████▋   | 2668/4000 [3:07:42<1:35:29,  4.30s/it]
 67%|██████▋   | 2669/4000 [3:07:46<1:34:16,  4.25s/it]
 67%|██████▋   | 2670/4000 [3:07:50<1:34:11,  4.25s/it]
                                                       
{'loss': '0.4512', 'grad_norm': '0.1369', 'learning_rate': '0.0002017', 'epoch': '0.6469'}

 67%|██████▋   | 2670/4000 [3:07:50<1:34:11,  4.25s/it]
 67%|██████▋   | 2671/4000 [3:07:55<1:33:51,  4.24s/it]
 67%|██████▋   | 2672/4000 [3:07:59<1:34:02,  4.25s/it]
 67%|██████▋   | 2673/4000 [3:08:03<1:33:11,  4.21s/it]
 67%|██████▋   | 2674/4000 [3:08:07<1:32:28,  4.18s/it]
 67%|██████▋   | 2675/4000 [3:08:12<1:34:23,  4.27s/it]
 67%|██████▋   | 2676/4000 [3:08:16<1:33:32,  4.24s/it]
 67%|██████▋   | 2677/4000 [3:08:20<1:33:32,  4.24s/it]
 67%|██████▋   | 2678/4000 [3:08:24<1:34:04,  4.27s/it]
 67%|██████▋   | 2679/4000 [3:08:28<1:33:14,  4.23s/it]
 67%|██████▋   | 2680/4000 [3:08:33<1:32:27,  4.20s/it]
                                                       
{'loss': '0.4546', 'grad_norm': '0.1234', 'learning_rate': '0.0002002', 'epoch': '0.6494'}

 67%|██████▋   | 2680/4000 [3:08:33<1:32:27,  4.20s/it]
 67%|██████▋   | 2681/4000 [3:08:37<1:31:57,  4.18s/it]
 67%|██████▋   | 2682/4000 [3:08:41<1:33:39,  4.26s/it]
 67%|██████▋   | 2683/4000 [3:08:45<1:32:38,  4.22s/it]
 67%|██████▋   | 2684/4000 [3:08:50<1:33:16,  4.25s/it]
 67%|██████▋   | 2685/4000 [3:08:54<1:32:32,  4.22s/it]
 67%|██████▋   | 2686/4000 [3:08:58<1:31:50,  4.19s/it]
 67%|██████▋   | 2687/4000 [3:09:02<1:31:19,  4.17s/it]
 67%|██████▋   | 2688/4000 [3:09:07<1:33:58,  4.30s/it]
 67%|██████▋   | 2689/4000 [3:09:11<1:34:22,  4.32s/it]
 67%|██████▋   | 2690/4000 [3:09:15<1:33:09,  4.27s/it]
                                                       
{'loss': '0.4204', 'grad_norm': '0.1305', 'learning_rate': '0.0001986', 'epoch': '0.6518'}

 67%|██████▋   | 2690/4000 [3:09:15<1:33:09,  4.27s/it]
 67%|██████▋   | 2691/4000 [3:09:19<1:32:07,  4.22s/it]
 67%|██████▋   | 2692/4000 [3:09:23<1:31:26,  4.19s/it]
 67%|██████▋   | 2693/4000 [3:09:28<1:32:03,  4.23s/it]
 67%|██████▋   | 2694/4000 [3:09:32<1:31:25,  4.20s/it]
 67%|██████▋   | 2695/4000 [3:09:36<1:34:17,  4.34s/it]
 67%|██████▋   | 2696/4000 [3:09:41<1:32:53,  4.27s/it]
 67%|██████▋   | 2697/4000 [3:09:45<1:31:51,  4.23s/it]
 67%|██████▋   | 2698/4000 [3:09:49<1:31:08,  4.20s/it]
 67%|██████▋   | 2699/4000 [3:09:53<1:30:36,  4.18s/it]
 68%|██████▊   | 2700/4000 [3:09:57<1:30:14,  4.16s/it]
                                                       
{'loss': '0.4062', 'grad_norm': '0.1601', 'learning_rate': '0.0001971', 'epoch': '0.6542'}

 68%|██████▊   | 2700/4000 [3:09:57<1:30:14,  4.16s/it]
 68%|██████▊   | 2701/4000 [3:10:01<1:30:36,  4.18s/it]
 68%|██████▊   | 2702/4000 [3:10:06<1:33:30,  4.32s/it]
 68%|██████▊   | 2703/4000 [3:10:10<1:32:10,  4.26s/it]
 68%|██████▊   | 2704/4000 [3:10:14<1:31:12,  4.22s/it]
 68%|██████▊   | 2705/4000 [3:10:19<1:31:47,  4.25s/it]
 68%|██████▊   | 2706/4000 [3:10:23<1:33:23,  4.33s/it]
 68%|██████▊   | 2707/4000 [3:10:27<1:32:09,  4.28s/it]
 68%|██████▊   | 2708/4000 [3:10:32<1:32:24,  4.29s/it]
 68%|██████▊   | 2709/4000 [3:10:36<1:32:15,  4.29s/it]
 68%|██████▊   | 2710/4000 [3:10:40<1:32:29,  4.30s/it]
                                                       
{'loss': '0.4095', 'grad_norm': '0.121', 'learning_rate': '0.0001956', 'epoch': '0.6566'}

 68%|██████▊   | 2710/4000 [3:10:40<1:32:29,  4.30s/it]
 68%|██████▊   | 2711/4000 [3:10:44<1:32:10,  4.29s/it]
 68%|██████▊   | 2712/4000 [3:10:49<1:31:05,  4.24s/it]
 68%|██████▊   | 2713/4000 [3:10:53<1:30:17,  4.21s/it]
 68%|██████▊   | 2714/4000 [3:10:57<1:29:50,  4.19s/it]
 68%|██████▊   | 2715/4000 [3:11:01<1:31:23,  4.27s/it]
 68%|██████▊   | 2716/4000 [3:11:05<1:30:28,  4.23s/it]
 68%|██████▊   | 2717/4000 [3:11:10<1:29:47,  4.20s/it]
 68%|██████▊   | 2718/4000 [3:11:14<1:30:01,  4.21s/it]
 68%|██████▊   | 2719/4000 [3:11:18<1:30:28,  4.24s/it]
 68%|██████▊   | 2720/4000 [3:11:22<1:29:42,  4.20s/it]
                                                       
{'loss': '0.4102', 'grad_norm': '0.1308', 'learning_rate': '0.0001941', 'epoch': '0.6591'}

 68%|██████▊   | 2720/4000 [3:11:22<1:29:42,  4.20s/it]
 68%|██████▊   | 2721/4000 [3:11:26<1:29:06,  4.18s/it]
 68%|██████▊   | 2722/4000 [3:11:31<1:32:08,  4.33s/it]
 68%|██████▊   | 2723/4000 [3:11:35<1:32:38,  4.35s/it]
 68%|██████▊   | 2724/4000 [3:11:40<1:31:07,  4.28s/it]
 68%|██████▊   | 2725/4000 [3:11:44<1:30:04,  4.24s/it]
 68%|██████▊   | 2726/4000 [3:11:48<1:29:15,  4.20s/it]
 68%|██████▊   | 2727/4000 [3:11:52<1:29:54,  4.24s/it]
 68%|██████▊   | 2728/4000 [3:11:56<1:30:29,  4.27s/it]
 68%|██████▊   | 2729/4000 [3:12:01<1:31:53,  4.34s/it]
 68%|██████▊   | 2730/4000 [3:12:05<1:30:30,  4.28s/it]
                                                       
{'loss': '0.4362', 'grad_norm': '0.1407', 'learning_rate': '0.0001926', 'epoch': '0.6615'}

 68%|██████▊   | 2730/4000 [3:12:05<1:30:30,  4.28s/it]
 68%|██████▊   | 2731/4000 [3:12:09<1:29:29,  4.23s/it]
 68%|██████▊   | 2732/4000 [3:12:13<1:28:50,  4.20s/it]
 68%|██████▊   | 2733/4000 [3:12:17<1:28:20,  4.18s/it]
 68%|██████▊   | 2734/4000 [3:12:22<1:28:02,  4.17s/it]
 68%|██████▊   | 2735/4000 [3:12:26<1:29:52,  4.26s/it]
 68%|██████▊   | 2736/4000 [3:12:31<1:31:09,  4.33s/it]
 68%|██████▊   | 2737/4000 [3:12:35<1:29:48,  4.27s/it]
 68%|██████▊   | 2738/4000 [3:12:39<1:28:56,  4.23s/it]
 68%|██████▊   | 2739/4000 [3:12:43<1:29:35,  4.26s/it]
 68%|██████▊   | 2740/4000 [3:12:48<1:30:01,  4.29s/it]
                                                       
{'loss': '0.4305', 'grad_norm': '0.1318', 'learning_rate': '0.0001911', 'epoch': '0.6639'}

 68%|██████▊   | 2740/4000 [3:12:48<1:30:01,  4.29s/it]
 69%|██████▊   | 2741/4000 [3:12:52<1:28:59,  4.24s/it]
 69%|██████▊   | 2742/4000 [3:12:56<1:30:24,  4.31s/it]
 69%|██████▊   | 2743/4000 [3:13:00<1:29:12,  4.26s/it]
 69%|██████▊   | 2744/4000 [3:13:05<1:29:44,  4.29s/it]
 69%|██████▊   | 2745/4000 [3:13:09<1:29:05,  4.26s/it]
 69%|██████▊   | 2746/4000 [3:13:13<1:28:14,  4.22s/it]
 69%|██████▊   | 2747/4000 [3:13:17<1:27:31,  4.19s/it]
 69%|██████▊   | 2748/4000 [3:13:21<1:27:05,  4.17s/it]
 69%|██████▊   | 2749/4000 [3:13:26<1:28:44,  4.26s/it]
 69%|██████▉   | 2750/4000 [3:13:30<1:27:51,  4.22s/it]
                                                       
{'loss': '0.4596', 'grad_norm': '0.127', 'learning_rate': '0.0001895', 'epoch': '0.6663'}

 69%|██████▉   | 2750/4000 [3:13:30<1:27:51,  4.22s/it]
 69%|██████▉   | 2751/4000 [3:13:34<1:27:17,  4.19s/it]
 69%|██████▉   | 2752/4000 [3:13:38<1:27:21,  4.20s/it]
 69%|██████▉   | 2753/4000 [3:13:42<1:27:40,  4.22s/it]
 69%|██████▉   | 2754/4000 [3:13:47<1:27:11,  4.20s/it]
 69%|██████▉   | 2755/4000 [3:13:51<1:26:38,  4.18s/it]
 69%|██████▉   | 2756/4000 [3:13:55<1:29:07,  4.30s/it]
 69%|██████▉   | 2757/4000 [3:13:59<1:28:37,  4.28s/it]
 69%|██████▉   | 2758/4000 [3:14:04<1:27:52,  4.25s/it]
 69%|██████▉   | 2759/4000 [3:14:08<1:27:07,  4.21s/it]
 69%|██████▉   | 2760/4000 [3:14:12<1:26:30,  4.19s/it]
                                                       
{'loss': '0.401', 'grad_norm': '0.1253', 'learning_rate': '0.000188', 'epoch': '0.6687'}

 69%|██████▉   | 2760/4000 [3:14:12<1:26:30,  4.19s/it]
 69%|██████▉   | 2761/4000 [3:14:17<1:29:04,  4.31s/it]
 69%|██████▉   | 2762/4000 [3:14:21<1:29:18,  4.33s/it]
 69%|██████▉   | 2763/4000 [3:14:25<1:29:00,  4.32s/it]
 69%|██████▉   | 2764/4000 [3:14:29<1:27:45,  4.26s/it]
 69%|██████▉   | 2765/4000 [3:14:33<1:26:57,  4.22s/it]
 69%|██████▉   | 2766/4000 [3:14:38<1:26:16,  4.19s/it]
 69%|██████▉   | 2767/4000 [3:14:42<1:25:48,  4.18s/it]
 69%|██████▉   | 2768/4000 [3:14:46<1:25:26,  4.16s/it]
 69%|██████▉   | 2769/4000 [3:14:50<1:27:31,  4.27s/it]
 69%|██████▉   | 2770/4000 [3:14:55<1:27:23,  4.26s/it]
                                                       
{'loss': '0.406', 'grad_norm': '0.1118', 'learning_rate': '0.0001865', 'epoch': '0.6712'}

 69%|██████▉   | 2770/4000 [3:14:55<1:27:23,  4.26s/it]
 69%|██████▉   | 2771/4000 [3:14:59<1:27:25,  4.27s/it]
 69%|██████▉   | 2772/4000 [3:15:03<1:26:29,  4.23s/it]
 69%|██████▉   | 2773/4000 [3:15:07<1:26:06,  4.21s/it]
 69%|██████▉   | 2774/4000 [3:15:12<1:27:04,  4.26s/it]
 69%|██████▉   | 2775/4000 [3:15:16<1:26:44,  4.25s/it]
 69%|██████▉   | 2776/4000 [3:15:20<1:28:14,  4.33s/it]
 69%|██████▉   | 2777/4000 [3:15:24<1:27:12,  4.28s/it]
 69%|██████▉   | 2778/4000 [3:15:29<1:27:15,  4.28s/it]
 69%|██████▉   | 2779/4000 [3:15:33<1:26:32,  4.25s/it]
 70%|██████▉   | 2780/4000 [3:15:37<1:25:41,  4.21s/it]
                                                       
{'loss': '0.4112', 'grad_norm': '0.1211', 'learning_rate': '0.000185', 'epoch': '0.6736'}

 70%|██████▉   | 2780/4000 [3:15:37<1:25:41,  4.21s/it]
 70%|██████▉   | 2781/4000 [3:15:41<1:25:09,  4.19s/it]
 70%|██████▉   | 2782/4000 [3:15:45<1:24:44,  4.17s/it]
 70%|██████▉   | 2783/4000 [3:15:50<1:26:26,  4.26s/it]
 70%|██████▉   | 2784/4000 [3:15:54<1:25:46,  4.23s/it]
 70%|██████▉   | 2785/4000 [3:15:58<1:25:08,  4.20s/it]
 70%|██████▉   | 2786/4000 [3:16:02<1:24:42,  4.19s/it]
 70%|██████▉   | 2787/4000 [3:16:07<1:25:17,  4.22s/it]
 70%|██████▉   | 2788/4000 [3:16:11<1:25:50,  4.25s/it]
 70%|██████▉   | 2789/4000 [3:16:15<1:26:23,  4.28s/it]
 70%|██████▉   | 2790/4000 [3:16:19<1:26:21,  4.28s/it]
                                                       
{'loss': '0.363', 'grad_norm': '0.115', 'learning_rate': '0.0001835', 'epoch': '0.676'}

 70%|██████▉   | 2790/4000 [3:16:19<1:26:21,  4.28s/it]
 70%|██████▉   | 2791/4000 [3:16:24<1:26:29,  4.29s/it]
 70%|██████▉   | 2792/4000 [3:16:28<1:26:43,  4.31s/it]
 70%|██████▉   | 2793/4000 [3:16:32<1:25:41,  4.26s/it]
 70%|██████▉   | 2794/4000 [3:16:37<1:25:23,  4.25s/it]
 70%|██████▉   | 2795/4000 [3:16:41<1:25:42,  4.27s/it]
 70%|██████▉   | 2796/4000 [3:16:45<1:27:17,  4.35s/it]
 70%|██████▉   | 2797/4000 [3:16:50<1:25:54,  4.28s/it]
 70%|██████▉   | 2798/4000 [3:16:54<1:24:52,  4.24s/it]
 70%|██████▉   | 2799/4000 [3:16:58<1:24:12,  4.21s/it]
 70%|███████   | 2800/4000 [3:17:02<1:23:43,  4.19s/it]
                                                       
{'loss': '0.3853', 'grad_norm': '0.1256', 'learning_rate': '0.000182', 'epoch': '0.6784'}

 70%|███████   | 2800/4000 [3:17:02<1:23:43,  4.19s/it]
 70%|███████   | 2801/4000 [3:17:06<1:23:17,  4.17s/it]
 70%|███████   | 2802/4000 [3:17:10<1:23:02,  4.16s/it]
 70%|███████   | 2803/4000 [3:17:15<1:25:04,  4.26s/it]
 70%|███████   | 2804/4000 [3:17:19<1:24:36,  4.24s/it]
 70%|███████   | 2805/4000 [3:17:23<1:25:24,  4.29s/it]
 70%|███████   | 2806/4000 [3:17:27<1:24:29,  4.25s/it]
 70%|███████   | 2807/4000 [3:17:32<1:23:47,  4.21s/it]
 70%|███████   | 2808/4000 [3:17:36<1:23:50,  4.22s/it]
 70%|███████   | 2809/4000 [3:17:40<1:24:37,  4.26s/it]
 70%|███████   | 2810/4000 [3:17:45<1:26:05,  4.34s/it]
                                                       
{'loss': '0.4152', 'grad_norm': '0.1047', 'learning_rate': '0.0001805', 'epoch': '0.6809'}

 70%|███████   | 2810/4000 [3:17:45<1:26:05,  4.34s/it]
 70%|███████   | 2811/4000 [3:17:49<1:25:22,  4.31s/it]
 70%|███████   | 2812/4000 [3:17:53<1:24:51,  4.29s/it]
 70%|███████   | 2813/4000 [3:17:57<1:23:51,  4.24s/it]
 70%|███████   | 2814/4000 [3:18:01<1:23:12,  4.21s/it]
 70%|███████   | 2815/4000 [3:18:06<1:22:40,  4.19s/it]
 70%|███████   | 2816/4000 [3:18:10<1:24:18,  4.27s/it]
 70%|███████   | 2817/4000 [3:18:14<1:23:25,  4.23s/it]
 70%|███████   | 2818/4000 [3:18:18<1:22:42,  4.20s/it]
 70%|███████   | 2819/4000 [3:18:22<1:22:14,  4.18s/it]
 70%|███████   | 2820/4000 [3:18:27<1:21:51,  4.16s/it]
                                                       
{'loss': '0.3839', 'grad_norm': '0.1445', 'learning_rate': '0.0001789', 'epoch': '0.6833'}

 70%|███████   | 2820/4000 [3:18:27<1:21:51,  4.16s/it]
 71%|███████   | 2821/4000 [3:18:31<1:22:20,  4.19s/it]
 71%|███████   | 2822/4000 [3:18:35<1:23:39,  4.26s/it]
 71%|███████   | 2823/4000 [3:18:40<1:25:10,  4.34s/it]
 71%|███████   | 2824/4000 [3:18:44<1:23:55,  4.28s/it]
 71%|███████   | 2825/4000 [3:18:48<1:23:18,  4.25s/it]
 71%|███████   | 2826/4000 [3:18:52<1:23:22,  4.26s/it]
 71%|███████   | 2827/4000 [3:18:56<1:22:32,  4.22s/it]
 71%|███████   | 2828/4000 [3:19:01<1:22:41,  4.23s/it]
 71%|███████   | 2829/4000 [3:19:05<1:22:33,  4.23s/it]
 71%|███████   | 2830/4000 [3:19:09<1:24:05,  4.31s/it]
                                                       
{'loss': '0.4183', 'grad_norm': '0.131', 'learning_rate': '0.0001774', 'epoch': '0.6857'}

 71%|███████   | 2830/4000 [3:19:09<1:24:05,  4.31s/it]
 71%|███████   | 2831/4000 [3:19:14<1:22:57,  4.26s/it]
 71%|███████   | 2832/4000 [3:19:18<1:22:08,  4.22s/it]
 71%|███████   | 2833/4000 [3:19:22<1:21:32,  4.19s/it]
 71%|███████   | 2834/4000 [3:19:26<1:21:05,  4.17s/it]
 71%|███████   | 2835/4000 [3:19:30<1:20:46,  4.16s/it]
 71%|███████   | 2836/4000 [3:19:34<1:21:34,  4.21s/it]
 71%|███████   | 2837/4000 [3:19:39<1:22:09,  4.24s/it]
 71%|███████   | 2838/4000 [3:19:43<1:22:05,  4.24s/it]
 71%|███████   | 2839/4000 [3:19:47<1:22:33,  4.27s/it]
 71%|███████   | 2840/4000 [3:19:51<1:21:46,  4.23s/it]
                                                       
{'loss': '0.3865', 'grad_norm': '0.1168', 'learning_rate': '0.0001759', 'epoch': '0.6881'}

 71%|███████   | 2840/4000 [3:19:51<1:21:46,  4.23s/it]
 71%|███████   | 2841/4000 [3:19:56<1:21:10,  4.20s/it]
 71%|███████   | 2842/4000 [3:20:00<1:20:37,  4.18s/it]
 71%|███████   | 2843/4000 [3:20:05<1:24:25,  4.38s/it]
 71%|███████   | 2844/4000 [3:20:09<1:23:57,  4.36s/it]
 71%|███████   | 2845/4000 [3:20:13<1:23:17,  4.33s/it]
 71%|███████   | 2846/4000 [3:20:17<1:22:22,  4.28s/it]
 71%|███████   | 2847/4000 [3:20:21<1:21:25,  4.24s/it]
 71%|███████   | 2848/4000 [3:20:26<1:20:48,  4.21s/it]
 71%|███████   | 2849/4000 [3:20:30<1:20:20,  4.19s/it]
 71%|███████▏  | 2850/4000 [3:20:34<1:21:45,  4.27s/it]
                                                       
{'loss': '0.3943', 'grad_norm': '0.1227', 'learning_rate': '0.0001744', 'epoch': '0.6906'}

 71%|███████▏  | 2850/4000 [3:20:34<1:21:45,  4.27s/it]
 71%|███████▏  | 2851/4000 [3:20:38<1:21:03,  4.23s/it]
 71%|███████▏  | 2852/4000 [3:20:42<1:20:25,  4.20s/it]
 71%|███████▏  | 2853/4000 [3:20:47<1:20:00,  4.18s/it]
 71%|███████▏  | 2854/4000 [3:20:51<1:19:40,  4.17s/it]
 71%|███████▏  | 2855/4000 [3:20:55<1:19:40,  4.18s/it]
 71%|███████▏  | 2856/4000 [3:20:59<1:19:56,  4.19s/it]
 71%|███████▏  | 2857/4000 [3:21:04<1:22:14,  4.32s/it]
 71%|███████▏  | 2858/4000 [3:21:08<1:21:07,  4.26s/it]
 71%|███████▏  | 2859/4000 [3:21:12<1:20:19,  4.22s/it]
 72%|███████▏  | 2860/4000 [3:21:16<1:20:44,  4.25s/it]
                                                       
{'loss': '0.3776', 'grad_norm': '0.1255', 'learning_rate': '0.0001729', 'epoch': '0.693'}

 72%|███████▏  | 2860/4000 [3:21:16<1:20:44,  4.25s/it]
 72%|███████▏  | 2861/4000 [3:21:21<1:22:30,  4.35s/it]
 72%|███████▏  | 2862/4000 [3:21:25<1:22:37,  4.36s/it]
 72%|███████▏  | 2863/4000 [3:21:30<1:22:30,  4.35s/it]
 72%|███████▏  | 2864/4000 [3:21:34<1:22:10,  4.34s/it]
 72%|███████▏  | 2865/4000 [3:21:38<1:20:57,  4.28s/it]
 72%|███████▏  | 2866/4000 [3:21:42<1:20:00,  4.23s/it]
 72%|███████▏  | 2867/4000 [3:21:46<1:19:27,  4.21s/it]
 72%|███████▏  | 2868/4000 [3:21:51<1:18:57,  4.19s/it]
 72%|███████▏  | 2869/4000 [3:21:55<1:18:37,  4.17s/it]
 72%|███████▏  | 2870/4000 [3:21:59<1:20:30,  4.27s/it]
                                                       
{'loss': '0.418', 'grad_norm': '0.1241', 'learning_rate': '0.0001714', 'epoch': '0.6954'}

 72%|███████▏  | 2870/4000 [3:21:59<1:20:30,  4.27s/it]
 72%|███████▏  | 2871/4000 [3:22:03<1:19:39,  4.23s/it]
 72%|███████▏  | 2872/4000 [3:22:07<1:18:57,  4.20s/it]
 72%|███████▏  | 2873/4000 [3:22:12<1:19:00,  4.21s/it]
 72%|███████▏  | 2874/4000 [3:22:16<1:19:12,  4.22s/it]
 72%|███████▏  | 2875/4000 [3:22:20<1:18:34,  4.19s/it]
 72%|███████▏  | 2876/4000 [3:22:24<1:18:15,  4.18s/it]
 72%|███████▏  | 2877/4000 [3:22:29<1:20:44,  4.31s/it]
 72%|███████▏  | 2878/4000 [3:22:33<1:21:07,  4.34s/it]
 72%|███████▏  | 2879/4000 [3:22:38<1:21:34,  4.37s/it]
 72%|███████▏  | 2880/4000 [3:22:42<1:20:30,  4.31s/it]
                                                       
{'loss': '0.3798', 'grad_norm': '0.1186', 'learning_rate': '0.0001698', 'epoch': '0.6978'}

 72%|███████▏  | 2880/4000 [3:22:42<1:20:30,  4.31s/it]
 72%|███████▏  | 2881/4000 [3:22:46<1:19:23,  4.26s/it]
 72%|███████▏  | 2882/4000 [3:22:50<1:18:38,  4.22s/it]
 72%|███████▏  | 2883/4000 [3:22:54<1:18:05,  4.19s/it]
 72%|███████▏  | 2884/4000 [3:22:59<1:19:31,  4.28s/it]
 72%|███████▏  | 2885/4000 [3:23:03<1:18:42,  4.24s/it]
 72%|███████▏  | 2886/4000 [3:23:07<1:18:03,  4.20s/it]
 72%|███████▏  | 2887/4000 [3:23:11<1:17:33,  4.18s/it]
 72%|███████▏  | 2888/4000 [3:23:15<1:17:14,  4.17s/it]
 72%|███████▏  | 2889/4000 [3:23:19<1:17:01,  4.16s/it]
 72%|███████▏  | 2890/4000 [3:23:24<1:18:42,  4.25s/it]
                                                       
{'loss': '0.4351', 'grad_norm': '0.1104', 'learning_rate': '0.0001683', 'epoch': '0.7002'}

 72%|███████▏  | 2890/4000 [3:23:24<1:18:42,  4.25s/it]
 72%|███████▏  | 2891/4000 [3:23:28<1:20:01,  4.33s/it]
 72%|███████▏  | 2892/4000 [3:23:32<1:18:57,  4.28s/it]
 72%|███████▏  | 2893/4000 [3:23:37<1:18:41,  4.27s/it]
 72%|███████▏  | 2894/4000 [3:23:41<1:18:48,  4.28s/it]
 72%|███████▏  | 2895/4000 [3:23:45<1:18:52,  4.28s/it]
 72%|███████▏  | 2896/4000 [3:23:50<1:19:10,  4.30s/it]
 72%|███████▏  | 2897/4000 [3:23:54<1:21:03,  4.41s/it]
 72%|███████▏  | 2898/4000 [3:23:58<1:19:26,  4.33s/it]
 72%|███████▏  | 2899/4000 [3:24:03<1:18:18,  4.27s/it]
 72%|███████▎  | 2900/4000 [3:24:07<1:17:27,  4.22s/it]
                                                       
{'loss': '0.4209', 'grad_norm': '0.1284', 'learning_rate': '0.0001668', 'epoch': '0.7027'}

 72%|███████▎  | 2900/4000 [3:24:07<1:17:27,  4.22s/it]
 73%|███████▎  | 2901/4000 [3:24:11<1:16:52,  4.20s/it]
 73%|███████▎  | 2902/4000 [3:24:15<1:16:23,  4.17s/it]
 73%|███████▎  | 2903/4000 [3:24:19<1:16:08,  4.16s/it]
 73%|███████▎  | 2904/4000 [3:24:24<1:17:50,  4.26s/it]
 73%|███████▎  | 2905/4000 [3:24:28<1:17:01,  4.22s/it]
 73%|███████▎  | 2906/4000 [3:24:32<1:16:26,  4.19s/it]
 73%|███████▎  | 2907/4000 [3:24:36<1:16:02,  4.17s/it]
 73%|███████▎  | 2908/4000 [3:24:40<1:16:42,  4.21s/it]
 73%|███████▎  | 2909/4000 [3:24:44<1:16:28,  4.21s/it]
 73%|███████▎  | 2910/4000 [3:24:49<1:16:57,  4.24s/it]
                                                       
{'loss': '0.4031', 'grad_norm': '0.1219', 'learning_rate': '0.0001653', 'epoch': '0.7051'}

 73%|███████▎  | 2910/4000 [3:24:49<1:16:57,  4.24s/it]
 73%|███████▎  | 2911/4000 [3:24:54<1:19:31,  4.38s/it]
 73%|███████▎  | 2912/4000 [3:24:58<1:18:56,  4.35s/it]
 73%|███████▎  | 2913/4000 [3:25:02<1:18:49,  4.35s/it]
 73%|███████▎  | 2914/4000 [3:25:06<1:18:37,  4.34s/it]
 73%|███████▎  | 2915/4000 [3:25:11<1:17:21,  4.28s/it]
 73%|███████▎  | 2916/4000 [3:25:15<1:16:30,  4.23s/it]
 73%|███████▎  | 2917/4000 [3:25:19<1:16:49,  4.26s/it]
 73%|███████▎  | 2918/4000 [3:25:23<1:16:56,  4.27s/it]
 73%|███████▎  | 2919/4000 [3:25:27<1:16:04,  4.22s/it]
 73%|███████▎  | 2920/4000 [3:25:32<1:15:31,  4.20s/it]
                                                       
{'loss': '0.402', 'grad_norm': '0.1316', 'learning_rate': '0.0001638', 'epoch': '0.7075'}

 73%|███████▎  | 2920/4000 [3:25:32<1:15:31,  4.20s/it]
 73%|███████▎  | 2921/4000 [3:25:36<1:15:03,  4.17s/it]
 73%|███████▎  | 2922/4000 [3:25:40<1:14:46,  4.16s/it]
 73%|███████▎  | 2923/4000 [3:25:44<1:14:35,  4.16s/it]
 73%|███████▎  | 2924/4000 [3:25:48<1:16:24,  4.26s/it]
 73%|███████▎  | 2925/4000 [3:25:53<1:15:58,  4.24s/it]
 73%|███████▎  | 2926/4000 [3:25:57<1:16:00,  4.25s/it]
 73%|███████▎  | 2927/4000 [3:26:01<1:15:39,  4.23s/it]
 73%|███████▎  | 2928/4000 [3:26:05<1:15:11,  4.21s/it]
 73%|███████▎  | 2929/4000 [3:26:10<1:15:53,  4.25s/it]
 73%|███████▎  | 2930/4000 [3:26:14<1:16:04,  4.27s/it]
                                                       
{'loss': '0.3727', 'grad_norm': '0.1204', 'learning_rate': '0.0001623', 'epoch': '0.7099'}

 73%|███████▎  | 2930/4000 [3:26:14<1:16:04,  4.27s/it]
 73%|███████▎  | 2931/4000 [3:26:19<1:18:05,  4.38s/it]
 73%|███████▎  | 2932/4000 [3:26:23<1:16:41,  4.31s/it]
 73%|███████▎  | 2933/4000 [3:26:27<1:15:34,  4.25s/it]
 73%|███████▎  | 2934/4000 [3:26:31<1:14:57,  4.22s/it]
 73%|███████▎  | 2935/4000 [3:26:35<1:14:23,  4.19s/it]
 73%|███████▎  | 2936/4000 [3:26:39<1:13:58,  4.17s/it]
 73%|███████▎  | 2937/4000 [3:26:43<1:13:35,  4.15s/it]
 73%|███████▎  | 2938/4000 [3:26:48<1:15:11,  4.25s/it]
 73%|███████▎  | 2939/4000 [3:26:52<1:14:27,  4.21s/it]
 74%|███████▎  | 2940/4000 [3:26:56<1:13:53,  4.18s/it]
                                                       
{'loss': '0.4068', 'grad_norm': '0.1519', 'learning_rate': '0.0001608', 'epoch': '0.7124'}

 74%|███████▎  | 2940/4000 [3:26:56<1:13:53,  4.18s/it]
 74%|███████▎  | 2941/4000 [3:27:00<1:13:34,  4.17s/it]
 74%|███████▎  | 2942/4000 [3:27:04<1:13:32,  4.17s/it]
 74%|███████▎  | 2943/4000 [3:27:09<1:13:56,  4.20s/it]
 74%|███████▎  | 2944/4000 [3:27:13<1:16:35,  4.35s/it]
 74%|███████▎  | 2945/4000 [3:27:18<1:15:38,  4.30s/it]
 74%|███████▎  | 2946/4000 [3:27:22<1:15:26,  4.29s/it]
 74%|███████▎  | 2947/4000 [3:27:26<1:15:39,  4.31s/it]
 74%|███████▎  | 2948/4000 [3:27:30<1:15:45,  4.32s/it]
 74%|███████▎  | 2949/4000 [3:27:35<1:14:55,  4.28s/it]
 74%|███████▍  | 2950/4000 [3:27:39<1:14:04,  4.23s/it]
                                                       
{'loss': '0.369', 'grad_norm': '0.1085', 'learning_rate': '0.0001592', 'epoch': '0.7148'}

 74%|███████▍  | 2950/4000 [3:27:39<1:14:04,  4.23s/it]
 74%|███████▍  | 2951/4000 [3:27:43<1:15:03,  4.29s/it]
 74%|███████▍  | 2952/4000 [3:27:47<1:14:11,  4.25s/it]
 74%|███████▍  | 2953/4000 [3:27:52<1:13:38,  4.22s/it]
 74%|███████▍  | 2954/4000 [3:27:56<1:13:09,  4.20s/it]
 74%|███████▍  | 2955/4000 [3:28:00<1:12:48,  4.18s/it]
 74%|███████▍  | 2956/4000 [3:28:04<1:12:32,  4.17s/it]
 74%|███████▍  | 2957/4000 [3:28:08<1:12:17,  4.16s/it]
 74%|███████▍  | 2958/4000 [3:28:13<1:13:58,  4.26s/it]
 74%|███████▍  | 2959/4000 [3:28:17<1:13:19,  4.23s/it]
 74%|███████▍  | 2960/4000 [3:28:21<1:13:16,  4.23s/it]
                                                       
{'loss': '0.3512', 'grad_norm': '0.1349', 'learning_rate': '0.0001577', 'epoch': '0.7172'}

 74%|███████▍  | 2960/4000 [3:28:21<1:13:16,  4.23s/it]
 74%|███████▍  | 2961/4000 [3:28:25<1:13:07,  4.22s/it]
 74%|███████▍  | 2962/4000 [3:28:29<1:13:32,  4.25s/it]
 74%|███████▍  | 2963/4000 [3:28:34<1:13:18,  4.24s/it]
 74%|███████▍  | 2964/4000 [3:28:38<1:14:00,  4.29s/it]
 74%|███████▍  | 2965/4000 [3:28:43<1:15:32,  4.38s/it]
 74%|███████▍  | 2966/4000 [3:28:47<1:14:14,  4.31s/it]
 74%|███████▍  | 2967/4000 [3:28:51<1:13:11,  4.25s/it]
 74%|███████▍  | 2968/4000 [3:28:55<1:12:30,  4.22s/it]
 74%|███████▍  | 2969/4000 [3:28:59<1:12:04,  4.19s/it]
 74%|███████▍  | 2970/4000 [3:29:03<1:11:37,  4.17s/it]
                                                       
{'loss': '0.3859', 'grad_norm': '0.1437', 'learning_rate': '0.0001562', 'epoch': '0.7196'}

 74%|███████▍  | 2970/4000 [3:29:03<1:11:37,  4.17s/it]
 74%|███████▍  | 2971/4000 [3:29:08<1:13:21,  4.28s/it]
 74%|███████▍  | 2972/4000 [3:29:12<1:12:30,  4.23s/it]
 74%|███████▍  | 2973/4000 [3:29:16<1:11:51,  4.20s/it]
 74%|███████▍  | 2974/4000 [3:29:20<1:11:29,  4.18s/it]
 74%|███████▍  | 2975/4000 [3:29:24<1:11:12,  4.17s/it]
 74%|███████▍  | 2976/4000 [3:29:29<1:10:53,  4.15s/it]
 74%|███████▍  | 2977/4000 [3:29:33<1:11:15,  4.18s/it]
 74%|███████▍  | 2978/4000 [3:29:37<1:13:47,  4.33s/it]
 74%|███████▍  | 2979/4000 [3:29:42<1:13:40,  4.33s/it]
 74%|███████▍  | 2980/4000 [3:29:46<1:12:47,  4.28s/it]
                                                       
{'loss': '0.4049', 'grad_norm': '0.1137', 'learning_rate': '0.0001547', 'epoch': '0.7221'}

 74%|███████▍  | 2980/4000 [3:29:46<1:12:47,  4.28s/it]
 75%|███████▍  | 2981/4000 [3:29:50<1:12:28,  4.27s/it]
 75%|███████▍  | 2982/4000 [3:29:54<1:12:05,  4.25s/it]
 75%|███████▍  | 2983/4000 [3:29:59<1:11:44,  4.23s/it]
 75%|███████▍  | 2984/4000 [3:30:03<1:11:14,  4.21s/it]
 75%|███████▍  | 2985/4000 [3:30:07<1:12:22,  4.28s/it]
 75%|███████▍  | 2986/4000 [3:30:11<1:11:29,  4.23s/it]
 75%|███████▍  | 2987/4000 [3:30:15<1:10:55,  4.20s/it]
 75%|███████▍  | 2988/4000 [3:30:20<1:10:27,  4.18s/it]
 75%|███████▍  | 2989/4000 [3:30:24<1:10:06,  4.16s/it]
 75%|███████▍  | 2990/4000 [3:30:28<1:09:52,  4.15s/it]
                                                       
{'loss': '0.3667', 'grad_norm': '0.1142', 'learning_rate': '0.0001532', 'epoch': '0.7245'}

 75%|███████▍  | 2990/4000 [3:30:28<1:09:52,  4.15s/it]
 75%|███████▍  | 2991/4000 [3:30:32<1:09:40,  4.14s/it]
 75%|███████▍  | 2992/4000 [3:30:36<1:11:29,  4.26s/it]
 75%|███████▍  | 2993/4000 [3:30:41<1:10:45,  4.22s/it]
 75%|███████▍  | 2994/4000 [3:30:45<1:10:39,  4.21s/it]
 75%|███████▍  | 2995/4000 [3:30:49<1:10:36,  4.22s/it]
 75%|███████▍  | 2996/4000 [3:30:53<1:10:37,  4.22s/it]
 75%|███████▍  | 2997/4000 [3:30:58<1:11:02,  4.25s/it]
 75%|███████▍  | 2998/4000 [3:31:02<1:12:42,  4.35s/it]
 75%|███████▍  | 2999/4000 [3:31:06<1:12:14,  4.33s/it]
 75%|███████▌  | 3000/4000 [3:31:11<1:11:37,  4.30s/it]
                                                       
{'loss': '0.3712', 'grad_norm': '0.1242', 'learning_rate': '0.0001517', 'epoch': '0.7269'}

 75%|███████▌  | 3000/4000 [3:31:11<1:11:37,  4.30s/it]
 75%|███████▌  | 3001/4000 [3:31:15<1:10:43,  4.25s/it]
 75%|███████▌  | 3002/4000 [3:31:19<1:10:03,  4.21s/it]
 75%|███████▌  | 3003/4000 [3:31:23<1:09:34,  4.19s/it]
 75%|███████▌  | 3004/4000 [3:31:27<1:09:19,  4.18s/it]
 75%|███████▌  | 3005/4000 [3:31:32<1:10:54,  4.28s/it]
 75%|███████▌  | 3006/4000 [3:31:36<1:10:07,  4.23s/it]
 75%|███████▌  | 3007/4000 [3:31:40<1:09:31,  4.20s/it]
 75%|███████▌  | 3008/4000 [3:31:44<1:09:08,  4.18s/it]
 75%|███████▌  | 3009/4000 [3:31:48<1:08:50,  4.17s/it]
 75%|███████▌  | 3010/4000 [3:31:52<1:08:33,  4.16s/it]
                                                       
{'loss': '0.3987', 'grad_norm': '0.1528', 'learning_rate': '0.0001502', 'epoch': '0.7293'}

 75%|███████▌  | 3010/4000 [3:31:52<1:08:33,  4.16s/it]
 75%|███████▌  | 3011/4000 [3:31:57<1:08:40,  4.17s/it]
 75%|███████▌  | 3012/4000 [3:32:01<1:10:58,  4.31s/it]
 75%|███████▌  | 3013/4000 [3:32:05<1:10:24,  4.28s/it]
 75%|███████▌  | 3014/4000 [3:32:10<1:11:11,  4.33s/it]
 75%|███████▌  | 3015/4000 [3:32:14<1:10:38,  4.30s/it]
 75%|███████▌  | 3016/4000 [3:32:18<1:10:55,  4.33s/it]
 75%|███████▌  | 3017/4000 [3:32:23<1:09:51,  4.26s/it]
 75%|███████▌  | 3018/4000 [3:32:27<1:09:47,  4.26s/it]
 75%|███████▌  | 3019/4000 [3:32:31<1:09:44,  4.27s/it]
 76%|███████▌  | 3020/4000 [3:32:35<1:08:52,  4.22s/it]
                                                       
{'loss': '0.3992', 'grad_norm': '0.1216', 'learning_rate': '0.0001486', 'epoch': '0.7317'}

 76%|███████▌  | 3020/4000 [3:32:35<1:08:52,  4.22s/it]
 76%|███████▌  | 3021/4000 [3:32:39<1:08:16,  4.18s/it]
 76%|███████▌  | 3022/4000 [3:32:43<1:07:51,  4.16s/it]
 76%|███████▌  | 3023/4000 [3:32:48<1:07:29,  4.14s/it]
 76%|███████▌  | 3024/4000 [3:32:52<1:07:19,  4.14s/it]
 76%|███████▌  | 3025/4000 [3:32:56<1:09:42,  4.29s/it]
 76%|███████▌  | 3026/4000 [3:33:00<1:08:53,  4.24s/it]
 76%|███████▌  | 3027/4000 [3:33:05<1:08:12,  4.21s/it]
 76%|███████▌  | 3028/4000 [3:33:09<1:07:48,  4.19s/it]
 76%|███████▌  | 3029/4000 [3:33:13<1:07:24,  4.17s/it]
 76%|███████▌  | 3030/4000 [3:33:17<1:07:10,  4.16s/it]
                                                       
{'loss': '0.427', 'grad_norm': '0.1293', 'learning_rate': '0.0001471', 'epoch': '0.7342'}

 76%|███████▌  | 3030/4000 [3:33:17<1:07:10,  4.16s/it]
 76%|███████▌  | 3031/4000 [3:33:21<1:06:57,  4.15s/it]
 76%|███████▌  | 3032/4000 [3:33:26<1:08:44,  4.26s/it]
 76%|███████▌  | 3033/4000 [3:33:30<1:08:03,  4.22s/it]
 76%|███████▌  | 3034/4000 [3:33:34<1:07:31,  4.19s/it]
 76%|███████▌  | 3035/4000 [3:33:38<1:07:09,  4.18s/it]
 76%|███████▌  | 3036/4000 [3:33:42<1:06:50,  4.16s/it]
 76%|███████▌  | 3037/4000 [3:33:46<1:06:30,  4.14s/it]
 76%|███████▌  | 3038/4000 [3:33:50<1:06:18,  4.14s/it]
 76%|███████▌  | 3039/4000 [3:33:55<1:07:45,  4.23s/it]
 76%|███████▌  | 3040/4000 [3:33:59<1:07:07,  4.19s/it]
                                                       
{'loss': '0.3702', 'grad_norm': '0.09895', 'learning_rate': '0.0001456', 'epoch': '0.7366'}

 76%|███████▌  | 3040/4000 [3:33:59<1:07:07,  4.19s/it]
 76%|███████▌  | 3041/4000 [3:34:03<1:06:39,  4.17s/it]
 76%|███████▌  | 3042/4000 [3:34:07<1:06:18,  4.15s/it]
 76%|███████▌  | 3043/4000 [3:34:11<1:06:04,  4.14s/it]
 76%|███████▌  | 3044/4000 [3:34:15<1:05:51,  4.13s/it]
 76%|███████▌  | 3045/4000 [3:34:20<1:06:32,  4.18s/it]
 76%|███████▌  | 3046/4000 [3:34:24<1:07:14,  4.23s/it]
 76%|███████▌  | 3047/4000 [3:34:28<1:06:52,  4.21s/it]
 76%|███████▌  | 3048/4000 [3:34:32<1:06:18,  4.18s/it]
 76%|███████▌  | 3049/4000 [3:34:36<1:05:55,  4.16s/it]
 76%|███████▋  | 3050/4000 [3:34:40<1:05:41,  4.15s/it]
                                                       
{'loss': '0.3753', 'grad_norm': '0.1163', 'learning_rate': '0.0001441', 'epoch': '0.739'}

 76%|███████▋  | 3050/4000 [3:34:41<1:05:41,  4.15s/it]
 76%|███████▋  | 3051/4000 [3:34:45<1:05:25,  4.14s/it]
 76%|███████▋  | 3052/4000 [3:34:49<1:06:48,  4.23s/it]
 76%|███████▋  | 3053/4000 [3:34:53<1:06:13,  4.20s/it]
 76%|███████▋  | 3054/4000 [3:34:57<1:05:46,  4.17s/it]
 76%|███████▋  | 3055/4000 [3:35:01<1:05:24,  4.15s/it]
 76%|███████▋  | 3056/4000 [3:35:06<1:05:10,  4.14s/it]
 76%|███████▋  | 3057/4000 [3:35:10<1:05:01,  4.14s/it]
 76%|███████▋  | 3058/4000 [3:35:14<1:05:01,  4.14s/it]
 76%|███████▋  | 3059/4000 [3:35:18<1:06:47,  4.26s/it]
 76%|███████▋  | 3060/4000 [3:35:23<1:06:26,  4.24s/it]
                                                       
{'loss': '0.3737', 'grad_norm': '0.1183', 'learning_rate': '0.0001426', 'epoch': '0.7414'}

 76%|███████▋  | 3060/4000 [3:35:23<1:06:26,  4.24s/it]
 77%|███████▋  | 3061/4000 [3:35:27<1:05:53,  4.21s/it]
 77%|███████▋  | 3062/4000 [3:35:31<1:05:30,  4.19s/it]
 77%|███████▋  | 3063/4000 [3:35:35<1:05:19,  4.18s/it]
 77%|███████▋  | 3064/4000 [3:35:39<1:05:07,  4.17s/it]
 77%|███████▋  | 3065/4000 [3:35:43<1:04:54,  4.17s/it]
 77%|███████▋  | 3066/4000 [3:35:48<1:06:26,  4.27s/it]
 77%|███████▋  | 3067/4000 [3:35:52<1:05:46,  4.23s/it]
 77%|███████▋  | 3068/4000 [3:35:56<1:05:18,  4.20s/it]
 77%|███████▋  | 3069/4000 [3:36:00<1:04:56,  4.18s/it]
 77%|███████▋  | 3070/4000 [3:36:04<1:04:39,  4.17s/it]
                                                       
{'loss': '0.4057', 'grad_norm': '0.1133', 'learning_rate': '0.0001411', 'epoch': '0.7439'}

 77%|███████▋  | 3070/4000 [3:36:04<1:04:39,  4.17s/it]
 77%|███████▋  | 3071/4000 [3:36:08<1:04:27,  4.16s/it]
 77%|███████▋  | 3072/4000 [3:36:13<1:05:59,  4.27s/it]
 77%|███████▋  | 3073/4000 [3:36:17<1:05:20,  4.23s/it]
 77%|███████▋  | 3074/4000 [3:36:21<1:04:50,  4.20s/it]
 77%|███████▋  | 3075/4000 [3:36:25<1:04:30,  4.18s/it]
 77%|███████▋  | 3076/4000 [3:36:30<1:04:14,  4.17s/it]
 77%|███████▋  | 3077/4000 [3:36:34<1:05:02,  4.23s/it]
 77%|███████▋  | 3078/4000 [3:36:38<1:05:02,  4.23s/it]
 77%|███████▋  | 3079/4000 [3:36:43<1:07:06,  4.37s/it]
 77%|███████▋  | 3080/4000 [3:36:47<1:06:27,  4.33s/it]
                                                       
{'loss': '0.4005', 'grad_norm': '0.117', 'learning_rate': '0.0001395', 'epoch': '0.7463'}

 77%|███████▋  | 3080/4000 [3:36:47<1:06:27,  4.33s/it]
 77%|███████▋  | 3081/4000 [3:36:51<1:05:59,  4.31s/it]
 77%|███████▋  | 3082/4000 [3:36:56<1:05:43,  4.30s/it]
 77%|███████▋  | 3083/4000 [3:37:00<1:05:19,  4.27s/it]
 77%|███████▋  | 3084/4000 [3:37:04<1:04:39,  4.23s/it]
 77%|███████▋  | 3085/4000 [3:37:08<1:04:04,  4.20s/it]
 77%|███████▋  | 3086/4000 [3:37:13<1:05:21,  4.29s/it]
 77%|███████▋  | 3087/4000 [3:37:17<1:04:34,  4.24s/it]
 77%|███████▋  | 3088/4000 [3:37:21<1:04:02,  4.21s/it]
 77%|███████▋  | 3089/4000 [3:37:25<1:03:34,  4.19s/it]
 77%|███████▋  | 3090/4000 [3:37:29<1:03:16,  4.17s/it]
                                                       
{'loss': '0.3755', 'grad_norm': '0.1228', 'learning_rate': '0.000138', 'epoch': '0.7487'}

 77%|███████▋  | 3090/4000 [3:37:29<1:03:16,  4.17s/it]
 77%|███████▋  | 3091/4000 [3:37:33<1:02:57,  4.16s/it]
 77%|███████▋  | 3092/4000 [3:37:37<1:02:47,  4.15s/it]
 77%|███████▋  | 3093/4000 [3:37:42<1:04:23,  4.26s/it]
 77%|███████▋  | 3094/4000 [3:37:46<1:03:44,  4.22s/it]
 77%|███████▋  | 3095/4000 [3:37:50<1:03:14,  4.19s/it]
 77%|███████▋  | 3096/4000 [3:37:54<1:03:19,  4.20s/it]
 77%|███████▋  | 3097/4000 [3:37:59<1:03:17,  4.21s/it]
 77%|███████▋  | 3098/4000 [3:38:03<1:02:52,  4.18s/it]
 77%|███████▋  | 3099/4000 [3:38:07<1:04:22,  4.29s/it]
 78%|███████▊  | 3100/4000 [3:38:12<1:04:34,  4.31s/it]
                                                       
{'loss': '0.3929', 'grad_norm': '0.1191', 'learning_rate': '0.0001365', 'epoch': '0.7511'}

 78%|███████▊  | 3100/4000 [3:38:12<1:04:34,  4.31s/it]
 78%|███████▊  | 3101/4000 [3:38:16<1:04:28,  4.30s/it]
 78%|███████▊  | 3102/4000 [3:38:20<1:04:13,  4.29s/it]
 78%|███████▊  | 3103/4000 [3:38:24<1:04:09,  4.29s/it]
 78%|███████▊  | 3104/4000 [3:38:29<1:04:06,  4.29s/it]
 78%|███████▊  | 3105/4000 [3:38:33<1:03:17,  4.24s/it]
 78%|███████▊  | 3106/4000 [3:38:37<1:04:30,  4.33s/it]
 78%|███████▊  | 3107/4000 [3:38:42<1:03:31,  4.27s/it]
 78%|███████▊  | 3108/4000 [3:38:46<1:02:48,  4.23s/it]
 78%|███████▊  | 3109/4000 [3:38:50<1:02:21,  4.20s/it]
 78%|███████▊  | 3110/4000 [3:38:54<1:01:56,  4.18s/it]
                                                       
{'loss': '0.3758', 'grad_norm': '0.1098', 'learning_rate': '0.000135', 'epoch': '0.7536'}

 78%|███████▊  | 3110/4000 [3:38:54<1:01:56,  4.18s/it]
 78%|███████▊  | 3111/4000 [3:38:58<1:01:42,  4.16s/it]
 78%|███████▊  | 3112/4000 [3:39:02<1:01:52,  4.18s/it]
 78%|███████▊  | 3113/4000 [3:39:07<1:03:23,  4.29s/it]
 78%|███████▊  | 3114/4000 [3:39:11<1:02:34,  4.24s/it]
 78%|███████▊  | 3115/4000 [3:39:15<1:01:59,  4.20s/it]
 78%|███████▊  | 3116/4000 [3:39:19<1:01:36,  4.18s/it]
 78%|███████▊  | 3117/4000 [3:39:23<1:01:16,  4.16s/it]
 78%|███████▊  | 3118/4000 [3:39:27<1:01:04,  4.15s/it]
 78%|███████▊  | 3119/4000 [3:39:32<1:01:40,  4.20s/it]
 78%|███████▊  | 3120/4000 [3:39:36<1:02:16,  4.25s/it]
                                                       
{'loss': '0.3856', 'grad_norm': '0.1115', 'learning_rate': '0.0001335', 'epoch': '0.756'}

 78%|███████▊  | 3120/4000 [3:39:36<1:02:16,  4.25s/it]
 78%|███████▊  | 3121/4000 [3:39:40<1:02:36,  4.27s/it]
 78%|███████▊  | 3122/4000 [3:39:45<1:02:28,  4.27s/it]
 78%|███████▊  | 3123/4000 [3:39:49<1:02:19,  4.26s/it]
 78%|███████▊  | 3124/4000 [3:39:53<1:02:04,  4.25s/it]
 78%|███████▊  | 3125/4000 [3:39:57<1:01:30,  4.22s/it]
 78%|███████▊  | 3126/4000 [3:40:02<1:02:51,  4.31s/it]
 78%|███████▊  | 3127/4000 [3:40:06<1:01:56,  4.26s/it]
 78%|███████▊  | 3128/4000 [3:40:10<1:01:55,  4.26s/it]
 78%|███████▊  | 3129/4000 [3:40:14<1:01:15,  4.22s/it]
 78%|███████▊  | 3130/4000 [3:40:19<1:00:50,  4.20s/it]
                                                       
{'loss': '0.4092', 'grad_norm': '0.1183', 'learning_rate': '0.000132', 'epoch': '0.7584'}

 78%|███████▊  | 3130/4000 [3:40:19<1:00:50,  4.20s/it]
 78%|███████▊  | 3131/4000 [3:40:23<1:00:29,  4.18s/it]
 78%|███████▊  | 3132/4000 [3:40:27<1:00:12,  4.16s/it]
 78%|███████▊  | 3133/4000 [3:40:31<1:01:38,  4.27s/it]
 78%|███████▊  | 3134/4000 [3:40:35<1:00:59,  4.23s/it]
 78%|███████▊  | 3135/4000 [3:40:40<1:00:28,  4.19s/it]
 78%|███████▊  | 3136/4000 [3:40:44<1:00:31,  4.20s/it]
 78%|███████▊  | 3137/4000 [3:40:48<1:00:05,  4.18s/it]
 78%|███████▊  | 3138/4000 [3:40:52<59:51,  4.17s/it]  
 78%|███████▊  | 3139/4000 [3:40:56<59:35,  4.15s/it]
 78%|███████▊  | 3140/4000 [3:41:01<1:02:02,  4.33s/it]
                                                       
{'loss': '0.397', 'grad_norm': '0.1387', 'learning_rate': '0.0001305', 'epoch': '0.7608'}

 78%|███████▊  | 3140/4000 [3:41:01<1:02:02,  4.33s/it]
 79%|███████▊  | 3141/4000 [3:41:05<1:01:46,  4.31s/it]
 79%|███████▊  | 3142/4000 [3:41:09<1:01:42,  4.32s/it]
 79%|███████▊  | 3143/4000 [3:41:14<1:01:07,  4.28s/it]
 79%|███████▊  | 3144/4000 [3:41:18<1:01:52,  4.34s/it]
 79%|███████▊  | 3145/4000 [3:41:22<1:01:08,  4.29s/it]
 79%|███████▊  | 3146/4000 [3:41:27<1:01:20,  4.31s/it]
 79%|███████▊  | 3147/4000 [3:41:31<1:01:23,  4.32s/it]
 79%|███████▊  | 3148/4000 [3:41:35<1:00:34,  4.27s/it]
 79%|███████▊  | 3149/4000 [3:41:39<59:58,  4.23s/it]  
 79%|███████▉  | 3150/4000 [3:41:43<59:30,  4.20s/it]
                                                     
{'loss': '0.3628', 'grad_norm': '0.1247', 'learning_rate': '0.0001289', 'epoch': '0.7632'}

 79%|███████▉  | 3150/4000 [3:41:43<59:30,  4.20s/it]
 79%|███████▉  | 3151/4000 [3:41:48<59:38,  4.21s/it]
 79%|███████▉  | 3152/4000 [3:41:52<59:18,  4.20s/it]
 79%|███████▉  | 3153/4000 [3:41:56<1:00:27,  4.28s/it]
 79%|███████▉  | 3154/4000 [3:42:00<59:47,  4.24s/it]  
 79%|███████▉  | 3155/4000 [3:42:05<59:14,  4.21s/it]
 79%|███████▉  | 3156/4000 [3:42:09<58:54,  4.19s/it]
 79%|███████▉  | 3157/4000 [3:42:13<58:37,  4.17s/it]
 79%|███████▉  | 3158/4000 [3:42:17<59:19,  4.23s/it]
 79%|███████▉  | 3159/4000 [3:42:21<59:21,  4.24s/it]
 79%|███████▉  | 3160/4000 [3:42:26<1:01:38,  4.40s/it]
                                                       
{'loss': '0.3915', 'grad_norm': '0.1734', 'learning_rate': '0.0001274', 'epoch': '0.7657'}

 79%|███████▉  | 3160/4000 [3:42:26<1:01:38,  4.40s/it]
 79%|███████▉  | 3161/4000 [3:42:31<1:00:51,  4.35s/it]
 79%|███████▉  | 3162/4000 [3:42:35<59:54,  4.29s/it]  
 79%|███████▉  | 3163/4000 [3:42:39<59:09,  4.24s/it]
 79%|███████▉  | 3164/4000 [3:42:43<58:37,  4.21s/it]
 79%|███████▉  | 3165/4000 [3:42:47<58:13,  4.18s/it]
 79%|███████▉  | 3166/4000 [3:42:51<57:55,  4.17s/it]
 79%|███████▉  | 3167/4000 [3:42:56<59:37,  4.29s/it]
 79%|███████▉  | 3168/4000 [3:43:00<58:58,  4.25s/it]
 79%|███████▉  | 3169/4000 [3:43:04<58:24,  4.22s/it]
 79%|███████▉  | 3170/4000 [3:43:08<57:58,  4.19s/it]
                                                     
{'loss': '0.3829', 'grad_norm': '0.1434', 'learning_rate': '0.0001259', 'epoch': '0.7681'}

 79%|███████▉  | 3170/4000 [3:43:08<57:58,  4.19s/it]
 79%|███████▉  | 3171/4000 [3:43:12<57:38,  4.17s/it]
 79%|███████▉  | 3172/4000 [3:43:16<57:24,  4.16s/it]
 79%|███████▉  | 3173/4000 [3:43:21<57:55,  4.20s/it]
 79%|███████▉  | 3174/4000 [3:43:25<58:21,  4.24s/it]
 79%|███████▉  | 3175/4000 [3:43:29<58:16,  4.24s/it]
 79%|███████▉  | 3176/4000 [3:43:33<57:45,  4.21s/it]
 79%|███████▉  | 3177/4000 [3:43:38<58:03,  4.23s/it]
 79%|███████▉  | 3178/4000 [3:43:42<58:40,  4.28s/it]
 79%|███████▉  | 3179/4000 [3:43:47<59:05,  4.32s/it]
 80%|███████▉  | 3180/4000 [3:43:51<59:57,  4.39s/it]
                                                     
{'loss': '0.3915', 'grad_norm': '0.1346', 'learning_rate': '0.0001244', 'epoch': '0.7705'}

 80%|███████▉  | 3180/4000 [3:43:51<59:57,  4.39s/it]
 80%|███████▉  | 3181/4000 [3:43:55<58:53,  4.31s/it]
 80%|███████▉  | 3182/4000 [3:44:00<58:44,  4.31s/it]
 80%|███████▉  | 3183/4000 [3:44:04<58:30,  4.30s/it]
 80%|███████▉  | 3184/4000 [3:44:08<57:49,  4.25s/it]
 80%|███████▉  | 3185/4000 [3:44:12<57:17,  4.22s/it]
 80%|███████▉  | 3186/4000 [3:44:16<56:55,  4.20s/it]
 80%|███████▉  | 3187/4000 [3:44:21<57:54,  4.27s/it]
 80%|███████▉  | 3188/4000 [3:44:25<57:18,  4.23s/it]
 80%|███████▉  | 3189/4000 [3:44:29<56:50,  4.21s/it]
 80%|███████▉  | 3190/4000 [3:44:33<56:31,  4.19s/it]
                                                     
{'loss': '0.3685', 'grad_norm': '0.1206', 'learning_rate': '0.0001229', 'epoch': '0.7729'}

 80%|███████▉  | 3190/4000 [3:44:33<56:31,  4.19s/it]
 80%|███████▉  | 3191/4000 [3:44:37<56:21,  4.18s/it]
 80%|███████▉  | 3192/4000 [3:44:41<56:26,  4.19s/it]
 80%|███████▉  | 3193/4000 [3:44:46<56:06,  4.17s/it]
 80%|███████▉  | 3194/4000 [3:44:50<57:30,  4.28s/it]
 80%|███████▉  | 3195/4000 [3:44:54<57:13,  4.27s/it]
 80%|███████▉  | 3196/4000 [3:44:59<57:29,  4.29s/it]
 80%|███████▉  | 3197/4000 [3:45:03<57:10,  4.27s/it]
 80%|███████▉  | 3198/4000 [3:45:07<57:08,  4.27s/it]
 80%|███████▉  | 3199/4000 [3:45:12<57:12,  4.29s/it]
 80%|████████  | 3200/4000 [3:45:16<58:11,  4.36s/it]
                                                     
{'loss': '0.3733', 'grad_norm': '0.1379', 'learning_rate': '0.0001214', 'epoch': '0.7754'}

 80%|████████  | 3200/4000 [3:45:16<58:11,  4.36s/it]
 80%|████████  | 3201/4000 [3:45:20<57:10,  4.29s/it]
 80%|████████  | 3202/4000 [3:45:24<56:30,  4.25s/it]
 80%|████████  | 3203/4000 [3:45:29<56:00,  4.22s/it]
 80%|████████  | 3204/4000 [3:45:33<55:38,  4.19s/it]
 80%|████████  | 3205/4000 [3:45:37<55:21,  4.18s/it]
 80%|████████  | 3206/4000 [3:45:41<55:07,  4.17s/it]
 80%|████████  | 3207/4000 [3:45:45<56:24,  4.27s/it]
 80%|████████  | 3208/4000 [3:45:50<56:14,  4.26s/it]
 80%|████████  | 3209/4000 [3:45:54<55:54,  4.24s/it]
 80%|████████  | 3210/4000 [3:45:58<55:30,  4.22s/it]
                                                     
{'loss': '0.3659', 'grad_norm': '0.1027', 'learning_rate': '0.0001198', 'epoch': '0.7778'}

 80%|████████  | 3210/4000 [3:45:58<55:30,  4.22s/it]
 80%|████████  | 3211/4000 [3:46:02<55:08,  4.19s/it]
 80%|████████  | 3212/4000 [3:46:06<54:53,  4.18s/it]
 80%|████████  | 3213/4000 [3:46:11<54:49,  4.18s/it]
 80%|████████  | 3214/4000 [3:46:15<57:17,  4.37s/it]
 80%|████████  | 3215/4000 [3:46:20<57:21,  4.38s/it]
 80%|████████  | 3216/4000 [3:46:24<56:50,  4.35s/it]
 80%|████████  | 3217/4000 [3:46:28<56:03,  4.30s/it]
 80%|████████  | 3218/4000 [3:46:32<55:21,  4.25s/it]
 80%|████████  | 3219/4000 [3:46:36<54:53,  4.22s/it]
 80%|████████  | 3220/4000 [3:46:41<54:29,  4.19s/it]
                                                     
{'loss': '0.394', 'grad_norm': '0.1129', 'learning_rate': '0.0001183', 'epoch': '0.7802'}

 80%|████████  | 3220/4000 [3:46:41<54:29,  4.19s/it]
 81%|████████  | 3221/4000 [3:46:45<55:33,  4.28s/it]
 81%|████████  | 3222/4000 [3:46:49<54:56,  4.24s/it]
 81%|████████  | 3223/4000 [3:46:53<54:26,  4.20s/it]
 81%|████████  | 3224/4000 [3:46:58<54:48,  4.24s/it]
 81%|████████  | 3225/4000 [3:47:02<54:24,  4.21s/it]
 81%|████████  | 3226/4000 [3:47:06<54:03,  4.19s/it]
 81%|████████  | 3227/4000 [3:47:10<54:39,  4.24s/it]
 81%|████████  | 3228/4000 [3:47:15<54:48,  4.26s/it]
 81%|████████  | 3229/4000 [3:47:19<54:15,  4.22s/it]
 81%|████████  | 3230/4000 [3:47:23<54:12,  4.22s/it]
                                                     
{'loss': '0.4145', 'grad_norm': '0.1248', 'learning_rate': '0.0001168', 'epoch': '0.7826'}

 81%|████████  | 3230/4000 [3:47:23<54:12,  4.22s/it]
 81%|████████  | 3231/4000 [3:47:27<53:51,  4.20s/it]
 81%|████████  | 3232/4000 [3:47:31<53:46,  4.20s/it]
 81%|████████  | 3233/4000 [3:47:36<53:58,  4.22s/it]
 81%|████████  | 3234/4000 [3:47:40<55:46,  4.37s/it]
 81%|████████  | 3235/4000 [3:47:44<54:59,  4.31s/it]
 81%|████████  | 3236/4000 [3:47:49<54:13,  4.26s/it]
 81%|████████  | 3237/4000 [3:47:53<53:40,  4.22s/it]
 81%|████████  | 3238/4000 [3:47:57<53:14,  4.19s/it]
 81%|████████  | 3239/4000 [3:48:01<52:59,  4.18s/it]
 81%|████████  | 3240/4000 [3:48:05<53:56,  4.26s/it]
                                                     
{'loss': '0.3627', 'grad_norm': '0.1018', 'learning_rate': '0.0001153', 'epoch': '0.7851'}

 81%|████████  | 3240/4000 [3:48:05<53:56,  4.26s/it]
 81%|████████  | 3241/4000 [3:48:10<54:46,  4.33s/it]
 81%|████████  | 3242/4000 [3:48:14<53:55,  4.27s/it]
 81%|████████  | 3243/4000 [3:48:18<53:18,  4.23s/it]
 81%|████████  | 3244/4000 [3:48:22<52:51,  4.20s/it]
 81%|████████  | 3245/4000 [3:48:26<52:31,  4.17s/it]
 81%|████████  | 3246/4000 [3:48:31<52:15,  4.16s/it]
 81%|████████  | 3247/4000 [3:48:35<53:17,  4.25s/it]
 81%|████████  | 3248/4000 [3:48:39<53:31,  4.27s/it]
 81%|████████  | 3249/4000 [3:48:43<52:54,  4.23s/it]
 81%|████████▏ | 3250/4000 [3:48:48<52:27,  4.20s/it]
                                                     
{'loss': '0.3812', 'grad_norm': '0.1384', 'learning_rate': '0.0001138', 'epoch': '0.7875'}

 81%|████████▏ | 3250/4000 [3:48:48<52:27,  4.20s/it]
 81%|████████▏ | 3251/4000 [3:48:52<52:13,  4.18s/it]
 81%|████████▏ | 3252/4000 [3:48:56<52:31,  4.21s/it]
 81%|████████▏ | 3253/4000 [3:49:00<52:37,  4.23s/it]
 81%|████████▏ | 3254/4000 [3:49:05<54:16,  4.37s/it]
 81%|████████▏ | 3255/4000 [3:49:09<54:10,  4.36s/it]
 81%|████████▏ | 3256/4000 [3:49:14<53:19,  4.30s/it]
 81%|████████▏ | 3257/4000 [3:49:18<52:40,  4.25s/it]
 81%|████████▏ | 3258/4000 [3:49:22<52:07,  4.21s/it]
 81%|████████▏ | 3259/4000 [3:49:26<51:45,  4.19s/it]
 82%|████████▏ | 3260/4000 [3:49:30<51:30,  4.18s/it]
                                                     
{'loss': '0.3587', 'grad_norm': '0.1204', 'learning_rate': '0.0001123', 'epoch': '0.7899'}

 82%|████████▏ | 3260/4000 [3:49:30<51:30,  4.18s/it]
 82%|████████▏ | 3261/4000 [3:49:35<52:39,  4.27s/it]
 82%|████████▏ | 3262/4000 [3:49:39<52:01,  4.23s/it]
 82%|████████▏ | 3263/4000 [3:49:43<52:31,  4.28s/it]
 82%|████████▏ | 3264/4000 [3:49:47<51:59,  4.24s/it]
 82%|████████▏ | 3265/4000 [3:49:51<51:33,  4.21s/it]
 82%|████████▏ | 3266/4000 [3:49:56<51:14,  4.19s/it]
 82%|████████▏ | 3267/4000 [3:50:00<50:55,  4.17s/it]
 82%|████████▏ | 3268/4000 [3:50:04<52:04,  4.27s/it]
 82%|████████▏ | 3269/4000 [3:50:08<51:33,  4.23s/it]
 82%|████████▏ | 3270/4000 [3:50:12<51:11,  4.21s/it]
                                                     
{'loss': '0.3955', 'grad_norm': '0.1113', 'learning_rate': '0.0001108', 'epoch': '0.7923'}

 82%|████████▏ | 3270/4000 [3:50:12<51:11,  4.21s/it]
 82%|████████▏ | 3271/4000 [3:50:17<51:37,  4.25s/it]
 82%|████████▏ | 3272/4000 [3:50:21<51:37,  4.26s/it]
 82%|████████▏ | 3273/4000 [3:50:25<51:40,  4.26s/it]
 82%|████████▏ | 3274/4000 [3:50:30<52:38,  4.35s/it]
 82%|████████▏ | 3275/4000 [3:50:34<52:49,  4.37s/it]
 82%|████████▏ | 3276/4000 [3:50:38<51:53,  4.30s/it]
 82%|████████▏ | 3277/4000 [3:50:43<51:11,  4.25s/it]
 82%|████████▏ | 3278/4000 [3:50:47<50:41,  4.21s/it]
 82%|████████▏ | 3279/4000 [3:50:51<50:48,  4.23s/it]
 82%|████████▏ | 3280/4000 [3:50:55<50:26,  4.20s/it]
                                                     
{'loss': '0.3985', 'grad_norm': '0.123', 'learning_rate': '0.0001092', 'epoch': '0.7947'}

 82%|████████▏ | 3280/4000 [3:50:55<50:26,  4.20s/it]
 82%|████████▏ | 3281/4000 [3:51:00<51:29,  4.30s/it]
 82%|████████▏ | 3282/4000 [3:51:04<50:50,  4.25s/it]
 82%|████████▏ | 3283/4000 [3:51:08<50:19,  4.21s/it]
 82%|████████▏ | 3284/4000 [3:51:12<49:57,  4.19s/it]
 82%|████████▏ | 3285/4000 [3:51:16<49:39,  4.17s/it]
 82%|████████▏ | 3286/4000 [3:51:20<49:29,  4.16s/it]
 82%|████████▏ | 3287/4000 [3:51:24<49:20,  4.15s/it]
 82%|████████▏ | 3288/4000 [3:51:29<50:20,  4.24s/it]
 82%|████████▏ | 3289/4000 [3:51:33<49:51,  4.21s/it]
 82%|████████▏ | 3290/4000 [3:51:37<49:31,  4.19s/it]
                                                     
{'loss': '0.3756', 'grad_norm': '0.1178', 'learning_rate': '0.0001077', 'epoch': '0.7972'}

 82%|████████▏ | 3290/4000 [3:51:37<49:31,  4.19s/it]
 82%|████████▏ | 3291/4000 [3:51:41<50:02,  4.23s/it]
 82%|████████▏ | 3292/4000 [3:51:46<50:01,  4.24s/it]
 82%|████████▏ | 3293/4000 [3:51:50<50:10,  4.26s/it]
 82%|████████▏ | 3294/4000 [3:51:54<50:03,  4.25s/it]
 82%|████████▏ | 3295/4000 [3:51:59<51:35,  4.39s/it]
 82%|████████▏ | 3296/4000 [3:52:03<51:35,  4.40s/it]
 82%|████████▏ | 3297/4000 [3:52:08<50:37,  4.32s/it]
 82%|████████▏ | 3298/4000 [3:52:12<49:55,  4.27s/it]
 82%|████████▏ | 3299/4000 [3:52:16<49:25,  4.23s/it]
 82%|████████▎ | 3300/4000 [3:52:20<49:01,  4.20s/it]
                                                     
{'loss': '0.3911', 'grad_norm': '0.1171', 'learning_rate': '0.0001062', 'epoch': '0.7996'}

 82%|████████▎ | 3300/4000 [3:52:20<49:01,  4.20s/it]
 83%|████████▎ | 3301/4000 [3:52:24<49:31,  4.25s/it]
 83%|████████▎ | 3302/4000 [3:52:29<49:41,  4.27s/it]
 83%|████████▎ | 3303/4000 [3:52:33<49:09,  4.23s/it]
 83%|████████▎ | 3304/4000 [3:52:37<48:47,  4.21s/it]
 83%|████████▎ | 3305/4000 [3:52:41<48:28,  4.18s/it]
 83%|████████▎ | 3306/4000 [3:52:45<48:15,  4.17s/it]
 83%|████████▎ | 3307/4000 [3:52:49<48:04,  4.16s/it]
 83%|████████▎ | 3308/4000 [3:52:54<49:11,  4.27s/it]
 83%|████████▎ | 3309/4000 [3:52:58<48:45,  4.23s/it]
 83%|████████▎ | 3310/4000 [3:53:02<48:39,  4.23s/it]
                                                     
{'loss': '0.4119', 'grad_norm': '0.1113', 'learning_rate': '0.0001047', 'epoch': '0.802'}

 83%|████████▎ | 3310/4000 [3:53:02<48:39,  4.23s/it]
 83%|████████▎ | 3311/4000 [3:53:07<48:57,  4.26s/it]
 83%|████████▎ | 3312/4000 [3:53:11<48:41,  4.25s/it]
 83%|████████▎ | 3313/4000 [3:53:15<48:41,  4.25s/it]
 83%|████████▎ | 3314/4000 [3:53:19<48:37,  4.25s/it]
 83%|████████▎ | 3315/4000 [3:53:24<50:10,  4.40s/it]
 83%|████████▎ | 3316/4000 [3:53:28<49:27,  4.34s/it]
 83%|████████▎ | 3317/4000 [3:53:32<48:40,  4.28s/it]
 83%|████████▎ | 3318/4000 [3:53:36<48:04,  4.23s/it]
 83%|████████▎ | 3319/4000 [3:53:41<47:38,  4.20s/it]
 83%|████████▎ | 3320/4000 [3:53:45<47:19,  4.18s/it]
                                                     
{'loss': '0.3591', 'grad_norm': '0.1191', 'learning_rate': '0.0001032', 'epoch': '0.8044'}

 83%|████████▎ | 3320/4000 [3:53:45<47:19,  4.18s/it]
 83%|████████▎ | 3321/4000 [3:53:49<47:05,  4.16s/it]
 83%|████████▎ | 3322/4000 [3:53:53<48:06,  4.26s/it]
 83%|████████▎ | 3323/4000 [3:53:57<47:38,  4.22s/it]
 83%|████████▎ | 3324/4000 [3:54:02<47:13,  4.19s/it]
 83%|████████▎ | 3325/4000 [3:54:06<46:58,  4.17s/it]
 83%|████████▎ | 3326/4000 [3:54:10<46:47,  4.16s/it]
 83%|████████▎ | 3327/4000 [3:54:14<46:33,  4.15s/it]
 83%|████████▎ | 3328/4000 [3:54:18<47:38,  4.25s/it]
 83%|████████▎ | 3329/4000 [3:54:23<47:56,  4.29s/it]
 83%|████████▎ | 3330/4000 [3:54:27<47:31,  4.26s/it]
                                                     
{'loss': '0.3953', 'grad_norm': '0.1186', 'learning_rate': '0.0001017', 'epoch': '0.8069'}

 83%|████████▎ | 3330/4000 [3:54:27<47:31,  4.26s/it]
 83%|████████▎ | 3331/4000 [3:54:31<47:04,  4.22s/it]
 83%|████████▎ | 3332/4000 [3:54:36<47:26,  4.26s/it]
 83%|████████▎ | 3333/4000 [3:54:40<47:27,  4.27s/it]
 83%|████████▎ | 3334/4000 [3:54:44<47:18,  4.26s/it]
 83%|████████▎ | 3335/4000 [3:54:49<48:50,  4.41s/it]
 83%|████████▎ | 3336/4000 [3:54:53<48:06,  4.35s/it]
 83%|████████▎ | 3337/4000 [3:54:57<47:21,  4.29s/it]
 83%|████████▎ | 3338/4000 [3:55:01<46:46,  4.24s/it]
 83%|████████▎ | 3339/4000 [3:55:05<46:20,  4.21s/it]
 84%|████████▎ | 3340/4000 [3:55:10<46:03,  4.19s/it]
                                                     
{'loss': '0.3978', 'grad_norm': '0.1635', 'learning_rate': '0.0001002', 'epoch': '0.8093'}

 84%|████████▎ | 3340/4000 [3:55:10<46:03,  4.19s/it]
 84%|████████▎ | 3341/4000 [3:55:14<45:49,  4.17s/it]
 84%|████████▎ | 3342/4000 [3:55:18<46:55,  4.28s/it]
 84%|████████▎ | 3343/4000 [3:55:22<46:27,  4.24s/it]
 84%|████████▎ | 3344/4000 [3:55:27<46:04,  4.21s/it]
 84%|████████▎ | 3345/4000 [3:55:31<45:44,  4.19s/it]
 84%|████████▎ | 3346/4000 [3:55:35<45:52,  4.21s/it]
 84%|████████▎ | 3347/4000 [3:55:39<45:58,  4.22s/it]
 84%|████████▎ | 3348/4000 [3:55:43<45:38,  4.20s/it]
 84%|████████▎ | 3349/4000 [3:55:48<46:34,  4.29s/it]
 84%|████████▍ | 3350/4000 [3:55:52<46:26,  4.29s/it]
                                                     
{'loss': '0.3527', 'grad_norm': '0.1133', 'learning_rate': '9.864e-05', 'epoch': '0.8117'}

 84%|████████▍ | 3350/4000 [3:55:52<46:26,  4.29s/it]
 84%|████████▍ | 3351/4000 [3:55:56<45:54,  4.24s/it]
 84%|████████▍ | 3352/4000 [3:56:00<45:45,  4.24s/it]
 84%|████████▍ | 3353/4000 [3:56:05<45:41,  4.24s/it]
 84%|████████▍ | 3354/4000 [3:56:09<46:19,  4.30s/it]
 84%|████████▍ | 3355/4000 [3:56:14<47:00,  4.37s/it]
 84%|████████▍ | 3356/4000 [3:56:18<46:09,  4.30s/it]
 84%|████████▍ | 3357/4000 [3:56:22<45:51,  4.28s/it]
 84%|████████▍ | 3358/4000 [3:56:26<45:15,  4.23s/it]
 84%|████████▍ | 3359/4000 [3:56:30<44:54,  4.20s/it]
 84%|████████▍ | 3360/4000 [3:56:34<44:37,  4.18s/it]
                                                     
{'loss': '0.4003', 'grad_norm': '0.1286', 'learning_rate': '9.712e-05', 'epoch': '0.8141'}

 84%|████████▍ | 3360/4000 [3:56:34<44:37,  4.18s/it]
 84%|████████▍ | 3361/4000 [3:56:39<44:22,  4.17s/it]
 84%|████████▍ | 3362/4000 [3:56:43<45:26,  4.27s/it]
 84%|████████▍ | 3363/4000 [3:56:47<45:37,  4.30s/it]
 84%|████████▍ | 3364/4000 [3:56:52<45:10,  4.26s/it]
 84%|████████▍ | 3365/4000 [3:56:56<45:03,  4.26s/it]
 84%|████████▍ | 3366/4000 [3:57:00<44:36,  4.22s/it]
 84%|████████▍ | 3367/4000 [3:57:04<44:29,  4.22s/it]
 84%|████████▍ | 3368/4000 [3:57:08<44:17,  4.20s/it]
 84%|████████▍ | 3369/4000 [3:57:13<45:54,  4.36s/it]
 84%|████████▍ | 3370/4000 [3:57:17<45:09,  4.30s/it]
                                                     
{'loss': '0.3655', 'grad_norm': '0.1472', 'learning_rate': '9.561e-05', 'epoch': '0.8165'}

 84%|████████▍ | 3370/4000 [3:57:17<45:09,  4.30s/it]
 84%|████████▍ | 3371/4000 [3:57:21<44:34,  4.25s/it]
 84%|████████▍ | 3372/4000 [3:57:26<45:06,  4.31s/it]
 84%|████████▍ | 3373/4000 [3:57:30<44:32,  4.26s/it]
 84%|████████▍ | 3374/4000 [3:57:34<44:03,  4.22s/it]
 84%|████████▍ | 3375/4000 [3:57:38<44:20,  4.26s/it]
 84%|████████▍ | 3376/4000 [3:57:43<44:24,  4.27s/it]
 84%|████████▍ | 3377/4000 [3:57:47<43:53,  4.23s/it]
 84%|████████▍ | 3378/4000 [3:57:51<43:29,  4.20s/it]
 84%|████████▍ | 3379/4000 [3:57:55<43:11,  4.17s/it]
 84%|████████▍ | 3380/4000 [3:57:59<43:12,  4.18s/it]
                                                     
{'loss': '0.3818', 'grad_norm': '0.1262', 'learning_rate': '9.409e-05', 'epoch': '0.819'}

 84%|████████▍ | 3380/4000 [3:57:59<43:12,  4.18s/it]
 85%|████████▍ | 3381/4000 [3:58:04<43:27,  4.21s/it]
 85%|████████▍ | 3382/4000 [3:58:08<44:49,  4.35s/it]
 85%|████████▍ | 3383/4000 [3:58:12<44:04,  4.29s/it]
 85%|████████▍ | 3384/4000 [3:58:17<43:54,  4.28s/it]
 85%|████████▍ | 3385/4000 [3:58:21<43:45,  4.27s/it]
 85%|████████▍ | 3386/4000 [3:58:25<43:17,  4.23s/it]
 85%|████████▍ | 3387/4000 [3:58:29<42:57,  4.20s/it]
 85%|████████▍ | 3388/4000 [3:58:33<42:38,  4.18s/it]
 85%|████████▍ | 3389/4000 [3:58:38<43:46,  4.30s/it]
 85%|████████▍ | 3390/4000 [3:58:42<44:04,  4.33s/it]
                                                     
{'loss': '0.3634', 'grad_norm': '0.1173', 'learning_rate': '9.258e-05', 'epoch': '0.8214'}

 85%|████████▍ | 3390/4000 [3:58:42<44:04,  4.33s/it]
 85%|████████▍ | 3391/4000 [3:58:47<43:28,  4.28s/it]
 85%|████████▍ | 3392/4000 [3:58:51<42:57,  4.24s/it]
 85%|████████▍ | 3393/4000 [3:58:55<42:32,  4.20s/it]
 85%|████████▍ | 3394/4000 [3:58:59<42:14,  4.18s/it]
 85%|████████▍ | 3395/4000 [3:59:03<42:00,  4.17s/it]
 85%|████████▍ | 3396/4000 [3:59:08<43:02,  4.28s/it]
 85%|████████▍ | 3397/4000 [3:59:12<42:32,  4.23s/it]
 85%|████████▍ | 3398/4000 [3:59:16<42:28,  4.23s/it]
 85%|████████▍ | 3399/4000 [3:59:20<42:29,  4.24s/it]
 85%|████████▌ | 3400/4000 [3:59:24<42:13,  4.22s/it]
                                                     
{'loss': '0.3955', 'grad_norm': '0.114', 'learning_rate': '9.106e-05', 'epoch': '0.8238'}

 85%|████████▌ | 3400/4000 [3:59:24<42:13,  4.22s/it]
 85%|████████▌ | 3401/4000 [3:59:29<42:31,  4.26s/it]
 85%|████████▌ | 3402/4000 [3:59:33<43:14,  4.34s/it]
 85%|████████▌ | 3403/4000 [3:59:38<43:12,  4.34s/it]
 85%|████████▌ | 3404/4000 [3:59:42<42:28,  4.28s/it]
 85%|████████▌ | 3405/4000 [3:59:46<41:58,  4.23s/it]
 85%|████████▌ | 3406/4000 [3:59:50<41:35,  4.20s/it]
 85%|████████▌ | 3407/4000 [3:59:54<42:15,  4.28s/it]
 85%|████████▌ | 3408/4000 [3:59:59<42:01,  4.26s/it]
 85%|████████▌ | 3409/4000 [4:00:03<42:42,  4.34s/it]
 85%|████████▌ | 3410/4000 [4:00:07<42:04,  4.28s/it]
                                                     
{'loss': '0.3678', 'grad_norm': '0.1184', 'learning_rate': '8.955e-05', 'epoch': '0.8262'}

 85%|████████▌ | 3410/4000 [4:00:07<42:04,  4.28s/it]
 85%|████████▌ | 3411/4000 [4:00:11<41:33,  4.23s/it]
 85%|████████▌ | 3412/4000 [4:00:16<41:10,  4.20s/it]
 85%|████████▌ | 3413/4000 [4:00:20<40:55,  4.18s/it]
 85%|████████▌ | 3414/4000 [4:00:24<40:39,  4.16s/it]
 85%|████████▌ | 3415/4000 [4:00:28<41:11,  4.23s/it]
 85%|████████▌ | 3416/4000 [4:00:33<42:52,  4.41s/it]
 85%|████████▌ | 3417/4000 [4:00:37<42:06,  4.33s/it]
 85%|████████▌ | 3418/4000 [4:00:42<42:05,  4.34s/it]
 85%|████████▌ | 3419/4000 [4:00:46<41:36,  4.30s/it]
 86%|████████▌ | 3420/4000 [4:00:50<41:00,  4.24s/it]
                                                     
{'loss': '0.376', 'grad_norm': '0.1444', 'learning_rate': '8.803e-05', 'epoch': '0.8287'}

 86%|████████▌ | 3420/4000 [4:00:50<41:00,  4.24s/it]
 86%|████████▌ | 3421/4000 [4:00:54<40:40,  4.21s/it]
 86%|████████▌ | 3422/4000 [4:00:58<40:23,  4.19s/it]
 86%|████████▌ | 3423/4000 [4:01:03<41:07,  4.28s/it]
 86%|████████▌ | 3424/4000 [4:01:07<41:25,  4.32s/it]
 86%|████████▌ | 3425/4000 [4:01:11<41:20,  4.31s/it]
 86%|████████▌ | 3426/4000 [4:01:15<40:46,  4.26s/it]
 86%|████████▌ | 3427/4000 [4:01:20<40:19,  4.22s/it]
 86%|████████▌ | 3428/4000 [4:01:24<39:58,  4.19s/it]
 86%|████████▌ | 3429/4000 [4:01:28<40:17,  4.23s/it]
 86%|████████▌ | 3430/4000 [4:01:32<40:25,  4.26s/it]
                                                     
{'loss': '0.397', 'grad_norm': '0.1081', 'learning_rate': '8.652e-05', 'epoch': '0.8311'}

 86%|████████▌ | 3430/4000 [4:01:32<40:25,  4.26s/it]
 86%|████████▌ | 3431/4000 [4:01:36<39:59,  4.22s/it]
 86%|████████▌ | 3432/4000 [4:01:41<40:25,  4.27s/it]
 86%|████████▌ | 3433/4000 [4:01:45<40:32,  4.29s/it]
 86%|████████▌ | 3434/4000 [4:01:50<40:40,  4.31s/it]
 86%|████████▌ | 3435/4000 [4:01:54<40:22,  4.29s/it]
 86%|████████▌ | 3436/4000 [4:01:58<41:18,  4.39s/it]
 86%|████████▌ | 3437/4000 [4:02:03<40:28,  4.31s/it]
 86%|████████▌ | 3438/4000 [4:02:07<39:51,  4.26s/it]
 86%|████████▌ | 3439/4000 [4:02:11<39:25,  4.22s/it]
 86%|████████▌ | 3440/4000 [4:02:15<39:05,  4.19s/it]
                                                     
{'loss': '0.3547', 'grad_norm': '0.1245', 'learning_rate': '8.5e-05', 'epoch': '0.8335'}

 86%|████████▌ | 3440/4000 [4:02:15<39:05,  4.19s/it]
 86%|████████▌ | 3441/4000 [4:02:19<39:03,  4.19s/it]
 86%|████████▌ | 3442/4000 [4:02:23<38:55,  4.19s/it]
 86%|████████▌ | 3443/4000 [4:02:28<39:50,  4.29s/it]
 86%|████████▌ | 3444/4000 [4:02:32<39:16,  4.24s/it]
 86%|████████▌ | 3445/4000 [4:02:36<38:52,  4.20s/it]
 86%|████████▌ | 3446/4000 [4:02:40<38:36,  4.18s/it]
 86%|████████▌ | 3447/4000 [4:02:44<38:22,  4.16s/it]
 86%|████████▌ | 3448/4000 [4:02:48<38:10,  4.15s/it]
 86%|████████▌ | 3449/4000 [4:02:53<38:31,  4.20s/it]
 86%|████████▋ | 3450/4000 [4:02:57<39:25,  4.30s/it]
                                                     
{'loss': '0.4024', 'grad_norm': '0.117', 'learning_rate': '8.348e-05', 'epoch': '0.8359'}

 86%|████████▋ | 3450/4000 [4:02:57<39:25,  4.30s/it]
 86%|████████▋ | 3451/4000 [4:03:02<39:43,  4.34s/it]
 86%|████████▋ | 3452/4000 [4:03:06<39:45,  4.35s/it]
 86%|████████▋ | 3453/4000 [4:03:10<39:05,  4.29s/it]
 86%|████████▋ | 3454/4000 [4:03:14<38:36,  4.24s/it]
 86%|████████▋ | 3455/4000 [4:03:19<38:12,  4.21s/it]
 86%|████████▋ | 3456/4000 [4:03:23<38:21,  4.23s/it]
 86%|████████▋ | 3457/4000 [4:03:27<38:27,  4.25s/it]
 86%|████████▋ | 3458/4000 [4:03:31<38:32,  4.27s/it]
 86%|████████▋ | 3459/4000 [4:03:36<38:25,  4.26s/it]
 86%|████████▋ | 3460/4000 [4:03:40<38:10,  4.24s/it]
                                                     
{'loss': '0.3786', 'grad_norm': '0.1311', 'learning_rate': '8.197e-05', 'epoch': '0.8384'}

 86%|████████▋ | 3460/4000 [4:03:40<38:10,  4.24s/it]
 87%|████████▋ | 3461/4000 [4:03:44<37:49,  4.21s/it]
 87%|████████▋ | 3462/4000 [4:03:48<37:33,  4.19s/it]
 87%|████████▋ | 3463/4000 [4:03:53<38:20,  4.28s/it]
 87%|████████▋ | 3464/4000 [4:03:57<37:50,  4.24s/it]
 87%|████████▋ | 3465/4000 [4:04:01<37:29,  4.20s/it]
 87%|████████▋ | 3466/4000 [4:04:05<37:25,  4.20s/it]
 87%|████████▋ | 3467/4000 [4:04:09<37:26,  4.21s/it]
 87%|████████▋ | 3468/4000 [4:04:14<37:29,  4.23s/it]
 87%|████████▋ | 3469/4000 [4:04:18<37:45,  4.27s/it]
 87%|████████▋ | 3470/4000 [4:04:23<38:33,  4.37s/it]
                                                     
{'loss': '0.3573', 'grad_norm': '0.1071', 'learning_rate': '8.045e-05', 'epoch': '0.8408'}

 87%|████████▋ | 3470/4000 [4:04:23<38:33,  4.37s/it]
 87%|████████▋ | 3471/4000 [4:04:27<37:51,  4.29s/it]
 87%|████████▋ | 3472/4000 [4:04:31<37:23,  4.25s/it]
 87%|████████▋ | 3473/4000 [4:04:35<37:00,  4.21s/it]
 87%|████████▋ | 3474/4000 [4:04:39<36:43,  4.19s/it]
 87%|████████▋ | 3475/4000 [4:04:43<37:10,  4.25s/it]
 87%|████████▋ | 3476/4000 [4:04:48<36:50,  4.22s/it]
 87%|████████▋ | 3477/4000 [4:04:52<37:55,  4.35s/it]
 87%|████████▋ | 3478/4000 [4:04:56<37:24,  4.30s/it]
 87%|████████▋ | 3479/4000 [4:05:01<36:54,  4.25s/it]
 87%|████████▋ | 3480/4000 [4:05:05<36:30,  4.21s/it]
                                                     
{'loss': '0.3821', 'grad_norm': '0.1146', 'learning_rate': '7.894e-05', 'epoch': '0.8432'}

 87%|████████▋ | 3480/4000 [4:05:05<36:30,  4.21s/it]
 87%|████████▋ | 3481/4000 [4:05:09<36:14,  4.19s/it]
 87%|████████▋ | 3482/4000 [4:05:13<36:00,  4.17s/it]
 87%|████████▋ | 3483/4000 [4:05:18<37:12,  4.32s/it]
 87%|████████▋ | 3484/4000 [4:05:22<36:48,  4.28s/it]
 87%|████████▋ | 3485/4000 [4:05:26<36:48,  4.29s/it]
 87%|████████▋ | 3486/4000 [4:05:30<36:44,  4.29s/it]
 87%|████████▋ | 3487/4000 [4:05:35<36:31,  4.27s/it]
 87%|████████▋ | 3488/4000 [4:05:39<36:07,  4.23s/it]
 87%|████████▋ | 3489/4000 [4:05:43<35:49,  4.21s/it]
 87%|████████▋ | 3490/4000 [4:05:47<36:29,  4.29s/it]
                                                     
{'loss': '0.4101', 'grad_norm': '0.1131', 'learning_rate': '7.742e-05', 'epoch': '0.8456'}

 87%|████████▋ | 3490/4000 [4:05:47<36:29,  4.29s/it]
 87%|████████▋ | 3491/4000 [4:05:52<36:01,  4.25s/it]
 87%|████████▋ | 3492/4000 [4:05:56<35:54,  4.24s/it]
 87%|████████▋ | 3493/4000 [4:06:00<35:32,  4.21s/it]
 87%|████████▋ | 3494/4000 [4:06:04<35:40,  4.23s/it]
 87%|████████▋ | 3495/4000 [4:06:08<35:22,  4.20s/it]
 87%|████████▋ | 3496/4000 [4:06:13<35:09,  4.19s/it]
 87%|████████▋ | 3497/4000 [4:06:17<35:53,  4.28s/it]
 87%|████████▋ | 3498/4000 [4:06:21<35:25,  4.23s/it]
 87%|████████▋ | 3499/4000 [4:06:25<35:05,  4.20s/it]
 88%|████████▊ | 3500/4000 [4:06:29<34:50,  4.18s/it]
                                                     
{'loss': '0.375', 'grad_norm': '0.121', 'learning_rate': '7.591e-05', 'epoch': '0.848'}

 88%|████████▊ | 3500/4000 [4:06:29<34:50,  4.18s/it]
 88%|████████▊ | 3501/4000 [4:06:34<35:24,  4.26s/it]
 88%|████████▊ | 3502/4000 [4:06:38<35:30,  4.28s/it]
 88%|████████▊ | 3503/4000 [4:06:43<35:38,  4.30s/it]
 88%|████████▊ | 3504/4000 [4:06:47<36:18,  4.39s/it]
 88%|████████▊ | 3505/4000 [4:06:51<35:40,  4.32s/it]
 88%|████████▊ | 3506/4000 [4:06:55<35:05,  4.26s/it]
 88%|████████▊ | 3507/4000 [4:07:00<34:41,  4.22s/it]
 88%|████████▊ | 3508/4000 [4:07:04<34:44,  4.24s/it]
 88%|████████▊ | 3509/4000 [4:07:08<34:45,  4.25s/it]
 88%|████████▊ | 3510/4000 [4:07:13<35:17,  4.32s/it]
                                                     
{'loss': '0.3447', 'grad_norm': '0.1257', 'learning_rate': '7.439e-05', 'epoch': '0.8505'}

 88%|████████▊ | 3510/4000 [4:07:13<35:17,  4.32s/it]
 88%|████████▊ | 3511/4000 [4:07:17<34:53,  4.28s/it]
 88%|████████▊ | 3512/4000 [4:07:21<34:27,  4.24s/it]
 88%|████████▊ | 3513/4000 [4:07:25<34:06,  4.20s/it]
 88%|████████▊ | 3514/4000 [4:07:29<33:49,  4.18s/it]
 88%|████████▊ | 3515/4000 [4:07:33<33:36,  4.16s/it]
 88%|████████▊ | 3516/4000 [4:07:37<33:29,  4.15s/it]
 88%|████████▊ | 3517/4000 [4:07:42<34:15,  4.26s/it]
 88%|████████▊ | 3518/4000 [4:07:46<34:06,  4.25s/it]
 88%|████████▊ | 3519/4000 [4:07:50<34:03,  4.25s/it]
 88%|████████▊ | 3520/4000 [4:07:55<33:51,  4.23s/it]
                                                     
{'loss': '0.3932', 'grad_norm': '0.1423', 'learning_rate': '7.288e-05', 'epoch': '0.8529'}

 88%|████████▊ | 3520/4000 [4:07:55<33:51,  4.23s/it]
 88%|████████▊ | 3521/4000 [4:07:59<34:19,  4.30s/it]
 88%|████████▊ | 3522/4000 [4:08:03<33:59,  4.27s/it]
 88%|████████▊ | 3523/4000 [4:08:07<33:36,  4.23s/it]
 88%|████████▊ | 3524/4000 [4:08:12<34:05,  4.30s/it]
 88%|████████▊ | 3525/4000 [4:08:16<34:13,  4.32s/it]
 88%|████████▊ | 3526/4000 [4:08:20<33:45,  4.27s/it]
 88%|████████▊ | 3527/4000 [4:08:24<33:21,  4.23s/it]
 88%|████████▊ | 3528/4000 [4:08:29<33:05,  4.21s/it]
 88%|████████▊ | 3529/4000 [4:08:33<33:00,  4.20s/it]
 88%|████████▊ | 3530/4000 [4:08:37<32:45,  4.18s/it]
                                                     
{'loss': '0.4123', 'grad_norm': '0.1177', 'learning_rate': '7.136e-05', 'epoch': '0.8553'}

 88%|████████▊ | 3530/4000 [4:08:37<32:45,  4.18s/it]
 88%|████████▊ | 3531/4000 [4:08:41<33:28,  4.28s/it]
 88%|████████▊ | 3532/4000 [4:08:46<33:00,  4.23s/it]
 88%|████████▊ | 3533/4000 [4:08:50<32:43,  4.20s/it]
 88%|████████▊ | 3534/4000 [4:08:54<32:30,  4.18s/it]
 88%|████████▊ | 3535/4000 [4:08:58<32:33,  4.20s/it]
 88%|████████▊ | 3536/4000 [4:09:02<32:22,  4.19s/it]
 88%|████████▊ | 3537/4000 [4:09:07<33:28,  4.34s/it]
 88%|████████▊ | 3538/4000 [4:09:11<33:29,  4.35s/it]
 88%|████████▊ | 3539/4000 [4:09:16<33:14,  4.33s/it]
 88%|████████▊ | 3540/4000 [4:09:20<32:44,  4.27s/it]
                                                     
{'loss': '0.361', 'grad_norm': '0.1338', 'learning_rate': '6.985e-05', 'epoch': '0.8577'}

 88%|████████▊ | 3540/4000 [4:09:20<32:44,  4.27s/it]
 89%|████████▊ | 3541/4000 [4:09:24<32:22,  4.23s/it]
 89%|████████▊ | 3542/4000 [4:09:28<32:22,  4.24s/it]
 89%|████████▊ | 3543/4000 [4:09:32<32:04,  4.21s/it]
 89%|████████▊ | 3544/4000 [4:09:37<32:39,  4.30s/it]
 89%|████████▊ | 3545/4000 [4:09:41<32:13,  4.25s/it]
 89%|████████▊ | 3546/4000 [4:09:45<32:12,  4.26s/it]
 89%|████████▊ | 3547/4000 [4:09:49<31:51,  4.22s/it]
 89%|████████▊ | 3548/4000 [4:09:53<31:34,  4.19s/it]
 89%|████████▊ | 3549/4000 [4:09:58<31:21,  4.17s/it]
 89%|████████▉ | 3550/4000 [4:10:02<31:09,  4.15s/it]
                                                     
{'loss': '0.3551', 'grad_norm': '0.1287', 'learning_rate': '6.833e-05', 'epoch': '0.8602'}

 89%|████████▉ | 3550/4000 [4:10:02<31:09,  4.15s/it]
 89%|████████▉ | 3551/4000 [4:10:06<31:55,  4.27s/it]
 89%|████████▉ | 3552/4000 [4:10:11<31:58,  4.28s/it]
 89%|████████▉ | 3553/4000 [4:10:15<31:34,  4.24s/it]
 89%|████████▉ | 3554/4000 [4:10:19<31:34,  4.25s/it]
 89%|████████▉ | 3555/4000 [4:10:23<31:50,  4.29s/it]
 89%|████████▉ | 3556/4000 [4:10:28<31:36,  4.27s/it]
 89%|████████▉ | 3557/4000 [4:10:32<31:35,  4.28s/it]
 89%|████████▉ | 3558/4000 [4:10:36<31:32,  4.28s/it]
 89%|████████▉ | 3559/4000 [4:10:40<31:27,  4.28s/it]
 89%|████████▉ | 3560/4000 [4:10:45<31:05,  4.24s/it]
                                                     
{'loss': '0.3892', 'grad_norm': '0.1238', 'learning_rate': '6.682e-05', 'epoch': '0.8626'}

 89%|████████▉ | 3560/4000 [4:10:45<31:05,  4.24s/it]
 89%|████████▉ | 3561/4000 [4:10:49<30:45,  4.20s/it]
 89%|████████▉ | 3562/4000 [4:10:53<30:30,  4.18s/it]
 89%|████████▉ | 3563/4000 [4:10:57<30:25,  4.18s/it]
 89%|████████▉ | 3564/4000 [4:11:01<31:03,  4.27s/it]
 89%|████████▉ | 3565/4000 [4:11:06<30:41,  4.23s/it]
 89%|████████▉ | 3566/4000 [4:11:10<30:25,  4.21s/it]
 89%|████████▉ | 3567/4000 [4:11:14<30:11,  4.18s/it]
 89%|████████▉ | 3568/4000 [4:11:18<30:00,  4.17s/it]
 89%|████████▉ | 3569/4000 [4:11:22<30:07,  4.19s/it]
 89%|████████▉ | 3570/4000 [4:11:26<29:56,  4.18s/it]
                                                     
{'loss': '0.3515', 'grad_norm': '0.1203', 'learning_rate': '6.53e-05', 'epoch': '0.865'}

 89%|████████▉ | 3570/4000 [4:11:26<29:56,  4.18s/it]
 89%|████████▉ | 3571/4000 [4:11:31<30:57,  4.33s/it]
 89%|████████▉ | 3572/4000 [4:11:35<30:59,  4.35s/it]
 89%|████████▉ | 3573/4000 [4:11:40<30:40,  4.31s/it]
 89%|████████▉ | 3574/4000 [4:11:44<30:17,  4.27s/it]
 89%|████████▉ | 3575/4000 [4:11:48<29:54,  4.22s/it]
 89%|████████▉ | 3576/4000 [4:11:52<29:50,  4.22s/it]
 89%|████████▉ | 3577/4000 [4:11:56<29:37,  4.20s/it]
 89%|████████▉ | 3578/4000 [4:12:01<30:10,  4.29s/it]
 89%|████████▉ | 3579/4000 [4:12:05<29:44,  4.24s/it]
 90%|████████▉ | 3580/4000 [4:12:09<29:32,  4.22s/it]
                                                     
{'loss': '0.3733', 'grad_norm': '0.1103', 'learning_rate': '6.379e-05', 'epoch': '0.8674'}

 90%|████████▉ | 3580/4000 [4:12:09<29:32,  4.22s/it]
 90%|████████▉ | 3581/4000 [4:12:13<29:20,  4.20s/it]
 90%|████████▉ | 3582/4000 [4:12:17<29:06,  4.18s/it]
 90%|████████▉ | 3583/4000 [4:12:22<28:57,  4.17s/it]
 90%|████████▉ | 3584/4000 [4:12:26<29:12,  4.21s/it]
 90%|████████▉ | 3585/4000 [4:12:30<29:20,  4.24s/it]
 90%|████████▉ | 3586/4000 [4:12:35<29:38,  4.30s/it]
 90%|████████▉ | 3587/4000 [4:12:39<29:19,  4.26s/it]
 90%|████████▉ | 3588/4000 [4:12:43<29:27,  4.29s/it]
 90%|████████▉ | 3589/4000 [4:12:47<29:14,  4.27s/it]
 90%|████████▉ | 3590/4000 [4:12:52<29:03,  4.25s/it]
                                                     
{'loss': '0.3689', 'grad_norm': '0.1228', 'learning_rate': '6.227e-05', 'epoch': '0.8699'}

 90%|████████▉ | 3590/4000 [4:12:52<29:03,  4.25s/it]
 90%|████████▉ | 3591/4000 [4:12:56<29:50,  4.38s/it]
 90%|████████▉ | 3592/4000 [4:13:01<29:30,  4.34s/it]
 90%|████████▉ | 3593/4000 [4:13:05<29:19,  4.32s/it]
 90%|████████▉ | 3594/4000 [4:13:09<28:56,  4.28s/it]
 90%|████████▉ | 3595/4000 [4:13:13<28:35,  4.24s/it]
 90%|████████▉ | 3596/4000 [4:13:17<28:22,  4.21s/it]
 90%|████████▉ | 3597/4000 [4:13:21<28:15,  4.21s/it]
 90%|████████▉ | 3598/4000 [4:13:26<28:48,  4.30s/it]
 90%|████████▉ | 3599/4000 [4:13:30<28:23,  4.25s/it]
 90%|█████████ | 3600/4000 [4:13:34<28:04,  4.21s/it]
                                                     
{'loss': '0.4017', 'grad_norm': '0.1289', 'learning_rate': '6.076e-05', 'epoch': '0.8723'}

 90%|█████████ | 3600/4000 [4:13:34<28:04,  4.21s/it]
 90%|█████████ | 3601/4000 [4:13:38<27:49,  4.18s/it]
 90%|█████████ | 3602/4000 [4:13:42<27:38,  4.17s/it]
 90%|█████████ | 3603/4000 [4:13:47<27:36,  4.17s/it]
 90%|█████████ | 3604/4000 [4:13:51<27:31,  4.17s/it]
 90%|█████████ | 3605/4000 [4:13:56<28:26,  4.32s/it]
 90%|█████████ | 3606/4000 [4:14:00<28:29,  4.34s/it]
 90%|█████████ | 3607/4000 [4:14:04<28:05,  4.29s/it]
 90%|█████████ | 3608/4000 [4:14:08<28:16,  4.33s/it]
 90%|█████████ | 3609/4000 [4:14:13<27:50,  4.27s/it]
 90%|█████████ | 3610/4000 [4:14:17<27:46,  4.27s/it]
                                                     
{'loss': '0.367', 'grad_norm': '0.1241', 'learning_rate': '5.924e-05', 'epoch': '0.8747'}

 90%|█████████ | 3610/4000 [4:14:17<27:46,  4.27s/it]
 90%|█████████ | 3611/4000 [4:14:21<28:10,  4.35s/it]
 90%|█████████ | 3612/4000 [4:14:26<27:41,  4.28s/it]
 90%|█████████ | 3613/4000 [4:14:30<27:18,  4.23s/it]
 90%|█████████ | 3614/4000 [4:14:34<27:11,  4.23s/it]
 90%|█████████ | 3615/4000 [4:14:38<26:59,  4.21s/it]
 90%|█████████ | 3616/4000 [4:14:42<26:46,  4.18s/it]
 90%|█████████ | 3617/4000 [4:14:46<26:35,  4.17s/it]
 90%|█████████ | 3618/4000 [4:14:51<27:10,  4.27s/it]
 90%|█████████ | 3619/4000 [4:14:55<26:51,  4.23s/it]
 90%|█████████ | 3620/4000 [4:14:59<26:56,  4.25s/it]
                                                     
{'loss': '0.3656', 'grad_norm': '0.1169', 'learning_rate': '5.773e-05', 'epoch': '0.8771'}

 90%|█████████ | 3620/4000 [4:14:59<26:56,  4.25s/it]
 91%|█████████ | 3621/4000 [4:15:03<26:45,  4.24s/it]
 91%|█████████ | 3622/4000 [4:15:08<26:56,  4.28s/it]
 91%|█████████ | 3623/4000 [4:15:12<27:07,  4.32s/it]
 91%|█████████ | 3624/4000 [4:15:16<26:43,  4.26s/it]
 91%|█████████ | 3625/4000 [4:15:21<27:21,  4.38s/it]
 91%|█████████ | 3626/4000 [4:15:25<26:50,  4.31s/it]
 91%|█████████ | 3627/4000 [4:15:29<26:41,  4.29s/it]
 91%|█████████ | 3628/4000 [4:15:34<26:22,  4.25s/it]
 91%|█████████ | 3629/4000 [4:15:38<26:05,  4.22s/it]
 91%|█████████ | 3630/4000 [4:15:42<25:52,  4.19s/it]
                                                     
{'loss': '0.3462', 'grad_norm': '0.1212', 'learning_rate': '5.621e-05', 'epoch': '0.8795'}

 91%|█████████ | 3630/4000 [4:15:42<25:52,  4.19s/it]
 91%|█████████ | 3631/4000 [4:15:46<25:43,  4.18s/it]
 91%|█████████ | 3632/4000 [4:15:51<26:23,  4.30s/it]
 91%|█████████ | 3633/4000 [4:15:55<26:00,  4.25s/it]
 91%|█████████ | 3634/4000 [4:15:59<25:43,  4.22s/it]
 91%|█████████ | 3635/4000 [4:16:03<25:28,  4.19s/it]
 91%|█████████ | 3636/4000 [4:16:07<25:17,  4.17s/it]
 91%|█████████ | 3637/4000 [4:16:11<25:18,  4.18s/it]
 91%|█████████ | 3638/4000 [4:16:16<26:02,  4.32s/it]
 91%|█████████ | 3639/4000 [4:16:20<26:00,  4.32s/it]
 91%|█████████ | 3640/4000 [4:16:25<26:03,  4.34s/it]
                                                     
{'loss': '0.3344', 'grad_norm': '0.1189', 'learning_rate': '5.47e-05', 'epoch': '0.882'}

 91%|█████████ | 3640/4000 [4:16:25<26:03,  4.34s/it]
 91%|█████████ | 3641/4000 [4:16:29<25:37,  4.28s/it]
 91%|█████████ | 3642/4000 [4:16:33<25:24,  4.26s/it]
 91%|█████████ | 3643/4000 [4:16:37<25:07,  4.22s/it]
 91%|█████████ | 3644/4000 [4:16:41<25:02,  4.22s/it]
 91%|█████████ | 3645/4000 [4:16:46<25:26,  4.30s/it]
 91%|█████████ | 3646/4000 [4:16:50<25:02,  4.24s/it]
 91%|█████████ | 3647/4000 [4:16:54<24:44,  4.21s/it]
 91%|█████████ | 3648/4000 [4:16:58<24:38,  4.20s/it]
 91%|█████████ | 3649/4000 [4:17:02<24:32,  4.19s/it]
 91%|█████████▏| 3650/4000 [4:17:07<24:19,  4.17s/it]
                                                     
{'loss': '0.3955', 'grad_norm': '0.1442', 'learning_rate': '5.318e-05', 'epoch': '0.8844'}

 91%|█████████▏| 3650/4000 [4:17:07<24:19,  4.17s/it]
 91%|█████████▏| 3651/4000 [4:17:11<24:10,  4.16s/it]
 91%|█████████▏| 3652/4000 [4:17:15<24:39,  4.25s/it]
 91%|█████████▏| 3653/4000 [4:17:19<24:21,  4.21s/it]
 91%|█████████▏| 3654/4000 [4:17:24<24:16,  4.21s/it]
 91%|█████████▏| 3655/4000 [4:17:28<24:23,  4.24s/it]
 91%|█████████▏| 3656/4000 [4:17:32<24:19,  4.24s/it]
 91%|█████████▏| 3657/4000 [4:17:36<24:21,  4.26s/it]
 91%|█████████▏| 3658/4000 [4:17:41<24:25,  4.28s/it]
 91%|█████████▏| 3659/4000 [4:17:45<24:32,  4.32s/it]
 92%|█████████▏| 3660/4000 [4:17:49<24:08,  4.26s/it]
                                                     
{'loss': '0.3687', 'grad_norm': '0.1402', 'learning_rate': '5.167e-05', 'epoch': '0.8868'}

 92%|█████████▏| 3660/4000 [4:17:49<24:08,  4.26s/it]
 92%|█████████▏| 3661/4000 [4:17:53<24:02,  4.26s/it]
 92%|█████████▏| 3662/4000 [4:17:58<23:47,  4.22s/it]
 92%|█████████▏| 3663/4000 [4:18:02<23:32,  4.19s/it]
 92%|█████████▏| 3664/4000 [4:18:06<23:22,  4.17s/it]
 92%|█████████▏| 3665/4000 [4:18:10<23:47,  4.26s/it]
 92%|█████████▏| 3666/4000 [4:18:15<23:34,  4.23s/it]
 92%|█████████▏| 3667/4000 [4:18:19<23:18,  4.20s/it]
 92%|█████████▏| 3668/4000 [4:18:23<23:06,  4.18s/it]
 92%|█████████▏| 3669/4000 [4:18:27<22:56,  4.16s/it]
 92%|█████████▏| 3670/4000 [4:18:31<22:48,  4.15s/it]
                                                     
{'loss': '0.363', 'grad_norm': '0.1105', 'learning_rate': '5.015e-05', 'epoch': '0.8892'}

 92%|█████████▏| 3670/4000 [4:18:31<22:48,  4.15s/it]
 92%|█████████▏| 3671/4000 [4:18:35<22:41,  4.14s/it]
 92%|█████████▏| 3672/4000 [4:18:40<23:35,  4.32s/it]
 92%|█████████▏| 3673/4000 [4:18:44<23:26,  4.30s/it]
 92%|█████████▏| 3674/4000 [4:18:48<23:17,  4.29s/it]
 92%|█████████▏| 3675/4000 [4:18:53<23:07,  4.27s/it]
 92%|█████████▏| 3676/4000 [4:18:57<22:56,  4.25s/it]
 92%|█████████▏| 3677/4000 [4:19:01<22:42,  4.22s/it]
 92%|█████████▏| 3678/4000 [4:19:05<22:38,  4.22s/it]
 92%|█████████▏| 3679/4000 [4:19:10<23:11,  4.33s/it]
 92%|█████████▏| 3680/4000 [4:19:14<22:47,  4.27s/it]
                                                     
{'loss': '0.3841', 'grad_norm': '0.1173', 'learning_rate': '4.864e-05', 'epoch': '0.8917'}

 92%|█████████▏| 3680/4000 [4:19:14<22:47,  4.27s/it]
 92%|█████████▏| 3681/4000 [4:19:18<22:28,  4.23s/it]
 92%|█████████▏| 3682/4000 [4:19:22<22:15,  4.20s/it]
 92%|█████████▏| 3683/4000 [4:19:26<22:10,  4.20s/it]
 92%|█████████▏| 3684/4000 [4:19:30<22:00,  4.18s/it]
 92%|█████████▏| 3685/4000 [4:19:35<22:08,  4.22s/it]
 92%|█████████▏| 3686/4000 [4:19:39<22:09,  4.24s/it]
 92%|█████████▏| 3687/4000 [4:19:43<21:55,  4.20s/it]
 92%|█████████▏| 3688/4000 [4:19:47<21:43,  4.18s/it]
 92%|█████████▏| 3689/4000 [4:19:52<21:51,  4.22s/it]
 92%|█████████▏| 3690/4000 [4:19:56<21:45,  4.21s/it]
                                                     
{'loss': '0.3604', 'grad_norm': '0.1114', 'learning_rate': '4.712e-05', 'epoch': '0.8941'}

 92%|█████████▏| 3690/4000 [4:19:56<21:45,  4.21s/it]
 92%|█████████▏| 3691/4000 [4:20:00<21:34,  4.19s/it]
 92%|█████████▏| 3692/4000 [4:20:04<22:01,  4.29s/it]
 92%|█████████▏| 3693/4000 [4:20:09<21:45,  4.25s/it]
 92%|█████████▏| 3694/4000 [4:20:13<21:35,  4.23s/it]
 92%|█████████▏| 3695/4000 [4:20:17<21:21,  4.20s/it]
 92%|█████████▏| 3696/4000 [4:20:21<21:20,  4.21s/it]
 92%|█████████▏| 3697/4000 [4:20:25<21:09,  4.19s/it]
 92%|█████████▏| 3698/4000 [4:20:29<20:58,  4.17s/it]
 92%|█████████▏| 3699/4000 [4:20:34<21:22,  4.26s/it]
 92%|█████████▎| 3700/4000 [4:20:38<21:16,  4.26s/it]
                                                     
{'loss': '0.3999', 'grad_norm': '0.138', 'learning_rate': '4.561e-05', 'epoch': '0.8965'}

 92%|█████████▎| 3700/4000 [4:20:38<21:16,  4.26s/it]
 93%|█████████▎| 3701/4000 [4:20:42<21:01,  4.22s/it]
 93%|█████████▎| 3702/4000 [4:20:46<20:48,  4.19s/it]
 93%|█████████▎| 3703/4000 [4:20:51<20:38,  4.17s/it]
 93%|█████████▎| 3704/4000 [4:20:55<20:29,  4.15s/it]
 93%|█████████▎| 3705/4000 [4:20:59<20:23,  4.15s/it]
 93%|█████████▎| 3706/4000 [4:21:04<21:15,  4.34s/it]
 93%|█████████▎| 3707/4000 [4:21:08<21:02,  4.31s/it]
 93%|█████████▎| 3708/4000 [4:21:12<20:47,  4.27s/it]
 93%|█████████▎| 3709/4000 [4:21:16<20:45,  4.28s/it]
 93%|█████████▎| 3710/4000 [4:21:20<20:29,  4.24s/it]
                                                     
{'loss': '0.361', 'grad_norm': '0.1257', 'learning_rate': '4.409e-05', 'epoch': '0.8989'}

 93%|█████████▎| 3710/4000 [4:21:20<20:29,  4.24s/it]
 93%|█████████▎| 3711/4000 [4:21:25<20:26,  4.24s/it]
 93%|█████████▎| 3712/4000 [4:21:29<20:28,  4.27s/it]
 93%|█████████▎| 3713/4000 [4:21:34<20:55,  4.38s/it]
 93%|█████████▎| 3714/4000 [4:21:38<20:31,  4.31s/it]
 93%|█████████▎| 3715/4000 [4:21:42<20:10,  4.25s/it]
 93%|█████████▎| 3716/4000 [4:21:46<19:55,  4.21s/it]
 93%|█████████▎| 3717/4000 [4:21:50<19:49,  4.20s/it]
 93%|█████████▎| 3718/4000 [4:21:54<19:39,  4.18s/it]
 93%|█████████▎| 3719/4000 [4:21:59<20:01,  4.28s/it]
 93%|█████████▎| 3720/4000 [4:22:03<19:44,  4.23s/it]
                                                     
{'loss': '0.3827', 'grad_norm': '0.1181', 'learning_rate': '4.258e-05', 'epoch': '0.9014'}

 93%|█████████▎| 3720/4000 [4:22:03<19:44,  4.23s/it]
 93%|█████████▎| 3721/4000 [4:22:07<19:31,  4.20s/it]
 93%|█████████▎| 3722/4000 [4:22:11<19:21,  4.18s/it]
 93%|█████████▎| 3723/4000 [4:22:16<19:35,  4.24s/it]
 93%|█████████▎| 3724/4000 [4:22:20<19:41,  4.28s/it]
 93%|█████████▎| 3725/4000 [4:22:24<19:27,  4.25s/it]
 93%|█████████▎| 3726/4000 [4:22:29<19:53,  4.36s/it]
 93%|█████████▎| 3727/4000 [4:22:33<19:34,  4.30s/it]
 93%|█████████▎| 3728/4000 [4:22:37<19:30,  4.30s/it]
 93%|█████████▎| 3729/4000 [4:22:41<19:11,  4.25s/it]
 93%|█████████▎| 3730/4000 [4:22:46<19:21,  4.30s/it]
                                                     
{'loss': '0.4119', 'grad_norm': '0.1231', 'learning_rate': '4.106e-05', 'epoch': '0.9038'}

 93%|█████████▎| 3730/4000 [4:22:46<19:21,  4.30s/it]
 93%|█████████▎| 3731/4000 [4:22:50<19:03,  4.25s/it]
 93%|█████████▎| 3732/4000 [4:22:54<18:50,  4.22s/it]
 93%|█████████▎| 3733/4000 [4:22:59<19:06,  4.29s/it]
 93%|█████████▎| 3734/4000 [4:23:03<18:53,  4.26s/it]
 93%|█████████▎| 3735/4000 [4:23:07<18:39,  4.23s/it]
 93%|█████████▎| 3736/4000 [4:23:11<18:27,  4.20s/it]
 93%|█████████▎| 3737/4000 [4:23:15<18:17,  4.17s/it]
 93%|█████████▎| 3738/4000 [4:23:19<18:10,  4.16s/it]
 93%|█████████▎| 3739/4000 [4:23:24<18:32,  4.26s/it]
 94%|█████████▎| 3740/4000 [4:23:28<18:35,  4.29s/it]
                                                     
{'loss': '0.3433', 'grad_norm': '0.1236', 'learning_rate': '3.955e-05', 'epoch': '0.9062'}

 94%|█████████▎| 3740/4000 [4:23:28<18:35,  4.29s/it]
 94%|█████████▎| 3741/4000 [4:23:32<18:35,  4.31s/it]
 94%|█████████▎| 3742/4000 [4:23:37<18:18,  4.26s/it]
 94%|█████████▎| 3743/4000 [4:23:41<18:15,  4.26s/it]
 94%|█████████▎| 3744/4000 [4:23:45<18:01,  4.22s/it]
 94%|█████████▎| 3745/4000 [4:23:49<18:04,  4.25s/it]
 94%|█████████▎| 3746/4000 [4:23:54<18:17,  4.32s/it]
 94%|█████████▎| 3747/4000 [4:23:58<18:23,  4.36s/it]
 94%|█████████▎| 3748/4000 [4:24:02<18:04,  4.31s/it]
 94%|█████████▎| 3749/4000 [4:24:07<17:47,  4.25s/it]
 94%|█████████▍| 3750/4000 [4:24:11<17:34,  4.22s/it]
                                                     
{'loss': '0.3984', 'grad_norm': '0.1077', 'learning_rate': '3.803e-05', 'epoch': '0.9086'}

 94%|█████████▍| 3750/4000 [4:24:11<17:34,  4.22s/it]
 94%|█████████▍| 3751/4000 [4:24:15<17:30,  4.22s/it]
 94%|█████████▍| 3752/4000 [4:24:19<17:24,  4.21s/it]
 94%|█████████▍| 3753/4000 [4:24:24<17:39,  4.29s/it]
 94%|█████████▍| 3754/4000 [4:24:28<17:24,  4.25s/it]
 94%|█████████▍| 3755/4000 [4:24:32<17:10,  4.21s/it]
 94%|█████████▍| 3756/4000 [4:24:36<17:00,  4.18s/it]
 94%|█████████▍| 3757/4000 [4:24:40<17:14,  4.26s/it]
 94%|█████████▍| 3758/4000 [4:24:45<17:18,  4.29s/it]
 94%|█████████▍| 3759/4000 [4:24:49<17:03,  4.24s/it]
 94%|█████████▍| 3760/4000 [4:24:54<17:30,  4.38s/it]
                                                     
{'loss': '0.3751', 'grad_norm': '0.1149', 'learning_rate': '3.652e-05', 'epoch': '0.911'}

 94%|█████████▍| 3760/4000 [4:24:54<17:30,  4.38s/it]
 94%|█████████▍| 3761/4000 [4:24:58<17:12,  4.32s/it]
 94%|█████████▍| 3762/4000 [4:25:02<17:03,  4.30s/it]
 94%|█████████▍| 3763/4000 [4:25:06<16:48,  4.25s/it]
 94%|█████████▍| 3764/4000 [4:25:10<16:47,  4.27s/it]
 94%|█████████▍| 3765/4000 [4:25:15<16:34,  4.23s/it]
 94%|█████████▍| 3766/4000 [4:25:19<16:36,  4.26s/it]
 94%|█████████▍| 3767/4000 [4:25:23<16:34,  4.27s/it]
 94%|█████████▍| 3768/4000 [4:25:27<16:22,  4.24s/it]
 94%|█████████▍| 3769/4000 [4:25:32<16:13,  4.21s/it]
 94%|█████████▍| 3770/4000 [4:25:36<16:03,  4.19s/it]
                                                     
{'loss': '0.34', 'grad_norm': '0.1119', 'learning_rate': '3.5e-05', 'epoch': '0.9135'}

 94%|█████████▍| 3770/4000 [4:25:36<16:03,  4.19s/it]
 94%|█████████▍| 3771/4000 [4:25:40<15:54,  4.17s/it]
 94%|█████████▍| 3772/4000 [4:25:44<15:46,  4.15s/it]
 94%|█████████▍| 3773/4000 [4:25:48<16:04,  4.25s/it]
 94%|█████████▍| 3774/4000 [4:25:53<16:09,  4.29s/it]
 94%|█████████▍| 3775/4000 [4:25:57<16:07,  4.30s/it]
 94%|█████████▍| 3776/4000 [4:26:01<15:52,  4.25s/it]
 94%|█████████▍| 3777/4000 [4:26:06<15:58,  4.30s/it]
 94%|█████████▍| 3778/4000 [4:26:10<15:44,  4.25s/it]
 94%|█████████▍| 3779/4000 [4:26:14<15:51,  4.30s/it]
 94%|█████████▍| 3780/4000 [4:26:19<15:58,  4.36s/it]
                                                     
{'loss': '0.3786', 'grad_norm': '0.122', 'learning_rate': '3.348e-05', 'epoch': '0.9159'}

 94%|█████████▍| 3780/4000 [4:26:19<15:58,  4.36s/it]
 95%|█████████▍| 3781/4000 [4:26:23<15:53,  4.36s/it]
 95%|█████████▍| 3782/4000 [4:26:27<15:35,  4.29s/it]
 95%|█████████▍| 3783/4000 [4:26:31<15:20,  4.24s/it]
 95%|█████████▍| 3784/4000 [4:26:35<15:08,  4.21s/it]
 95%|█████████▍| 3785/4000 [4:26:40<15:01,  4.19s/it]
 95%|█████████▍| 3786/4000 [4:26:44<15:08,  4.25s/it]
 95%|█████████▍| 3787/4000 [4:26:48<15:07,  4.26s/it]
 95%|█████████▍| 3788/4000 [4:26:52<14:53,  4.22s/it]
 95%|█████████▍| 3789/4000 [4:26:56<14:43,  4.19s/it]
 95%|█████████▍| 3790/4000 [4:27:01<14:36,  4.17s/it]
                                                     
{'loss': '0.3527', 'grad_norm': '0.1165', 'learning_rate': '3.197e-05', 'epoch': '0.9183'}

 95%|█████████▍| 3790/4000 [4:27:01<14:36,  4.17s/it]
 95%|█████████▍| 3791/4000 [4:27:05<14:36,  4.20s/it]
 95%|█████████▍| 3792/4000 [4:27:09<14:32,  4.19s/it]
 95%|█████████▍| 3793/4000 [4:27:14<14:46,  4.28s/it]
 95%|█████████▍| 3794/4000 [4:27:18<14:41,  4.28s/it]
 95%|█████████▍| 3795/4000 [4:27:22<14:28,  4.24s/it]
 95%|█████████▍| 3796/4000 [4:27:26<14:36,  4.30s/it]
 95%|█████████▍| 3797/4000 [4:27:31<14:21,  4.25s/it]
 95%|█████████▍| 3798/4000 [4:27:35<14:18,  4.25s/it]
 95%|█████████▍| 3799/4000 [4:27:39<14:13,  4.24s/it]
 95%|█████████▌| 3800/4000 [4:27:44<14:23,  4.32s/it]
                                                     
{'loss': '0.3989', 'grad_norm': '0.1184', 'learning_rate': '3.045e-05', 'epoch': '0.9207'}

 95%|█████████▌| 3800/4000 [4:27:44<14:23,  4.32s/it]
 95%|█████████▌| 3801/4000 [4:27:48<14:07,  4.26s/it]
 95%|█████████▌| 3802/4000 [4:27:52<13:56,  4.22s/it]
 95%|█████████▌| 3803/4000 [4:27:56<13:49,  4.21s/it]
 95%|█████████▌| 3804/4000 [4:28:00<13:40,  4.18s/it]
 95%|█████████▌| 3805/4000 [4:28:04<13:32,  4.17s/it]
 95%|█████████▌| 3806/4000 [4:28:08<13:26,  4.16s/it]
 95%|█████████▌| 3807/4000 [4:28:13<13:41,  4.25s/it]
 95%|█████████▌| 3808/4000 [4:28:17<13:44,  4.30s/it]
 95%|█████████▌| 3809/4000 [4:28:21<13:35,  4.27s/it]
 95%|█████████▌| 3810/4000 [4:28:26<13:23,  4.23s/it]
                                                     
{'loss': '0.3577', 'grad_norm': '0.1155', 'learning_rate': '2.894e-05', 'epoch': '0.9232'}

 95%|█████████▌| 3810/4000 [4:28:26<13:23,  4.23s/it]
 95%|█████████▌| 3811/4000 [4:28:30<13:29,  4.28s/it]
 95%|█████████▌| 3812/4000 [4:28:34<13:18,  4.25s/it]
 95%|█████████▌| 3813/4000 [4:28:39<13:35,  4.36s/it]
 95%|█████████▌| 3814/4000 [4:28:43<13:27,  4.34s/it]
 95%|█████████▌| 3815/4000 [4:28:47<13:18,  4.31s/it]
 95%|█████████▌| 3816/4000 [4:28:51<13:07,  4.28s/it]
 95%|█████████▌| 3817/4000 [4:28:56<12:54,  4.23s/it]
 95%|█████████▌| 3818/4000 [4:29:00<12:44,  4.20s/it]
 95%|█████████▌| 3819/4000 [4:29:04<12:35,  4.18s/it]
 96%|█████████▌| 3820/4000 [4:29:08<12:53,  4.30s/it]
                                                     
{'loss': '0.3699', 'grad_norm': '0.141', 'learning_rate': '2.742e-05', 'epoch': '0.9256'}

 96%|█████████▌| 3820/4000 [4:29:08<12:53,  4.30s/it]
 96%|█████████▌| 3821/4000 [4:29:13<12:41,  4.25s/it]
 96%|█████████▌| 3822/4000 [4:29:17<12:30,  4.22s/it]
 96%|█████████▌| 3823/4000 [4:29:21<12:21,  4.19s/it]
 96%|█████████▌| 3824/4000 [4:29:25<12:13,  4.17s/it]
 96%|█████████▌| 3825/4000 [4:29:29<12:12,  4.19s/it]
 96%|█████████▌| 3826/4000 [4:29:33<12:08,  4.18s/it]
 96%|█████████▌| 3827/4000 [4:29:38<12:18,  4.27s/it]
 96%|█████████▌| 3828/4000 [4:29:42<12:23,  4.32s/it]
 96%|█████████▌| 3829/4000 [4:29:46<12:10,  4.27s/it]
 96%|█████████▌| 3830/4000 [4:29:51<12:03,  4.26s/it]
                                                     
{'loss': '0.3517', 'grad_norm': '0.133', 'learning_rate': '2.591e-05', 'epoch': '0.928'}

 96%|█████████▌| 3830/4000 [4:29:51<12:03,  4.26s/it]
 96%|█████████▌| 3831/4000 [4:29:55<11:53,  4.22s/it]
 96%|█████████▌| 3832/4000 [4:29:59<11:45,  4.20s/it]
 96%|█████████▌| 3833/4000 [4:30:03<11:51,  4.26s/it]
 96%|█████████▌| 3834/4000 [4:30:08<11:58,  4.33s/it]
 96%|█████████▌| 3835/4000 [4:30:12<11:43,  4.26s/it]
 96%|█████████▌| 3836/4000 [4:30:16<11:34,  4.24s/it]
 96%|█████████▌| 3837/4000 [4:30:20<11:28,  4.22s/it]
 96%|█████████▌| 3838/4000 [4:30:24<11:19,  4.19s/it]
 96%|█████████▌| 3839/4000 [4:30:29<11:12,  4.17s/it]
 96%|█████████▌| 3840/4000 [4:30:33<11:14,  4.21s/it]
                                                     
{'loss': '0.3491', 'grad_norm': '0.128', 'learning_rate': '2.439e-05', 'epoch': '0.9304'}

 96%|█████████▌| 3840/4000 [4:30:33<11:14,  4.21s/it]
 96%|█████████▌| 3841/4000 [4:30:37<11:14,  4.24s/it]
 96%|█████████▌| 3842/4000 [4:30:42<11:19,  4.30s/it]
 96%|█████████▌| 3843/4000 [4:30:46<11:11,  4.28s/it]
 96%|█████████▌| 3844/4000 [4:30:50<11:02,  4.25s/it]
 96%|█████████▌| 3845/4000 [4:30:54<11:04,  4.29s/it]
 96%|█████████▌| 3846/4000 [4:30:59<10:54,  4.25s/it]
 96%|█████████▌| 3847/4000 [4:31:03<11:08,  4.37s/it]
 96%|█████████▌| 3848/4000 [4:31:07<10:54,  4.31s/it]
 96%|█████████▌| 3849/4000 [4:31:12<10:43,  4.26s/it]
 96%|█████████▋| 3850/4000 [4:31:16<10:40,  4.27s/it]
                                                     
{'loss': '0.3858', 'grad_norm': '0.1295', 'learning_rate': '2.288e-05', 'epoch': '0.9329'}

 96%|█████████▋| 3850/4000 [4:31:16<10:40,  4.27s/it]
 96%|█████████▋| 3851/4000 [4:31:20<10:29,  4.23s/it]
 96%|█████████▋| 3852/4000 [4:31:24<10:21,  4.20s/it]
 96%|█████████▋| 3853/4000 [4:31:28<10:14,  4.18s/it]
 96%|█████████▋| 3854/4000 [4:31:33<10:34,  4.35s/it]
 96%|█████████▋| 3855/4000 [4:31:37<10:20,  4.28s/it]
 96%|█████████▋| 3856/4000 [4:31:41<10:09,  4.24s/it]
 96%|█████████▋| 3857/4000 [4:31:45<10:01,  4.21s/it]
 96%|█████████▋| 3858/4000 [4:31:49<09:53,  4.18s/it]
 96%|█████████▋| 3859/4000 [4:31:54<09:55,  4.22s/it]
 96%|█████████▋| 3860/4000 [4:31:58<09:49,  4.21s/it]
                                                     
{'loss': '0.3705', 'grad_norm': '0.1089', 'learning_rate': '2.136e-05', 'epoch': '0.9353'}

 96%|█████████▋| 3860/4000 [4:31:58<09:49,  4.21s/it]
 97%|█████████▋| 3861/4000 [4:32:03<09:59,  4.31s/it]
 97%|█████████▋| 3862/4000 [4:32:07<10:00,  4.35s/it]
 97%|█████████▋| 3863/4000 [4:32:11<09:48,  4.29s/it]
 97%|█████████▋| 3864/4000 [4:32:15<09:42,  4.28s/it]
 97%|█████████▋| 3865/4000 [4:32:20<09:32,  4.24s/it]
 97%|█████████▋| 3866/4000 [4:32:24<09:24,  4.22s/it]
 97%|█████████▋| 3867/4000 [4:32:28<09:37,  4.34s/it]
 97%|█████████▋| 3868/4000 [4:32:33<09:30,  4.33s/it]
 97%|█████████▋| 3869/4000 [4:32:37<09:19,  4.27s/it]
 97%|█████████▋| 3870/4000 [4:32:41<09:09,  4.23s/it]
                                                     
{'loss': '0.3926', 'grad_norm': '0.1193', 'learning_rate': '1.985e-05', 'epoch': '0.9377'}

 97%|█████████▋| 3870/4000 [4:32:41<09:09,  4.23s/it]
 97%|█████████▋| 3871/4000 [4:32:45<09:05,  4.23s/it]
 97%|█████████▋| 3872/4000 [4:32:49<08:57,  4.20s/it]
 97%|█████████▋| 3873/4000 [4:32:53<08:50,  4.18s/it]
 97%|█████████▋| 3874/4000 [4:32:58<08:58,  4.27s/it]
 97%|█████████▋| 3875/4000 [4:33:02<08:48,  4.23s/it]
 97%|█████████▋| 3876/4000 [4:33:06<08:45,  4.24s/it]
 97%|█████████▋| 3877/4000 [4:33:10<08:39,  4.22s/it]
 97%|█████████▋| 3878/4000 [4:33:15<08:35,  4.23s/it]
 97%|█████████▋| 3879/4000 [4:33:19<08:36,  4.27s/it]
 97%|█████████▋| 3880/4000 [4:33:23<08:28,  4.24s/it]
                                                     
{'loss': '0.364', 'grad_norm': '0.1004', 'learning_rate': '1.833e-05', 'epoch': '0.9401'}

 97%|█████████▋| 3880/4000 [4:33:23<08:28,  4.24s/it]
 97%|█████████▋| 3881/4000 [4:33:28<08:40,  4.38s/it]
 97%|█████████▋| 3882/4000 [4:33:32<08:28,  4.31s/it]
 97%|█████████▋| 3883/4000 [4:33:36<08:21,  4.29s/it]
 97%|█████████▋| 3884/4000 [4:33:41<08:18,  4.29s/it]
 97%|█████████▋| 3885/4000 [4:33:45<08:12,  4.28s/it]
 97%|█████████▋| 3886/4000 [4:33:49<08:02,  4.23s/it]
 97%|█████████▋| 3887/4000 [4:33:53<07:54,  4.20s/it]
 97%|█████████▋| 3888/4000 [4:33:58<08:04,  4.32s/it]
 97%|█████████▋| 3889/4000 [4:34:02<07:53,  4.27s/it]
 97%|█████████▋| 3890/4000 [4:34:06<07:44,  4.23s/it]
                                                     
{'loss': '0.3782', 'grad_norm': '0.1145', 'learning_rate': '1.682e-05', 'epoch': '0.9425'}

 97%|█████████▋| 3890/4000 [4:34:06<07:44,  4.23s/it]
 97%|█████████▋| 3891/4000 [4:34:10<07:37,  4.20s/it]
 97%|█████████▋| 3892/4000 [4:34:14<07:31,  4.18s/it]
 97%|█████████▋| 3893/4000 [4:34:19<07:32,  4.23s/it]
 97%|█████████▋| 3894/4000 [4:34:23<07:41,  4.35s/it]
 97%|█████████▋| 3895/4000 [4:34:27<07:31,  4.30s/it]
 97%|█████████▋| 3896/4000 [4:34:32<07:29,  4.32s/it]
 97%|█████████▋| 3897/4000 [4:34:36<07:19,  4.26s/it]
 97%|█████████▋| 3898/4000 [4:34:40<07:18,  4.30s/it]
 97%|█████████▋| 3899/4000 [4:34:44<07:09,  4.25s/it]
 98%|█████████▊| 3900/4000 [4:34:49<07:04,  4.24s/it]
                                                     
{'loss': '0.3661', 'grad_norm': '0.1121', 'learning_rate': '1.53e-05', 'epoch': '0.945'}

 98%|█████████▊| 3900/4000 [4:34:49<07:04,  4.24s/it]
 98%|█████████▊| 3901/4000 [4:34:53<07:16,  4.41s/it]
 98%|█████████▊| 3902/4000 [4:34:58<07:04,  4.33s/it]
 98%|█████████▊| 3903/4000 [4:35:02<06:53,  4.27s/it]
 98%|█████████▊| 3904/4000 [4:35:06<06:48,  4.26s/it]
 98%|█████████▊| 3905/4000 [4:35:10<06:40,  4.21s/it]
 98%|█████████▊| 3906/4000 [4:35:14<06:32,  4.18s/it]
 98%|█████████▊| 3907/4000 [4:35:18<06:29,  4.19s/it]
 98%|█████████▊| 3908/4000 [4:35:23<06:34,  4.28s/it]
 98%|█████████▊| 3909/4000 [4:35:27<06:25,  4.23s/it]
 98%|█████████▊| 3910/4000 [4:35:31<06:17,  4.19s/it]
                                                     
{'loss': '0.3507', 'grad_norm': '0.1011', 'learning_rate': '1.379e-05', 'epoch': '0.9474'}

 98%|█████████▊| 3910/4000 [4:35:31<06:17,  4.19s/it]
 98%|█████████▊| 3911/4000 [4:35:35<06:10,  4.16s/it]
 98%|█████████▊| 3912/4000 [4:35:39<06:04,  4.14s/it]
 98%|█████████▊| 3913/4000 [4:35:43<05:59,  4.13s/it]
 98%|█████████▊| 3914/4000 [4:35:48<05:59,  4.18s/it]
 98%|█████████▊| 3915/4000 [4:35:52<05:57,  4.20s/it]
 98%|█████████▊| 3916/4000 [4:35:56<05:50,  4.17s/it]
 98%|█████████▊| 3917/4000 [4:36:00<05:44,  4.15s/it]
 98%|█████████▊| 3918/4000 [4:36:04<05:38,  4.13s/it]
 98%|█████████▊| 3919/4000 [4:36:08<05:33,  4.12s/it]
 98%|█████████▊| 3920/4000 [4:36:12<05:29,  4.12s/it]
                                                     
{'loss': '0.3492', 'grad_norm': '0.1289', 'learning_rate': '1.227e-05', 'epoch': '0.9498'}

 98%|█████████▊| 3920/4000 [4:36:12<05:29,  4.12s/it]
 98%|█████████▊| 3921/4000 [4:36:17<05:33,  4.22s/it]
 98%|█████████▊| 3922/4000 [4:36:21<05:26,  4.18s/it]
 98%|█████████▊| 3923/4000 [4:36:25<05:20,  4.16s/it]
 98%|█████████▊| 3924/4000 [4:36:29<05:14,  4.14s/it]
 98%|█████████▊| 3925/4000 [4:36:33<05:09,  4.13s/it]
 98%|█████████▊| 3926/4000 [4:36:37<05:04,  4.12s/it]
 98%|█████████▊| 3927/4000 [4:36:41<05:00,  4.11s/it]
 98%|█████████▊| 3928/4000 [4:36:46<05:02,  4.20s/it]
 98%|█████████▊| 3929/4000 [4:36:50<04:57,  4.19s/it]
 98%|█████████▊| 3930/4000 [4:36:54<04:52,  4.18s/it]
                                                     
{'loss': '0.3655', 'grad_norm': '0.1154', 'learning_rate': '1.076e-05', 'epoch': '0.9522'}

 98%|█████████▊| 3930/4000 [4:36:54<04:52,  4.18s/it]
 98%|█████████▊| 3931/4000 [4:36:58<04:46,  4.16s/it]
 98%|█████████▊| 3932/4000 [4:37:02<04:41,  4.14s/it]
 98%|█████████▊| 3933/4000 [4:37:07<04:36,  4.13s/it]
 98%|█████████▊| 3934/4000 [4:37:11<04:32,  4.13s/it]
 98%|█████████▊| 3935/4000 [4:37:15<04:34,  4.22s/it]
 98%|█████████▊| 3936/4000 [4:37:19<04:27,  4.19s/it]
 98%|█████████▊| 3937/4000 [4:37:23<04:22,  4.16s/it]
 98%|█████████▊| 3938/4000 [4:37:27<04:17,  4.15s/it]
 98%|█████████▊| 3939/4000 [4:37:32<04:12,  4.14s/it]
 98%|█████████▊| 3940/4000 [4:37:36<04:07,  4.13s/it]
                                                     
{'loss': '0.353', 'grad_norm': '0.1089', 'learning_rate': '9.242e-06', 'epoch': '0.9547'}

 98%|█████████▊| 3940/4000 [4:37:36<04:07,  4.13s/it]
 99%|█████████▊| 3941/4000 [4:37:40<04:06,  4.18s/it]
 99%|█████████▊| 3942/4000 [4:37:44<04:03,  4.20s/it]
 99%|█████████▊| 3943/4000 [4:37:48<03:57,  4.17s/it]
 99%|█████████▊| 3944/4000 [4:37:52<03:52,  4.15s/it]
 99%|█████████▊| 3945/4000 [4:37:56<03:47,  4.13s/it]
 99%|█████████▊| 3946/4000 [4:38:01<03:42,  4.12s/it]
 99%|█████████▊| 3947/4000 [4:38:05<03:37,  4.11s/it]
 99%|█████████▊| 3948/4000 [4:38:09<03:38,  4.21s/it]
 99%|█████████▊| 3949/4000 [4:38:13<03:32,  4.18s/it]
 99%|█████████▉| 3950/4000 [4:38:17<03:27,  4.15s/it]
                                                     
{'loss': '0.3895', 'grad_norm': '0.132', 'learning_rate': '7.727e-06', 'epoch': '0.9571'}

 99%|█████████▉| 3950/4000 [4:38:17<03:27,  4.15s/it]
 99%|█████████▉| 3951/4000 [4:38:21<03:22,  4.13s/it]
 99%|█████████▉| 3952/4000 [4:38:25<03:17,  4.12s/it]
 99%|█████████▉| 3953/4000 [4:38:30<03:13,  4.11s/it]
 99%|█████████▉| 3954/4000 [4:38:34<03:08,  4.10s/it]
 99%|█████████▉| 3955/4000 [4:38:38<03:09,  4.20s/it]
 99%|█████████▉| 3956/4000 [4:38:42<03:03,  4.17s/it]
 99%|█████████▉| 3957/4000 [4:38:46<02:58,  4.15s/it]
 99%|█████████▉| 3958/4000 [4:38:50<02:53,  4.14s/it]
 99%|█████████▉| 3959/4000 [4:38:54<02:48,  4.12s/it]
 99%|█████████▉| 3960/4000 [4:38:59<02:44,  4.11s/it]
                                                     
{'loss': '0.4014', 'grad_norm': '0.1627', 'learning_rate': '6.212e-06', 'epoch': '0.9595'}

 99%|█████████▉| 3960/4000 [4:38:59<02:44,  4.11s/it]
 99%|█████████▉| 3961/4000 [4:39:03<02:40,  4.10s/it]
 99%|█████████▉| 3962/4000 [4:39:07<02:39,  4.19s/it]
 99%|█████████▉| 3963/4000 [4:39:11<02:33,  4.16s/it]
 99%|█████████▉| 3964/4000 [4:39:15<02:29,  4.14s/it]
 99%|█████████▉| 3965/4000 [4:39:19<02:24,  4.13s/it]
 99%|█████████▉| 3966/4000 [4:39:23<02:20,  4.12s/it]
 99%|█████████▉| 3967/4000 [4:39:28<02:15,  4.11s/it]
 99%|█████████▉| 3968/4000 [4:39:32<02:13,  4.17s/it]
 99%|█████████▉| 3969/4000 [4:39:36<02:10,  4.20s/it]
 99%|█████████▉| 3970/4000 [4:39:40<02:05,  4.17s/it]
                                                     
{'loss': '0.3607', 'grad_norm': '0.1194', 'learning_rate': '4.697e-06', 'epoch': '0.9619'}

 99%|█████████▉| 3970/4000 [4:39:40<02:05,  4.17s/it]
 99%|█████████▉| 3971/4000 [4:39:44<02:00,  4.15s/it]
 99%|█████████▉| 3972/4000 [4:39:49<02:00,  4.31s/it]
 99%|█████████▉| 3973/4000 [4:39:53<01:54,  4.25s/it]
 99%|█████████▉| 3974/4000 [4:39:57<01:49,  4.20s/it]
 99%|█████████▉| 3975/4000 [4:40:02<01:51,  4.45s/it]
 99%|█████████▉| 3976/4000 [4:40:06<01:44,  4.35s/it]
 99%|█████████▉| 3977/4000 [4:40:10<01:38,  4.27s/it]
 99%|█████████▉| 3978/4000 [4:40:14<01:32,  4.22s/it]
 99%|█████████▉| 3979/4000 [4:40:19<01:27,  4.18s/it]
100%|█████████▉| 3980/4000 [4:40:23<01:23,  4.16s/it]
                                                     
{'loss': '0.3656', 'grad_norm': '0.1271', 'learning_rate': '3.182e-06', 'epoch': '0.9644'}

100%|█████████▉| 3980/4000 [4:40:23<01:23,  4.16s/it]
100%|█████████▉| 3981/4000 [4:40:27<01:18,  4.14s/it]
100%|█████████▉| 3982/4000 [4:40:31<01:16,  4.24s/it]
100%|█████████▉| 3983/4000 [4:40:35<01:11,  4.20s/it]
100%|█████████▉| 3984/4000 [4:40:39<01:06,  4.16s/it]
100%|█████████▉| 3985/4000 [4:40:44<01:02,  4.14s/it]
100%|█████████▉| 3986/4000 [4:40:48<00:57,  4.13s/it]
100%|█████████▉| 3987/4000 [4:40:52<00:53,  4.12s/it]
100%|█████████▉| 3988/4000 [4:40:56<00:49,  4.11s/it]
100%|█████████▉| 3989/4000 [4:41:00<00:46,  4.22s/it]
100%|█████████▉| 3990/4000 [4:41:04<00:41,  4.18s/it]
                                                     
{'loss': '0.3475', 'grad_norm': '0.1199', 'learning_rate': '1.667e-06', 'epoch': '0.9668'}

100%|█████████▉| 3990/4000 [4:41:04<00:41,  4.18s/it]
100%|█████████▉| 3991/4000 [4:41:08<00:37,  4.15s/it]
100%|█████████▉| 3992/4000 [4:41:13<00:33,  4.14s/it]
100%|█████████▉| 3993/4000 [4:41:17<00:28,  4.12s/it]
100%|█████████▉| 3994/4000 [4:41:21<00:24,  4.11s/it]
100%|█████████▉| 3995/4000 [4:41:25<00:20,  4.15s/it]
100%|█████████▉| 3996/4000 [4:41:29<00:16,  4.19s/it]
100%|█████████▉| 3997/4000 [4:41:33<00:12,  4.16s/it]
100%|█████████▉| 3998/4000 [4:41:37<00:08,  4.14s/it]
100%|█████████▉| 3999/4000 [4:41:42<00:04,  4.12s/it]
100%|██████████| 4000/4000 [4:41:46<00:00,  4.11s/it]
                                                     
{'loss': '0.3576', 'grad_norm': '0.1201', 'learning_rate': '1.515e-07', 'epoch': '0.9692'}

100%|██████████| 4000/4000 [4:41:46<00:00,  4.11s/it]

  0%|          | 0/2 [00:00<?, ?it/s]

100%|██████████| 2/2 [00:00<00:00,  4.23it/s]
                                                     

                                             
{'eval_loss': '2.231', 'eval_runtime': '1.011', 'eval_samples_per_second': '21.76', 'eval_steps_per_second': '1.978', 'epoch': '0.9692'}

100%|██████████| 4000/4000 [4:41:47<00:00,  4.11s/it]

100%|██████████| 2/2 [00:00<00:00,  4.23it/s]

                                             [transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
  - If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
  - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
  - If you are not the owner of the model architecture class, please contact the model code owner to update it.


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:00<00:00,  3.88it/s]
Writing model shards: 100%|██████████| 1/1 [00:00<00:00,  3.87it/s]

                                                     
{'train_runtime': '1.691e+04', 'train_samples_per_second': '7.569', 'train_steps_per_second': '0.237', 'train_loss': '0.6708', 'epoch': '0.9692'}

100%|██████████| 4000/4000 [4:41:50<00:00,  4.11s/it]
100%|██████████| 4000/4000 [4:41:50<00:00,  4.23s/it]
[transformers] LlamaForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
  - If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
  - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
  - If you are not the owner of the model architecture class, please contact the model code owner to update it.

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]
[Done] Model saved to outputs/devops-300m-4096-bf16-vast

Writing model shards: 100%|██████████| 1/1 [00:00<00:00,  3.80it/s]
Writing model shards: 100%|██████████| 1/1 [00:00<00:00,  3.80it/s]

[Done] Model saved to outputs/devops-300m-4096-bf16-vast
81d666a3c511:10693:10966 [1] NCCL INFO [Service thread] Connection closed by localRank 1
[rank0]:[W914 21:26:49.450832496 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present,  but this warning has only been added since PyTorch 2.4 (function operator())
81d666a3c511:10692:10965 [0] NCCL INFO [Service thread] Connection closed by localRank 0
81d666a3c511:10693:15114 [1] NCCL INFO comm 0x1aa33b30 rank 1 nranks 2 cudaDev 1 busId d6000 - Abort COMPLETE
81d666a3c511:10692:15115 [0] NCCL INFO comm 0x1adc3b90 rank 0 nranks 2 cudaDev 0 busId d2000 - Abort COMPLETE
[2026-09-14 21:26:51,207] [INFO] [launch.py:367:main] Process 10692 exits successfully.
[2026-09-14 21:26:51,208] [INFO] [launch.py:367:main] Process 10693 exits successfully.
=== full run done Mon Sep 14 21:26:53 UTC 2026 exit=0 ===